Is any way to Inpaint this image?

[Walter Roberson]

inpaint what part of it, and with what values?

[JAI PRAKASH]

I want to fill the all black areas with some interpolation or extrapolation. Thanks

[Walter Roberson]

"all" black areas? So you want the window area to be filled with advertisements and curtains, and the lampposts and hydro posts should get posters and "Lost Cat" notices all over them, and the tree should be changed to a birch tree (white bark) or perhaps Amur Cherry (a golden color bark) ?

[John D'Errico]

Therein lies the problem, and why I said that this needs to be done by hand, essentially by an artist, one able to provide intelligent inferences about the scene. The windows are pure black. So they might have the outline of curtains in them. Or perhaps a faint signature of window blinds. Maybe the outline of someone standing there, cup of coffee in hand as they contemplatively look outside.

You cannot use the region around the black hole of a window to infer what lies inside. Else that would turn one window with a brick surround into brick itself. And even that has issues, because texture inpainting is considerably more difficult than a simplistic inpainting algorithm, that merely blends the colors of the surround into a hole.

The front faces (risers) of what looks like steps should probably be inpainted with a different color than the top faces, the treads of the steps. Otherwise, steps would disappear, just appearing as mono-color ramps. And while you know that this makes sense, because they are lit in different ways from the sky, a computer inpainting algorithm will not understand that unless it can take apart the scene, representing it as such.

As far as extrapolation down into that black area, computers are notoriously poor at extrapolation. In fact, you really don't want to extrapolate, because even linear extrapolation would always generate strange looking colors that simply do not belong in that image.

The point of all this is, you cannot just use interpolative inpainting that is based purely on the boundaries of a "hole" to infer what lies inside a large blank region in an image. You certainly cannot use it to extrapolate into corners. The result will be garbage, nothing like a scene from any reality. Someone inpainting such large areas of an image needs to understand the scene, in a 3-dimensional way. You need to understand that one black hole is a window, instead of a tree trunk, instead of steps (properly lit), instead of the shadow cast from a pole. You need to make inferences of where the light comes from, based on those shadows seen in the image.

So as I suggested, open an image editing tool, and brush away. But don't expect MATLAB to do this task.

Answer 1

[JAI PRAKASH]

[Walter Roberson]

Note: you are not testing for black, you are testing for pixels with little red.

[JAI PRAKASH]

please note that the pixels which are little red, those are little in blue and little in green also. or otherwise u can create separate masks for blue and green also. And add them in the for loop. I choose red mask only considering performance enhancement. Thanks

[Walter Roberson]

The exception is associate with the left lights on the blue car, and some of the windows on the building a little to the left of the blue car.

Answer 2

[John D'Errico]

I won't touch this, and I've written the main inpainting tool on the File Exchange. Do NOT use that tool I've posted, as you will not be happy with the result. It simply is not designed to solve this class of problem, so you would be wasting your time, and then mine when you ask why it did not work.

Best is simply to use a tool like Photoshop, something that allows you to use your own brain to provide the information necessary. At that point, you will find that such large scale inpainting can be a subtle art in itself, forcing you to create content that is not available from the image itself.

This simply is NOT a problem that can be automated, at least if you hope to get something you will like.