PLease help me with filling the gape after doing simple subtraction of these images

I am trying to do background subtraction,by simply subtracting two image. However the resultant image has some gape. Can anyone help me solve this. I would appreciate if someone can help. Attached are the images

Answer 1

[Image Analyst]

You need to lower your threshold if you want to capture the man's tan coat over a tan walkway. Or else look at optical flow.

[BlueBee77]

Can you please explain with a few line of code how to lower the threshold, as i am not using any threshold. its simple subtraction of two images.

[BlueBee77]

Kindly help me, I would really appreciate if i can get a clue to move on.

[Walter Roberson]

If you were doing a simple subtraction of two images then the result would have to be color since the original images were color.

[Image Analyst]

There are many examples here http://www.mathworks.com/products/computer-vision/code-examples.html that could be adapted to do what you're asking.

[BlueBee77]

I tried so many different techniques but after no possible solutions i asked here.

[Image Analyst]

If you can't write one, then maybe you should consider purchasing a system to do your surveillance, or hire someone or some organization to develop the code for you. You've tried all the Mathworks-suggested solutions, and none worked, so it's time to look elsewhere for a solution that meets your needs.

[BlueBee77]

Ofcourse, i can write one and i have written one(i am not experienced as you ofcourse but i know how to code), but the results are not what i want. Thanks for your suggestion about hiring but I know the value of money and i would prefer to work more harder than to hire someone.

[Image Analyst]

I'm surprised a low threshold with optical flow didn't work, or that Guassian mixed models or simple subtraction didn't work. But you promised you tried all those methods plus more and they all failed.

So you'll have to scout out the literature. Start here http://www.visionbib.com/bibliography/contents.html to look for possible other methods not covered by the Computer Vision System Toolbox.

See this http://www.mee.tcd.ie/~sigmedia/Research/RigRemoval

[BlueBee77]

Thanks for sharing. I will go through these.

Answer 2

[Walter Roberson]

FirstImage = imread('BackgroundInt.png');
SecondImage = imread('BackInt4.png');

   diff_image = double(SecondImage) - double(FirstImage);
   sdiff_image = diff_image - min(diff_image(:));
   sdiff_image = sdiff_image ./ max(sdiff_image(:));

   pixels_changed = double( any(diff_image ~= 0, 3) );
   image(sdiff_image, 'AlphaData', pixels_changed);

   set(gca, 'color', [0 .5 0])

The green parts that show up will be the pixels with no change. You will see a lot of gray areas: those are areas where at least one of the pixel components changed a little. The person will have clearer outlines, including blue pants.

In order to get any further you would need to use a threshold to define changes as being too small to be meaningful. For example you could substitute

pixels_changed = double( any(abs(diff_image) >5, 3) );

Also, you can do

mask = repmat(logical(pixels_changed), 1, 1, 3);

FIm = FirstImage * 0;    %same size and datatype
FIm(mask) = FirstImage(mask);
subplot(1,2,1);
image(FIm);
axis image
SIm = SecondImage * 0;   %same size and datatype
SIm(mask) = SecondImage(mask);
subplot(1,2,2);
image(SIm);
axis image

this will pick out the pixels that have changed and show the "before" and "after" content of those pixels. You might want to experiment with

mask = logical(pixels_changed); mask = bwareafilt(mask,[10 inf]); mask = repmat(mask,1,1,3);

[BlueBee77]

Thanks alot for your response. I tried it but the problem is that when i convert the image to binary, i get the same result that i posted above. I want to obtain complete silhouette of human.

[Walter Roberson]

How can you convert the image to binary without using a threshold?

[BlueBee77]

I am using otsu's thresholding in matlab

[Walter Roberson]

I notice that the posted files are PNG files, but the pattern of mismatches that I see is something that I would associate more with JPEG files. Did the images go through a JPEG stage? If so then is it possible to avoid that stage and only use lossless image formats such as PNG or TIFF, including what you get from the camera itself?

[BlueBee77]

These images are actually extracted from a video using MPEG Streamclip. One thing more, how can i visualize the mask that you created in your above code??

[BlueBee77]

If i continue with such images with a gap, the problem is that in blob analysis that are considered as different blobs and i am having problem in how to connect these blobs that belong to oneperson

[Image Analyst]

Perhaps if they're within a certain distance of each other and they track/translate in synchrony with each other, then they're part of the same moving object. The easy tracking stuff is easy but you'll spend way way more code on handling these kinds of pathological situations like a man in a tan suit walking over a tan walkway, or people who align/overlap to produce one object and then suddenly turn around and way back in the direction they came, or a person getting in a car and then suddenly "disappearing", or two people in red shirts and blue jeans passing in front of each other, etc. Those kinds of bizarre situations always take a lot of special ad-hoc coding.

[BlueBee77]

Yes, you re right. I can ignore this image but the same happens with other images when the skin is exposed or a person has a bald head e.g., image attached. For situation like this i cannot get a perfect boundary or mask. Do you have any idea what should i do for it.