Accurate stereo reconstruction of high resolution image

I am doing stereo calibration and scene reconstruction. The disparity map I obtain is much better than what I was initially getting. Because my disparity is high, I have set:

DisparityRange = [192,352]

I will not be able to decrease the disparity (it's around 300 pixels when manually calculated using rectified images).

I have tried to decrease disparity by moving the object away. Disparity does decrease, so does accuracy. I am using a 10MP image sensor so I can get high accuracy. The baseline is 70mm. With this baseline, I want to get a reconstructed object as accurate as possible.

My disparity map is shown in the figure below:

Using this, I am getting gaps in my reconstructed object.

I want to remove these gaps from reconstructed scene. This is what I've tried:

 point_cloud_smooth =  smooth3(point_cloud,'gaussian');

The output of this plot seemed to be stretched, and not interpolated.

Are there any other ways by which I can get a better and accurate Reconstructed scene?

Mean Reprojection error after calibrating my setup is 0.3261.

Do you have any inputs?

I have uploaded here stereo parameters and a pair of stereo images (from Camera A and B respectively). I have also attached the point cloud of object (one which is filtered, i.e. points with high variance in z values are NaN).

Answer 1

[Dima Lisin]

Hi Meghana,

I am glad that you are getting better results! There are a couple of things you can try to fill in the holes in your disparity map:

• Increase the block size
• Post-process the disparity map by applying a median filter

Good luck!

[Meghana Dinesh]

Hi Dima,

Thank you for your continued support.

I did increase the disparity block size (from 15 initially, to 25). I am already doing the below mentioned steps (on rectified images) before finding disparity:

 I1_r = adapthisteq(I1_r); % done on both left and right images.
 J1 = imgaussfilt(I1_r, 40);

These are my input parameters to find disparity:

 disparityMap = disparity(J1, J2, 'DisparityRange', [192,352],...
        'BlockSize', 25, 'Method','SemiGlobal',...
        'ContrastThreshold' , 0.3);

I have posted a screen shot of the obtained disparity map below:

Should I change any other parameters like DistanceThreshold?

I seem to be getting the same issue for almost all my images. Disparity for the top and bottom part of object is not properly detected, hence stereo reconstruction isn't good enough.

Also, what is the difference between the minimum value mentioned in DisparityRange and DistanceThreshold?

Another point I have noticed is that there isn't much reflection on only the top and bottom of my rim in the images. So I am ruling out the possibility of this issue cropping up due to reflection of light. I have uploaded a stereo pair (for which I have calculated disparity in the screenshots above), could you please reconfirm that it is not because of light reflection?

[Dima Lisin]

Hi Meghana,

The DistanceThreshold parameter gives you a way to verify consistency of the matches. If you set it, it will make the function match each pixel from left image to the right image, and then back from right image into the left image. The DistanceThreshold is the limit on how far off you can be on your backward match. So this is probably not going to help you here.

Try other values for the BlockSize. Evidently, 25 makes things worse. Try other values less then 25. And definitely try applying the median filter to the resulting disparity map using medfilt2.

Answer 2

[xikang]

This is what I get with blocksize=15. I think the gaussian filter should not be added before the disparity function. It will make the disparity map worse. You may add any filter on the disparity Map image though.

[Meghana Dinesh]

Thank you xikang. I will do these changes and see if I am getting better results for all my images.