how to totally match two images from heterogeneous stereo cameras.

I set up a stereo cameras, one for color image and another for UV image. Due to the difficultiness to find lenses with same intrinsic parameters for both cameras.

I used stereo camera calibration app and rectified both images. But checkerboard didn't totally match. It seems vertical grid was almost matched but horizontal grid was

not matched(left image). For the next step, I translated UV image(magenta) using translation parameter obtained from the stereo calibration. It's been partially successful(right image).

Here're my questions.

1) How does rectify fuction works? why does it seems to match images' vertial grid?

2) Is it correct procedure to rectify images; stereo calibration -> rectify both images -> translate?

3) When I use 'imtranslate' function, how can I estmate the pixel translation? Looking up reprojectedpoints data does't seem to be easy and clear way.

Thank you for your answer in advance.

Answer 1

[Sai Bhargav Avula]

Rectification of an Image is the transformation used for projecting the images onto common image plane. This would make the epipolar lines lie parallel to horizontal axis. Also, the corresponding points have same vertical coordinates. The same is done by MATLAB rectification function rectifyStereoImages. There is no need of translating the image(imtranslate) after rectification. By looking at your image it looks like the reprojection error is high meaning there is something wrong in your camera calibration. You may try recalibrating the stereo camera and check for the reprojection and epipolar errors

Hope this helps !

[Jungkeuk Park]

Thank you. Your message really helped. I checked the stereoParameters and translation of camera 2 was totally wrong like distance from camera 1 was way too high. I know I should do calibration again but I don't know what to change to improve calibration results.

[Sai Bhargav Avula]

Try providing more agumented checkboard data to the camera calibration app. Also dont disturb the stereo camera set-up after calibration. Even a slight change in the setup will lead to different extrinsics meaning you need to recalibrate again.