Stereo Calibration of two cameras with different camera resolutions

Hi Everyone;

I have just started with camera calibration and found the awesome APP that MATLAB has to perform the calibration. Unfortunately, we cannot perform stereo calibration with it.

However, there is a matlab example that allows for stereo calibration but the problem I am having is that the resolutions of the two cameras I am using are different. One is high definition (1980X1080) and the other one is (512X412) so I cannot use the code from the example unless I try to modify the image.

So I am guessing I should kind of "cut out" the middle 512X412 of the 1980X1080 image to make the calibration? or would imsize work? Seems to modify the image too much.. Anyone has any idea on how this can be done or the camera calibration in general can be done?

Thanks; Ali

Answer 1

[Shida]

Hi, It's very normal to have to calibrate two cameras of different resolution when working in Multi-modal imaging. It's always the case that you have two different sensors with different technologies and highly unlikely that they have same resolution. Kinect is of course a very common example. Unfortunately, the official matlab app doesn't support this. There are some codes which make this possible for kinect 1 which you can google. But if you're looking for something with kinect 2, you can try first using the detectCheckerboardPoints from matlab for each set of images (check the matlab manual) and then when you have the points use : imagePoints = cat(4, poitnsRGB, pointsIR); Then feed these to the estimateCameraParameters (check the manual for more info). The catch is that I've done this for a HD camera and the kinect IR and I get terrible amount of error. I'm still trying to fix this. Worst case, one can use Kinect 1 with one of the available calibration codes.

Regarding what you said about cropping the images, that's a calibration sin ! If you look up info on how to calibrate cameras, they always tell you not to do that because that messes up everything.

Good luck.

[Zhongwei Xu]

Hi, I followed your way, and I can also get the stereoParamas by using estimateCameraParameters. The problem is the same as yours, i.e., the camera parameters got from stereoPamams are different from the independent camera calibaration results. Now I really doubt if the estimateCameraParameters works for two cameras with different resolutions.

Have you figured it out?

[Zhongwei Xu]

Hi, I just found your another answer in this link with more details: https://www.mathworks.com/matlabcentral/answers/268152-mapping-rgb-and-depth-kinect-v2#answer_209992

And it works! Now the reprojectionError and the cameraParameters are correct.

Thanks for your provided codes and explanations :)

[Shida]

You're welcome!

[Yash Rathore]

Hey Shida, I followed the same steps you provided, and it worked. Thanks a lot for sharing it. :)

Answer 2

[Image Analyst]

I'd buy another camera. Why complicate things when simply buying another camera will solve the problem?

[Mohammad]

Hi thanks for the response. Well my problem is that the cameras are housed together in one unit. So I need the to know the rotation and translation matrix of one camera w.r.t the other.

[Image Analyst]

So buy two new cameras.

[Mohammad]

Well the device I am using is the new Kinect for Windows V2 and I am trying to see how good the factory calibration is for this new device. So I kind of need the rotation and translation matrix inbetween the two new cameras. Thanks

[Walter Roberson]

Hania Alhamad comments to Image Analyst

The answer is irrelevant. Some times there is a need to calibrate two cameras with different resolutions. and buying cameras won't solve the issue. like for example calibrating a thermal and a RGB.

Answer 3

[muhammet balcilar]

Yes, it is still problem to calibrate different resolution camera for stereo vision system. I had the same difficulties. However I just modifie some functions and prepare a demo. here you can find my demo.

https://github.com/muhammetbalcilar/Calibration-Under_Different-Resolution

[Ehsan Zare Bidaki]

Hi Muhammet, I used your code for stereo rectification. It is working well for the calibration part. However, after rectification, the images are in the same size. It means the images were changed which is not good for my project. I was wondering if there is a way to consider the original size for the output images. Thanks!

[muhammet balcilar]

it is not possible to rectify the images in the same resolution of the given image resolution.

But it is possible to project one image into another image. I just updated the demo code and added the results of the projection to the low-resolution image onto the high-resolution image. But without knowing the real depth of the object it does not work 100% accurate. In the given demo, I just supposed all objects are on Z=0 plane where the top-left corner of the first calibration checkerboard is our reference (0,0,0).

within that projection for instance if you have one thermal camera and one USB camera you can know the temperature of each pixel on usb camera image under the assumption that all objects are at Z=0 plane . Just check the updated demo,

https://github.com/balcilar/Calibration-Under_Different-Resolution

[Archana Gupta]

Hi Muhammet,

I am trying your code for calibration of cameras with different resolutions, but one of my undistorted image is completely black.

Do you have any idea about this error.

[Image Analyst]

Maybe they're floating point images not in the range of 0-1 so you need to use [] in imshow():

imshow(floatingPointImage, []);

[Diwakar Manickavelu]

Hey Muhammet, Thank you very much for this code. It really helped me solved my issue. But, I would like to do this work without the use of toolbox. So, could you tell me what exact modifications did you do in the codes. It would really make my life much easier.

Thank you once again.

Diwakar