Reason for Checkerboard Corner Detection to Fail

I am using MATLAB R2014b to calibrate a stereo setup. I have 150 images of my checkerboard at various angles and distances. While using the Camera Calibrator app, I noticed that only 21 images were used for calibration, while the others were rejected. This is because detectCheckerboardPoints function is not able to detect the corner points in the remaining 129 images. Though the board is symmetric, the points have been detected properly in those 21 images (instead of 5X5, 5X4 points were detected, which isn't a problem for me)

This is the code used:

I1 = imread('T100009_141.jpg');
imshow(I1);
[imagePoints, boardSize] = detectCheckerboardPoints(I1);
figure;
imshow(I1, 'InitialMagnification', 50);
hold on;
plot(imagePoints(:, 1, 1, 1),imagePoints(:, 2, 1, 1), '*-g'); 
title('Successful Checkerboard Detection')

To improve the reconstruction algorithm, I want more images to be used in Calibration. Why aren't the corners detected in all these images, though they don't appear to have a lot of tilt?

This is the error message:

Index exceeds matrix dimensions.
Error in CheckerCornerDetection (line 7)
plot(imagePoints(:, 1, 1, 1), imagePoints(:, 2, 1, 1), '*-g');

I have attached a couple of images for your reference. Corner Points were detected in: PointsDetected(a).jpg and PointsDetected(b).jpg while the same function failed to detect checkerboard corner points in PointsNotDetected.jpg and PointsNotDetected(b).jpg.

Please help me figure out why it's not detected, though the image seems alright. What could the possible reasons be?

Answer 1

[Dima Lisin]

Hi Sanya,

I have noticed something strange about your images. There are white gaps between the corners of black squares that are supposed to be touching. I think this is what is confusing the detectCheckerboardPoints function. It is designed to detect "cross corners", where you have two white squares and two black squares touching at a point.

Try calling detectCheckerboardPoints on individual images, and see what you get. I am guessing that it will detect checkerboards of different sizes, because of those gaps. When you call the function on a set of images, and it detects different size boards, it will determine which board size is most common, and only accept the images where it detects a board of that size. That is why so many of your images are rejected.

How did you make the checkerboard? I would recommend printing the checkerboard board from the pdf that comes with the documentation on a good printer.

[Meghana Dinesh]

Hello Dima,

Thanks for your reply =) Yes, that could be one of the reasons. But what surprised me was that even if I call detectCheckerboardPoints on one image, such as PointsNotDetected.jpg, none of the corner points were detected. I expected at least a few corners where the black and white squares cross to be detected, if not all corners

In the code I have pointed to in my question above, I have called this function on an individual image. No corners are found.

I may be missing out on something......but cant figure out what.

This seems to be affecting my disparity map as well.

[Dima Lisin]

Hello Sanya,

It is definitely the gaps. I have saved PointsNotDetected.jpg, and then reduced its size by a factor of 4 in each dimension, to make the gaps disappear. Then I tried detectCheckerboardPoints and I got a perfect 5x5 checkerboard. Note that this is just to demonstrate that the gaps between the black corners are preventing the checkerboard from being detected. You should not resize your calibration images, because that will affect your accuracy. Rather you should definitely make a better checkerboard.

And one more thing. Please do not save your calibration images as jpeg. Jpeg compression creates image artifacts, which affect corner detection accuracy. You should either use a format with no compression or with lossless compression, like tiff or png.

[Meghana Dinesh]

Hi again,

Thanks a lot.......yeah, it was because of the gaps. Thanks for proving that using imresize.

One quick doubt.....

img = imread('PointsNotDetected.jpg');
img_resize = imresize(img,5);
[imagePoints, boardSize] = detectCheckerboardPoints(img_resize);
figure;
imshow(img_resize, 'InitialMagnification', 50);
hold on;
plot(imagePoints(:, 1, 1, 1),imagePoints(:, 2, 1, 1), '*-g');

This is the code I used. Although detectCheckerboardPoints seems to work fine, there is an error "Index exceeds matrix dimensions." in the last line..... I am unable to plot the detected corners.

What am I doing wrong?

[Dima Lisin]

Since you are detecting the checkerboard in a single image, the points should be of size M-by-2. So just do

plot(imagePoints(:, 1),imagePoints(:, 2), '*-g');

Also, you are making the image bigger, not smaller...

[Meghana Dinesh]

Thank you :) Yeah, changed it to 0.25.

[Meghana Dinesh]

In practice, using a combination of IR and visible light, for best results, this is how the checkerboard pattern is seen by camera. Is there any way I can preprocess these checkerboard images and make sure the corners are detected, neglecting the non cross corners? Can you suggest any processing technique?

[Dima Lisin]

detectCheckerboardPoints is meant to only detect the cross corners... As far as pre-processing, increasing the contrast may help. Generally, you want to be careful not to introduce artifacts, that may affect detection accuracy.

Answer 2

[Brian]

I have had this problem too. I have noticed that the "corner gap" problem is actually exaggerated with IR light illumination. From the few data sheets I've looked at, IR light often raises the minimum spatial frequency that can be captured by a CCD array (check the modulation transfer function / optical transfer function), though I'm not sure if this is a related issue. It might be how your camera adjusts for brightness and saturation as well. Even without IR illumination, I would say the detectCheckerboardPoints function works ~15% of the time in my hands (2 different cameras, a logitec C150 webcam and a Canon EOS DSLR), which sounds a lot like your 21/129. I think it might be a weakness of the detectCheckerboardPoints function to depend on the (generally finicky) handful of pixels that make up the intersection points while ignoring the other info available from the checkerboard, but this is necessary when the optical distortion of the image is unknown. However... it would be nice if there were a more robust checkerboard detection function for undistorted (calibrated camera) images!

[Dima Lisin]

Hi Brian,

If you absolutely need to detect the corners despite the gap, you can edit the detectCheckerboardPoints.m file, and increase the value of sigma on line 452 in function detectCheckerboadInOneImage.

I am confused, however. I assumed the "corner gaps" indicated a badly printed checkerboard pattern. If there is a visible gap between the corners, doesn't that mean that the single corner point cannot be reliably detected?

[Meghana Dinesh]

Hi Brian,

You're absolutely right! The gaps is because of the lighting, IR to be specific. For my algorithm development, I can use imresize and proceed with the later stages, but not during the actual implementation (this would reduce accuracy to a great extent). A more robust algorithm to detect corners would be very helpful.

Thanks Dima for your input. I just tried increasing sigma value. All corners are detected properly, even the ones with little gap (which weren't detected earlier).

What is the disadvantage of this? By increasing the value of sigma (more than 2), where am I likely to face problems?

[Dima Lisin]

The only disadvantage is that with larger sigma you may not be able to detect checkerboards with small squares. The value of 2 seems to be a reasonable compromise.

[Meghana Dinesh]

Alright. The size of squares printed on my checkerboard will be fixed. I can use a larger value of sigma in this case.

Thanks.

[Kukhokuhle Tsengwa]

Jesus! This was the problem I was having. Fixed it by setting the 'HighDistortion' parameter to true in the detectCheckerboardPoints() function.