Incorrect perspective from image

Hi,

I am trying to track the spot position within the box on the right of the image, using the checkerboard as a reference to find the perspective, using the method described in https://www.mathworks.com/help/vision/examples/measuring-planar-objects-with-a-calibrated-camera.html

Using the attached image (30V_40V.bmp), the reprojected origin axis' do not point towards the checkerboard corners (red dots). Therefore, the target origin (x_t, y_t) shoul be at the center of the large square, which it clearly isn't (results.jpg).

Can someone help me out with what doesn't work out in the code?

clear
close all
clc
addpath('Camera Calibration');
check_images = 0; %turn on if you want to check the corner detection in the calibration images

%% Get set of calibration images
numImages = 16;
files = cell(1, numImages);
for i = 1:numImages
    files{i} = fullfile(sprintf('snapshot%d.bmp', i));
end

%% Detect the checkerboard corners in the images
[imagePoints,boardSize,imagesUsed] = detectCheckerboardPoints(files);

%% Make sure that the checkerboard corners are properly recognized
if check_images
    if ishandle(1)
        close(1)
    end
    figure(1);
    filesUsed = files(imagesUsed);
    for ii = 1:sum(imagesUsed)
        I = imread(filesUsed{ii});
        imshow(I,[0 255]); hold on;
        plot(imagePoints(:,1,ii),imagePoints(:,2,ii),'.r');
        drawnow;
        pause;
        cla;
    end
    close(1);
end

%% Generate the world coordinates of the checkerboard corners in the
% pattern-centric coordinate system, with the upper-left corner at (0,0).
squareSize = 2; % in millimeters
worldPoints = generateCheckerboardPoints(boardSize, squareSize);

% Calibrate the camera.
[cameraParams, ~, estimationErrors] = estimateCameraParameters(imagePoints, worldPoints);
displayErrors(estimationErrors, cameraParams);

% Evaluate calibration accuracy.
if ishandle(2)
    close(2)
end
figure(2);
showReprojectionErrors(cameraParams);
title('Reprojection Errors');

%% ----------- Can modify the below to match our real setup -----------

%% Now open an image that contains the laser spot
folder = 'Test 3';
addpath(folder);
imOrig = imread(fullfile('30V_40V.bmp'));

%% Undistort the image
% [imUndistorted,newOrigin] = undistortImage(imOrig, cameraParams, 'OutputView', 'full' );
imUndistorted = imOrig;
[imagePoints, boardSize] = detectCheckerboardPoints(imUndistorted);

if ishandle(3)
    close(3)
end
figure(3) 
imshow(imUndistorted); hold on;
plot(imagePoints(:,1),imagePoints(:,2),'.r');
h = gcf;
h.Position = [6 619 599 366];
%% Detect the checkerboard

% imagePoints = imagePoints + newOrigin;

%% Compute extrinsics for this image
squareSize = 1; % in millimeters
worldPoints = generateCheckerboardPoints(boardSize, squareSize);
[R, t] = extrinsics(imagePoints, worldPoints,cameraParams);

% Note: This is what "worldToImage" does:
%cameraMatrix = [R; t] * cameraParams.IntrinsicMatrix;
%p = [0 0 0 1;...
%     4 0 0 1]*cameraMatrix;
%imagePointsTmp = bsxfun(@rdivide, p(:, 1:2), ...
%                p(:, 3));

%% Draw Image Coordinate System
oi = [2 2];
len = 50;
X = [oi(1) oi(1);... 
    len oi(1)];
Y = [oi(2) oi(2);... 
     oi(2) len];
hl = line(X,Y,'LineWidth',2,'Color','green');
text(len,oi(2),'x_{image}','Color','green','FontSize',10,...
        'HorizontalAlignment','left','VerticalAlignment','top');
text(oi(1),len,'y_{image}','Color','green','FontSize',10,...
        'HorizontalAlignment','left','VerticalAlignment','top'); 

%% Draw World Coordinate System
len = 1;
o1 = [0 0 0];
o2 = [len 0 0];
o3 = [0 len 0];
ow = worldToImage(cameraParams,R,t,[o1; o2; o3]);
X = [ow(1,1) ow(1,1);... 
     ow(2,1) ow(3,1)];
Y = [ow(1,2) ow(1,2);... 
     ow(2,2) ow(3,2)];
hl = line(X,Y,'LineWidth',2,'Color','green');
ht = text(ow(2,1),ow(2,2),'x_{w}','Color','green','FontSize',10,...
        'HorizontalAlignment','left','VerticalAlignment','top','Rotation',90);
text(ow(3,1),ow(3,2),'y_{w}','Color','green','FontSize',10,...
        'HorizontalAlignment','left','VerticalAlignment','top','Rotation',90);

%% Draw Target Coordinate System
Rt = [0 1 0;...
      1 0 0;...
      0 0 -1];
o1 = [4 16.35 0];     %This comes from our drawing file
o2 = o1+[len 0 0]*Rt; %rotates target basis vectors into world coordinate system
o3 = o1+[0 len 0]*Rt; %ditto
ot = worldToImage(cameraParams,R,t,[o1; o2; o3]);

X = [ot(1,1) ot(1,1);... 
     ot(2,1) ot(3,1)];
Y = [ot(1,2) ot(1,2);... 
     ot(2,2) ot(3,2)];
hl = line(X,Y,'LineWidth',2,'Color','green');
ht = text(ot(2,1),ot(2,2),'x_{t}','Color','green','FontSize',10,...
        'HorizontalAlignment','left','VerticalAlignment','top');
text(ot(3,1),ot(3,2),'y_{t}','Color','green','FontSize',10,...
        'HorizontalAlignment','right','VerticalAlignment','bottom'); 

%% Get Laser Spot in Image Coordinate Frame
threshold = 220;
imLaser = (imUndistorted > threshold);
hblob = vision.BlobAnalysis;
hblob.AreaOutputPort = false;
hblob.BoundingBoxOutputPort = false;
hblob.MinimumBlobArea = 40;   %minimum blob size in pixels
pim = step(hblob, imLaser);    %point of laser in image coordinates
figure(3); hold on;
plot(pim(1),pim(2),'*k','MarkerSize',6);

%% Get Laser Spot in Target Coordinate Frame
pw = pointsToWorld(cameraParams, R, t, pim);
T_wt = [0 1 0  4;...
        1 0 0 16.35;...
        0 0 -1 0;...
        0  0 0 1];
pt = inv(T_wt)*[pw 0 1]' %this is ultimately what we are after

Thanks a bunch!

Answer 1

[Nadia Aljuda]

hi mr. can u answer my quesitions on ur code? how know the suitable size of ur image that u has used (2mlm) instead of 29?