estimateMultiCameraParameters

Calibrate extrinsic parameters of multiple cameras with overlapping views

Since R2025a

Syntax

[cameraParams,imagesUsed] = estimateMultiCameraParameters(imagePoints,worldPoints,intrinsics)

cameraParams = estimateMultiCameraParameters(___,Name=Value)

Description

[cameraParams,imagesUsed] = estimateMultiCameraParameters(imagePoints,worldPoints,intrinsics) estimates extrinsic parameters of multiple cameras using images of one or more calibration patterns. The keypoints of the patterns in each image are specified in imagePoints. The keypoints of the calibration patterns in world coordinates is specified in worldPoints. The function adjusts all image points for distortion according to the lens distortion parameters specified in the intrinsics input. The function also returns the images you used, imagesUsed, to estimate the camera parameters.

Use this function to calibrate the extrinsic parameters of two cameras without overlapping fields of view. To calibrate the extrinsic parameters of two cameras with overlapping fields of view, use the estimateStereoBaseline function.

cameraParams = estimateMultiCameraParameters(___,Name=Value) specifies options using one or more name-value arguments in addition to any combination of arguments from previous syntaxes. For example, WorldUnits="mm" sets the world units to millimeters.

Examples

Calibrate a Multi-Camera System using a ChArUco board

Estimate the relative position and orientation of six cameras with overlapping fields of view by using calibration images that contain a single ChArUco board.

Download the calibration images.

calibImagesURL = "https://www.mathworks.com/supportfiles/vision/data/overlapping-cameras-charuco.zip";
calibImagesDir = fullfile(pwd,"overlapping-cameras-charuco");
calibImagesZip = fullfile(pwd,"overlapping-cameras-charuco.zip");
if ~exist(calibImagesZip,"file")
    disp("Downloading calibration images (52 MB)...")
    websave(calibImagesZip,calibImagesURL);
end

Downloading calibration images (52 MB)...

if ~exist(calibImagesDir,"dir")
    unzip(calibImagesZip,pwd)
end

Specify calibration image filenames for each camera.

numCameras = 6;
camDirPrefix = "Cam00";
imageFiles = cell(1,numCameras);
for i = 1:numCameras
    camDir = fullfile(calibImagesDir,camDirPrefix+i);
    imds = imageDatastore(camDir);
    imageFiles{i} = imds.Files;
end
imageFiles = [imageFiles{:}];

Define the ChArUco board properties. Specify checker size and marker size in centimeters.

markerFamily = "DICT_6X6_1000";
patternDims = [5 5];
markerSize = 6.8;   % in cm
checkerSize = 9.15; % in cm
numKeyPoints = prod(patternDims - 1);
minMarkerId = 144;

Detect the key points of the ChArUco board in the calibration images.

imagePoints = detectPatternPoints(imageFiles,"charuco-board",patternDims, ...
    markerFamily,checkerSize,markerSize,MinMarkerID=minMarkerId);

[==================================================] 100%
Elapsed time: 00:00:09
Estimated time remaining: 00:00:00

Generate the world points for the pattern.

worldPoints = patternWorldPoints("charuco-board",patternDims,checkerSize);

Load the intrinsic parameters of the six cameras. These parameters have been estimated using the Using the Single Camera Calibrator App.

ld = load("sixCameraIntrinsics.mat");

Perform multi-camera calibration.

params = estimateMultiCameraParameters(imagePoints,worldPoints,ld.intrinsics,WorldUnits="cm");

Visualize the calibration accuracy.

figure(Position=[100,100,1000,400])
showReprojectionErrors(params)

Visualize the camera extrinsic parameters.

figure
showExtrinsics(params)
view(2)

Multi-Camera Calibration with Overlapping and Non-Overlapping Cameras

Calibrate multiple cameras that have some cameras sharing fields-of-view and some cameras not sharing any field-of-view.

Download the calibration images containing two ChArUco boards that were captured from six cameras as per the guidelines outlined in Prepare Cameras and Capture Images for Multi-Camera Calibration. The calibration images were captured by moving the multi-camera system around the stationary ChArUco boards.

calibImagesURL = "https://www.mathworks.com/supportfiles/vision/data/non-overlapping-cameras-charuco.zip";
calibImagesDir = fullfile(pwd,"non-overlapping-cameras-charuco");
calibImagesZip = fullfile(pwd,"non-overlapping-cameras-charuco.zip");
if ~exist(calibImagesZip,"file")
    disp("Downloading calibration images (68 MB)...")
    websave(calibImagesZip,calibImagesURL);
end
if ~exist(calibImagesDir,"dir")
    unzip(calibImagesZip,pwd)
end

Specify calibration image filenames for each camera.

numCameras = 6;
camDirPrefix = "Cam00";
imageFiles = cell(1,numCameras);
for i = 1:numCameras
    camDir = fullfile(calibImagesDir,camDirPrefix+i);
    imds = imageDatastore(camDir);
    imageFiles{i} = imds.Files;
end
imageFiles = [imageFiles{:}];

Display images captured by the six cameras from one of the views.

figure
t = tiledlayout(2,3,TileSpacing="tight", Padding="tight");
for i = 1:numCameras
    nexttile
    imshow(imageFiles{25,i})
    title("Camera " + i)
end

Define the ChArUco board properties. Specify checker size and marker size in centimeters.

markerFamily = "DICT_6X6_1000";
patternDims = [5 5];
markerSize = 2.55;   % in cm
checkerSize = 3.40; % in cm

Detect the key points of the two ChArUco boards captured in the calibration images using the detectMultiPatternPoints function.

patternCount = 2;
imagePoints = detectMultiPatternPoints(imageFiles,patternCount,"charuco-board",patternDims,markerFamily, ...
    checkerSize,markerSize);

[==================================================] 100%
Elapsed time: 00:00:09
Estimated time remaining: 00:00:00

Generate the world coordinates of each pattern points in their respective pattern coordinate system.

worldPoints = patternWorldPoints("charuco-board",patternDims,checkerSize);

Load the intrinsic parameters of the six cameras. These parameters have been estimated using the Using the Single Camera Calibrator App.

ld = load("sixCameraIntrinsicsNoOverlap.mat");

Perform multi-camera calibration. Enable verbose mode to track the progress of the calibration process.

params = estimateMultiCameraParameters(imagePoints,worldPoints,ld.intrinsics,WorldUnits="cm",Verbose=true);

Estimating multi-camera extrinsic parameters
--------------------------------------------
* Number of cameras: 6
* Number of patterns: 2
* Number of views: 30
* Number of images used: 110

* Undistorting image points...done.

* Grouping cameras with overlapping field-of-view....done.
* Group 1: camera 3 4 5 6
* Group 2: camera 1 2

* Refining camera poses of each group:
* Group 1...done.
* Group 2...done.

* Estimating relative pose between camera groups:
* [Pair 1/1] Group 1 and 2...done.

* Refining all camera poses and pattern poses...done.
* Finished calibrating cameras.

Visualize the camera extrinsic parameters along with the patterns from a couple of views.

figure
showExtrinsics(params,"CameraCentric",ViewIndex=21:22)

Calibrate a Multi-Camera System using Multiple ChArUco Boards

Estimate the relative position and orientation of eight cameras with overlapping fields of view by using calibration images that contain multiple ChArUco boards.

Download the calibration images.

calibImagesURL = "https://www.mathworks.com/supportfiles/vision/data/overlapping-cameras-multiple-charuco.zip";
calibImagesDir = fullfile(pwd,"overlapping-cameras-multiple-charuco");
calibImagesZip = fullfile(pwd,"overlapping-cameras-multiple-charuco.zip");
if ~exist(calibImagesZip,"file")
    disp("Downloading calibration images (86 MB)...")
    websave(calibImagesZip,calibImagesURL);
end
if ~exist(calibImagesDir,"dir")
    unzip(calibImagesZip,pwd)
end

Specify calibration image filenames for each camera.

numCameras = 8;
camDirPrefix = "Cam00";
imageFiles = cell(1,numCameras);
for i = 1:numCameras
    camDir = fullfile(calibImagesDir,camDirPrefix+i);
    imds = imageDatastore(camDir);
    imageFiles{i} = imds.Files;
end
imageFiles = [imageFiles{:}];

Define the ChArUco board properties. Specify checker size and marker size in centimeters.

markerFamily = "DICT_6X6_1000";
patternDims = [5 5];
markerSize = 6.8;   % in cm
checkerSize = 9.15; % in cm

Detect the keypoints of the eight ChArUco boards captured in the calibration images using the detectMultiPatternPoints function. Specify the MinMarkerID for each pattern.

patternCount = 8;
patternIDs = 144:12:228;

imagePoints = detectMultiPatternPoints(imageFiles,patternCount,"charuco-board",patternDims,markerFamily, ...
    checkerSize,markerSize,MinMarkerID=patternIDs);

[==================================================] 100%
Elapsed time: 00:00:14
Estimated time remaining: 00:00:00

Display the detected keypoints in a couple of views and three of the eight cameras.

t = tiledlayout(2,3,TileSpacing="compact",Padding="compact");
colors = ["red","green","cyan","black","magenta","yellow","blue","white"];
for viewIdx = [19,24]
    for camIdx = [1,5,7]
        nexttile
        imshow(imageFiles{viewIdx,camIdx})
        title("Camera " + camIdx + " | View " + viewIdx)
        hold on
        for patternIdx = 1:patternCount
            plot(imagePoints(:,1,viewIdx,camIdx,patternIdx),imagePoints(:,2,viewIdx,camIdx,patternIdx),"yo",...
                MarkerSize=3,MarkerFaceColor=colors(patternIdx))
        end
    end
end
title(t,"Detected keypoints in multiple ChArUco boards")

Generate the world coordinates of each pattern points in their respective pattern coordinate system.

worldPoints = patternWorldPoints("charuco-board",patternDims,checkerSize);

Load the intrinsic parameters of the eight cameras. These parameters have been estimated using the Using the Single Camera Calibrator App.

ld = load("eightCameraIntrinsics.mat");

Perform multi-camera calibration.

params = estimateMultiCameraParameters(imagePoints,worldPoints,ld.intrinsics,WorldUnits="cm");

Plot the estimated camera poses.

figure
plotCamera(params.CameraPoses)
axis equal
grid on
view([0 1 0])

Input Arguments

imagePoints — Image coordinates of keypoints in calibration pattern

numKeyPoints-by-2-by-numViews-by-numCameras array | numKeyPoints-by-2-by-numViews-by-numCameras-by-patternCount array

Image coordinates of keypoints in calibration pattern, specified as a numKeyPoints-by-2-by-numViews-by-numCameras 4-D array or a numKeyPoints-by-2-by-numViews-by-numCameras-by-patternCount 5-D array of x-y image coordinates.

• numKeyPoints — Number of keypoints in each view, which must be greater than 3.

• numViews — Number of unique views of the calibration patterns with respect to each camera. At each view, the pattern must be visible to at least two cameras. Each camera should observe the pattern at least once along with some other cameras. This index represents the points from camera j when the pattern is located at the i^th view.

  • imagePoints(:,:,i,j)

• numCameras — Number of cameras to calibrate, which must be greater than 2.

• patternCount — Number of calibration patterns. When using multiple calibration patterns, each pattern must either be visible to at least two cameras in a view or be seen by a camera along with another pattern in the same view. Additionally, each camera must observe at least one pattern that is also seen by another camera in one of the views.

To include partially detected patterns, use [NaN,NaN] as x-y coordinates for missing keypoints. For calibration with a single pattern, you can create a 4-D array of keypoints using the detectPatternPoints function. For calibration with multiple patterns, you can create a 5-D array of keypoints using the detectMultiPatternPoints function.

Data Types: double

worldPoints — World coordinates of keypoints on calibration pattern

numKeyPoints-by-2 array

World coordinates of keypoints on calibration pattern, specified as an numKeyPoints-by-2 array. numKeyPoints is the number of keypoints. Each keypoint is of the form [x,y]. The pattern must be planar, so all z-coordinates are assumed to be 0.

Data Types: double

intrinsics — Intrinsic parameters of each camera

numCameras-element array

Intrinsic parameters of each camera, specified as a numCameras-element array. numCameras is the number of cameras and each element is either a cameraIntrinsics or fisheyeIntrinsics object.

• If all cameras share the same model (lens characteristics), the array is contains cameraIntrinsics or fisheyeIntrinsics objects.

• If different models are used across the cameras, the array is a cell array, with each cell containing a different type of camera intrinsics object.

The intrinsic parameters of the cameras are not refined by this function. By default, all the keypoints are corrected for distortion based on the lens distortion specified in the intrinsics input. To enhance speed, you can use the undistortImage or undistortFisheyeImage function to correct lens distortion of the images. After correcting the images, detect keypoints on these undistorted images. Then, use the undistorted keypoints along with the corresponding intrinsics object to perform calibration.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: estimateMultiCameraParameters(imagePoints,worldPoints,intrinsics,WorldUnits="mm") sets the world units to millimeters.

WorldUnits — Units of world points

"mm" (default) | character vector | string scalar

Units of world points, specified as a character vector or string scalar.

ReferenceCameraIndex — Index of camera that defines reference coordinate system

1 (default) | integer

Index of the camera that defines the reference coordinate system, specified as an integer. All other camera poses are defined relative to the reference camera. Use the ReferenceCameraPose name-value argument to specify the reference camera pose.

ReferenceCameraPose — Pose of reference camera

rigidtform3d(eye(4)) (default) | rigidtform3d scalar

Pose of reference camera with respect to a world coordinate system, specified as a rigidtform3d scalar.

Verbose — Display detection progress information

false or0 (default) | true or 1

Display detection progress information in the Command Window, specified as a numeric or logical 1 (true) or 0 (false). Set Verbose to true to display progress information.

Output Arguments

cameraParams — Parameters of multi-camera system

multiCameraParameters object

Parameters of a multi-camera system, returned as a multiCameraParameters object. The extrinsic parameters include the rotation and translation that describe the position and orientation of each camera with respect to the reference camera.

imagesUsed — Images used to estimate camera parameters

numViews-by-numCameras logical array

Images you use to estimate camera parameters, returned as a numViews-by-numCameras logical array. numViews corresponds to the number of camera views used and numCameras is the number of cameras. The array indicates which images you used to estimate the camera parameters. A logical true value in the array indicates which images you used to estimate the camera parameters.

References

[1] Rameau, Francois, Jinsun Park, Oleksandr Bailo, and In So Kweon. "MC-Calib: A generic and robust calibration toolbox for multi-camera systems."Computer Vision and Image Understanding"217 (2022): 103353.

[2] Tsai, Roger Y., and Reimar K. Lenz. "A new technique for fully autonomous and efficient 3 d robotics hand/eye calibration." IEEE Transactions on robotics and automation 5, no. 3 (1989): 345-358.

Version History

Introduced in R2025a

R2026a: Added support for multiple calibration patterns and cameras with non-overlapping fields of view

You can now specify a 5-D array for imagePoints to include the number of calibration patterns and from multiple cameras with or without overlapping fields of view. Use the ReferenceCameraPose name-value argument to specify the camera pose with respect to a world coordinate system.