Image registrations between depthmap and it's corresponding image

Hello MATLAB community.

I need help in spatially register two images.

I have an image in colormap(MXNX3 unit8) and it's corresponding depthmap array(M*N double). The depthmap is spatially a bit off and I want to register them and save the output. I tried doing it in registerEstimator app. It thrown errors saying "Registration failed. Try improving the quality of the matched features."

The code I have also not registered properly. Need help please.

My code:

clc
close all
img1 = imread('.jpg');
load('.mat');
img2 = A;
figure(1),imagesc(img2); figure(2),imagesc(img1);
img1_gray = rgb2gray(img1); 
fixed = img1_gray;
img2_double = im2double(img2);
moving = img2_double;
[optimizer, metric] = imregconfig('monomodal');
tform = imregtform(moving, fixed, 'translation', optimizer, metric);
registered = imwarp(moving, tform, 'OutputView', imref2d(size(moving)));
figure(3);
subplot(1,3,1); imshow(fixed); title('Fixed Image');
subplot(1,3,2); imshow(moving); title('Moving Image (Before)');
subplot(1,3,3); imshow(registered); title('Registered Image');

[Walter Roberson]

It would help to have your jpg and your mat for testing purposes.

[Eswar]

Thank you so much for your reply, Walter. Unfortunately I cannot share the images as it due to it's privacy. Will you be still able to help me with the code?

[Walter Roberson]

No.

Answer 1

[Ronit]

Hello @Eswar,

This is a common issue when registering a color image and a depth map since they have very different intensity patterns. Here is what I recommend you try to resolve this query:

• Normalize the depth map: Depth values can be very different in scale from image intensities. Normalizing helps registration algorithms compare them more effectively.

img2_norm = mat2gray(img2);

• Use feature-based registration: Since the two images are different modalities, feature-based methods (like SURF or ORB) work better than standard intensity-based registration.

% Extract and match features
pts1 = detectSURFFeatures(rgb2gray(img1));
pts2 = detectSURFFeatures(im2uint8(img2_norm));

• If not enough features, try intensity-based registration with 'multimodal': This is more robust for images with different intensity distributions.

[optimizer, metric] = imregconfig('multimodal');
tform = imregtform(img2_norm, rgb2gray(img1), 'translation', optimizer, metric);

• Visualize with 'imshowpair': This helps you check the alignment before and after registration.

imshowpair(rgb2gray(img1), img2_norm);       % Before
imshowpair(rgb2gray(img1), registered_img2); % After

Please refer to the documentation pages of the above-described MATLAB functions:

• 'mat2gray': https://www.mathworks.com/help/releases/R2024b/images/ref/mat2gray.html
• 'detectSURFFeatures': https://www.mathworks.com/help/releases/R2024b/vision/ref/detectsurffeatures.html
• 'imshowpair': https://www.mathworks.com/help/releases/R2024b/images/ref/imshowpair.html

I hope the above information helps you resolve the query!