Plot an interpolated plane in between two planes

I have two ground truth depth images (image 1 and image 2) which are located at different distances (150 cm and 203 cm) from the camera. Both the depth images are perpendicular to the camera. I am uploading here the original images.

I'd like to get a middle image by interpolation which will be in between these images (see the illustration).

Illustration:

To do that, I have first cropped the object from the images and then planning to use scatteredinterpolant function to get the interpolated plane in between these two images. However, I am struggling how exactly should I define the interpolation to get the interpolated plane.

I have attached here the depth images and the cropped images and a bit of codes which I did so far.

Any suggestions?

Note: I already have a middle ground truth depth image which is located at 178 cm. I need the interpolated plane so that I can compare the accuracy of the ground truth image with the interpolated plane/surface.

thresholdValue = 10;
originalImage1 = imread('depth_153.png');
originalImage2 = imread('depth_203.png');

binaryImage1 = originalImage1 > thresholdValue;
binaryImage2 = originalImage2 > thresholdValue;
EdgeMeasurements1 = regionprops(binaryImage1, originalImage1, 'all');
EdgeMeasurements2 = regionprops(binaryImage2, originalImage2, 'all');

croppedImage1 = imcrop(originalImage1, EdgeMeasurements1.BoundingBox);
croppedImage2 = imcrop(originalImage2, EdgeMeasurements2.BoundingBox);

x = 1:size(croppedImage1,1);
y = 1:size(croppedImage1,2);
[X,Y] = meshgrid(x,y);

Answer 1

[KSSV]

thresholdValue = 10;
originalImage1 = imread('depth_153.png');
originalImage2 = imread('depth_203.png');

binaryImage1 = originalImage1 > thresholdValue;
binaryImage2 = originalImage2 > thresholdValue;
I = cat(3,binaryImage1,binaryImage2) ;
[nx,ny] = size(binaryImage1) ;
[X,Y,D] = ndgrid(1:nx,1:ny,[153 203]) ;
[Xi,Yi,Di] = ndgrid(1:nx,1:ny,[153 178 203]) ;
Ii = interpn(X,Y,D,double(I),Xi,Yi,Di) ;

[Tariqul]

Thanks for your quick reply. I would give it a try. However, if you consider the cropped images, they are of different dimensions, due to the distance from the cameras (the illustration also shows it). Besides, there is also depth value difference on the surface of the images. For example, on Image 1 the object has depth of 111 cm for position (320,240) and have 113 cm at position (400, 260). Similar for the 2nd crop image. The values are given in millimeter in the xml files. How do I then get the interpolated plane for these two cropped images?

[KSSV]

It is interpolated on the whole image.....try cropping on Ii(:,:,1:3)

[Tariqul]

I have 2 followup questions: 1) The depth images have different intensity values over the surface. For example, for one image, max,min and mean intensities are 1152, 1094 and 1124. For other image, max, min and mean intensity are 2262, 2196 and 2229. So, should I use the mean intensities here:

[X,Y,D] = ndgrid(1:nx,1:ny,[1124 2229]) ;

2) Here, Ii (see below) has 3 channels.

Ii = interpn(X,Y,D,double(I),Xi,Yi,Di) ;

Which one would be the interpolated plane? Which channels values should I take as the intensity values of the interpolated plane?

[Tariqul]

I am getting this interpolated plane with your suggested method. I used the following codes as you suggested.

I = cat(3,originalImage1,originalImage2) ;
[nx,ny] = size(originalImage1) ;
[X,Y,D] = ndgrid(1:nx,1:ny,[1124 2229]) ;
[Xi,Yi,Di] = ndgrid(1:nx,1:ny,[1124 1623 2229]) ;
Ii = interpn(X,Y,D,double(I),Xi,Yi,Di) ;
imshow(Ii(:,:,1:3));

Interpolated image:

The middle white part has the dimension of the 2nd original image and the yellow part has the dimension of the 1st original image. By looking at the interpolated plane, I found that it has 2 different levels of intensity values. The small white part has the intensity values close to 1623 which matches with what we define here [1124 1623 2229] in the code:

[Xi,Yi,Di] = ndgrid(1:nx,1:ny,[1124 1623 2229]) ;

But the outside region of the white part has different value such as 621, 615 etc which are not as expected. Please look at the 3D plot below.

3D plot of the interpolated image:

I was expecting the interpolated plane to be in between the values of the original images and the plane would have a dimension smaller than the near image (originalImage1) and bigger than the far image (originalImage2) similar to the illustration I uploaded previously.

Any suggestions?

[Sumayyah Alhydary]

Thanks for your discussion it's very helpful.

I'm getting this error when I run the code:

Error using griddedInterpolant

The sample points arrays must have the same size as the sample values array.

I think the problem is with the way I'm reading the image, so my I = 250 x 250 x 6 while x,y,z are of size 250 x 250 x 2. How can I fix that if I'm using a grayscaled images? ( I don't want the colors to be either black or white, I want them to have different gray values).

how can I fix that?

Thanks in advance :)