convert pixels to object coordinates

I have a satellite with known coordinates as a function of time.

A sensor is installed on a gimbal attached to a satellite.

Gimbal is pointed to (pointAt) specific coordinates of a point of interest (POI). So, I can read azimuth and elevation angle of the Gimbal (gimbalAngles).

The sensor is CMOS, with known pixel size, number of pixels and focal distance of a lens.

When sensor observes a POI, the sensor surface is normal to the vector between the sesnor center and the POI.

I have another object X in the vicinity of the POI (at the same altitude).

My sensor has pixel coordinates of this object X.

How to transform these pixel coordinates to the coordinates of the object X in the same coordinate frame as the POI (e.g. ECEF, LLA etc)?

… and vice-versa: if I know the coordinates of the object X (ECEF, LLA), how to transform these coordinates to pixels of my sensor.

Can somebody please point me towards useful Matlab help page on this subject?

Image2world2d? But what is the world coordinates?

Answer 1

[Matt J]

Gimbal is pointed to (pointAt) specific coordinates of a point of interest (POI). So, I can read azimuth and elevation angle of the Gimbal (gimbalAngles).

It doesn't sound like you have enough information. You would need to know the distance of POI from the sensor, i.e. the full 3D set of spherical coordinates of POI relative to the gimbal.

Additionally, you need to know the orientation of the x,y,z coordinate axes of the gimbal in 3D world (ECEF, LLA) coordinates. You would probably need to have gimbal coordinates of not one, but three POIs to determine that.

[vasili]

Thanks Matt. The distance to POI from the gimbal can be calculated, because I know ECEF coordinates of the both. I know the coordinates of the gimbal (sesnor) in both ECEF and LLA (from the satelltite trajectory), in addition to azimuth and elevation towards the POI. I suspect there should be some transfer matrix to align the sensor frame to the frame of the POI and an object X, but I couldn't find any functions in Matlab so far.

[Matt J]

I suspect there should be some transfer matrix to align the sensor frame to the frame of the POI and an object X, but I couldn't find any functions in Matlab so far.

Because, as I mentioned, you don't have enough POIs. To align two frames, you need at least 3 non-colinear points whose 3D cartesian coordinates are known in both frames. So far, it sounds like you only have 2 (the POI and the sensor). If you can obtain 1 more point, then you can use this FEX download to find the rototranslation relating the frames,

https://www.mathworks.com/matlabcentral/fileexchange/26186-absolute-orientation-horn-s-method

[vasili]

ok, thanks Matt. Will think more about that

[Matt J]

OK. Maybe there is some station with known ECEF coordinates on the ground that is communicating with the satellite? Maybe that could be your 3rd point?