How do I get readImage (ROS) to work?

I have a file recorded with a RealSense-camera and it is saved as a rosbag file .bag.

I want to access the individual frames as images through Matlab and I have found information how to do from different forums. One example is from Matlab answers: https://se.mathworks.com/matlabcentral/answers/407981-access-images-from-within-a-rosbag

The problem is that that the function readImage doesn't work.

Depending on how I use it I get different error messages.

Either:

Array indices must be positive integers or logical values.

The error is generated with this code:

bag = rosbag('20190402_093734.bag');

bagInfo = rosbag('info', '20190402_093734.bag')

bSel = select(bag, 'Topic', '/device_0/sensor_1/Color_0/image/data');

msgs = readMessages(bSel,"DataFormat","struct");

img = readImage(msgs{1});

Or:

Undefined function or variable 'readImage'.

The error is generated with this code:

bag = rosbag('20190402_093734.bag');

bagselect1 = select(bag, 'Topic', '/device_0/sensor_0/Depth_0/image/data')

bagselect2 = select(bag, 'Time', [1 5], 'Topic', '/device_0/sensor_0/Depth_0/image/data')

bagselect2.AvailableTopics{1,end}

bagselect3 = select(bagselect2, 'Time', [1 5])

msgs = readMessages(bagselect3,"DataFormat","struct");

img = readImage(msgs{1});

I thought there was a problem with my installation of ROS so I tried using the online version of MATLAB but I got the same error message there.

I am assuming that there is a very simple, logical solution to this but I have no idea of what it might be. Could it be a problem with the .ros-file?

Answer 1

[Cam Salzberger]

Hello Ulrik,

It's good that you are experimenting with the 'DataFormat','struct' option for extracting messages from rosbags, since it is a big performance improvement. Unfortunately, the convenience functions (readImage, readPointCloud, etc.) don't yet support accepting structs as input, and are expecting ROS message objects. If you want to use readImage, I would suggest leaving off the 'DataFormat','struct' option, and passing the output to readImage. Otherwise, you can manually extract the image from the ROS message structure.

-Cam

[Ulrik Söderström]

Hi Cam.

I am unable to extract messages without the 'DataFormat','struct' option so this is why I am using it. I found a lot of discussions about generating custom messages so that ReadreadMessages should work but the content of an images should already be an existing message. So, at the moment I cannot remove the 'DataFormat','struct' option from my code.

But, you also say that i manually can extracct the image from the message structure. I can access the .Data in the message so I have all the pixel inforamtion that I need. My problem here is that I haven't found a way to assemble it into a correct image and I haven't found inofrmation about how it is saved in the message structure. Can you tell me how the pixels are arranged in the .Data vector? I can see that it is the the same amount of pixels in that vector as the amount needed to create the RGB image I need. I just don't know how I arrange them.

/Ulrik

[Cam Salzberger]

Hey Ulrik,

I'm sorry to hear that non-struct data extraction isn't working for you, and I can understand the annoyance of not being able to use readImage.

Depending on your particular image encoding, you could probably determine the color order and dimensions, and simply index and reshape to extrac the image. Most ROS image messages store color order by pixel, as in the vector is RGBRGBRGB or if there's a transparency channel, RGBARGBA etc. Once you determine that, you can simply index in to get a full red, green, blue, and maybe alpha channel (e.g. red = msg.Data(1:3:end), etc.). Then you can reshape (reshape(width, height), I think) it based on the height and width to get it in a usable format. You can look at the ros.msg.sensor_msgs.Image class to get an idea of how MATLAB does it.

Or, you could create an empty message of the same type, and copy all the fields over from the struct that you need (everything but the Header and MessageType, pretty much). Then you can feed that message into readImage to get it out automatically. It's slow, but it should work.

-Cam

[Ulrik Söderström]

Hi Cam.

Thanks a lot :).

When I create an empty message and copy all the content (except Header) I can use readImage perfectly.

/Ulrik

[Mohamed Atia]

Could you please post the code that you used to "create an empty message and copy all the content (except Header) " so you can use readImage ? Thanks

[Cam Salzberger]

Something like this should work, though I haven't tested it:

msgStructs = readMessages(bagSel, 'DataFormat', 'struct');
msg = copyImage(msgStructs{1});
I = readImage(msg);

function msg = copyImage(msgStruct)
    msg = rosmessage(msgStruct.MessageType);
    fNames = fieldnames(msg);
    for k = 1:numel(fNames)
        if ~any(strcmp(fNames{k}, {'MessageType', 'Header'}))
            msg.(fNames{k}) = msgStruct.(fNames{k});
        end
    end
end