How does the winvideo adaptor in Image Acquisition Toolbox exactly return YUY2 data in YCbCr colorspace?

I have a camera (Leopard USB camera LI-USB30-M021X) with winvideo compatible DirectShow driver which says it outputs images in YUY2 format. Now YUY2 is a format in which you normally use 16-bit per pixel to describe a color image. This camera does not actually capture color images though, in reality it captures 12-bit grayscale images which are then encoded into those 16-bits per pixel YUY2 format. Can I recover the original 12-bit data format in MATLAB somehow?

Answer 1

[MathWorks Support Team]

If you look at how YUY2 data is stored:

https://msdn.microsoft.com/en-us/library/windows/desktop/dd206750(v=vs.85).aspx#yuy2

 

We see, that for the first 4 pixels, it looks like:

Where with the colors I have tried to indicate how I assume the camera vendor has encoded the 12-bit data as 2-bytes of data in YUY2.

 

If using videoinput with a YUY2 format and the default YCbCr ReturnedColorSpace, MATLAB then maps this data to a YCbCr image "im" in MATLAB in the following way:

Y0 = im(1,1,1)
U0 = im(1,1,2) = im(1,2,2)
Y1 = im(1,2,1)
V0 = im(1,1,3) = im(1,2,3)
Y2 = im(1,3,1)
U1 = im(1,3,2) = im(1,4,2)
Y3 = im(1,4,1)
V1 = im(1,3,3) = im(1,4,3)

Since there is no scaling or averaging going on, you should be able to access the original raw 12-bit data by combining the right elements again. i.e. to get Y0|U0 we need to take im(1,1,1) and combine it with im(1,1,2) where, depending on how the vendor encoded the data exactly in which order, one of those two should be shifted by 8 bits (= multiplying by 256). So then to get the first 12-bit value you would use:

v = uint16(ImageFrame(1,1,2))*2^8 + uint16(ImageFrame(1,1,1));

Similarly for Y1|V0 we would combine im(1,2,1) with im(1,2,3). Or for a whole 1280x720 image we should then be able to write:

A = zeros(720,1280,'uint16');
A(:,1:2:end) = uint16(ImageFrame(:,1:2:end,2))*2^8 + uint16(ImageFrame(:,1:2:end,1));
A(:,2:2:end) = uint16(ImageFrame(:,2:2:end,3))*2^8 + uint16(ImageFrame(:,2:2:end,1));