speeding up fread for true 12bit data

I'm using fread to load 12bit data files (raw files from a high-speed camera). The relevant part of the code is here:

precision='ubit12=>uint16'; % convert 12bit to 16bit on reading
machinefmt='b';
fseek(mraw_file, 0, 'bof');
vid_segments=fread(mraw_file,inf,precision,machinefmt);
N = [im_width im_height length(frames)];
im=permute(reshape(vid_segments,N),[2 1 3]);

The code is working smoothly, however I've noticed that it is about 10x slower than if I use the same code to read other data stored as 16bit data using:

precision='*uint16';

I'm guessing this is because of how Matlab deals with data that isn't formatted as multiples of 1 byte? Is there any way I can read in the 12bit data files faster, using 8bit or 16bit precision and then converting to 12 bit?

For reference, the data is true 12bit so 2 x 12 bits of data = 3 bytes. It's not padded with zeros or random numbers so that 1 x 12bit of data = 2 bytes (which would simplify things).

Any help would be greatly appreciated!

[Jan]

If "1 x 12bit of data = 2 bytes", the data are padded with zeros bits, but not with zero bytes.

[Simon Walker]

but that's not case as I indicated. Sorry, I didn't write it clearly. the data is formatted so that 2x12bit = 3 bytes, NOT 1x12=2bytes

Answer 1

[Jan]

Read the data as bytes at first:

data = fread(mraw_file, inf, 'uint8=>uint16', machinefmt);
pad  = ceil(length(data) / 3) * 3 - length(data);
data = cat(1, data, zeros(pad, 1, 'uint16));
data = reshape(data, 3, []).';

Then convert the data in the memory:

video = [bitshift(data(:, 1), 4) + bitshift(data(:, 2), -4), ...
         bitshift(rem(data(:, 2), 16), 8) + data(:, 3)];

Is this faster? I'm not sure, if multiplications are faster than bit-shifting in Matlab. Try this:

video = [data(:, 1) * 16 + data(:, 2) / 16, ...
         rem(data(:, 2), 16) *256 + data(:, 3)];

A simple C-mex function would be faster, because it can avoid to create the temporary vectors. Do you have a C compiler installed?

[Simon Walker]

Thanks, that's made a big improvement!

It's still not quite as fast as for the same file saved in 16bit (4.5 seconds vs 0.9 seconds), but it's certainly a huge improvement as before it was taking ~35 seconds to run. I was able to get a marked improvement by reading the data in as little endian ordering and then reordering:

data = fread(mraw_file, inf, '*uint8', 'l');
data=uint16(swapbytes(data));

The two methods you suggested to convert the data took same time to run and is now the bottleneck (but still much better than before).

I do have a C compiler installed: 'Microsoft Windows SDK 7.1 (C)'

How could I go about making a C-mex function? I just tried putting your code into a function and then using mcc to create a c file, but when I then used the mex function on it, it came up with errors. I presume I'm doing it wrong.

[Jan]

It will be more efficient to write a hard coded function in C directly. When I find the time, I will post a working C-Mex function.

[Derek]

Any chance you ever made this C-Mex file?

[Star Strider]

@Derek — There is nothing by Jan in the File Exchange in that respect that I can find when I search using his full name.