How to uncompress data faster than using for loop

Hello,

I'm trying to uncompress data of row and col from 336 indexes in a RTable(basically a LUT which each index entry has different length of data). The first method is using "for" loop doing one at a time. The second method is using arrayfun and cell fun for the whole 336 indexes at once and it's actually slower (doubled the whole processing time in the end). Does anyone know why and how to make it faster?

Method 1 - using for loop for 1:336

LUT = RTable(Rstart:Rend);

row = int32(bitand(LUT,7))-4;

col = int32(bitand(bitshift(LUT,-3),7))-4;

Method 2 - doing all 336 at once

LUT = arrayfun(@(x) RTable(Rstart(x):Rend(x)),1:336,'UniformOutput', false);

row = cellfun(@(x) int32(bitand(x,7))-4, LUT ,'un', 0);

col = cellfun(@(x) int32(bitand(bitshift(x,-3),7))-4, LUT ,'un', 0);

I'd like to uncompress the max data possible all at once. Right now, I'm trying if I can do 336 at once faster and so far I haven't been sucessful.

I have access to 16 cores, 24 logical processor, Rtx 3080 gpu, 32GB RAM.

I appreciate your help.

Answer 1

[Joss Knight]

No, arrayfun and cellfun are just convenient ways of writing loops, they don't have any special magic and often cause certain optimizations to be impossible to apply. (The exception being arrayfun on gpuArray data.) In this case you now have three loops instead of one, so that's bound to be slower.

It looks like RTable is just a big array of encoded indices which are divided for some reason into groups of different lengths. Why then can you not just decode them all before dividing them into their groups?

allrows = int32(bitand(RTable,7))-4;              
allcols = int32(bitand(bitshift(RTable,-3),7))-4;

N = Rend-Rstart+1;
rows = mat2cell(allrows,N);
cols = mat2cell(allcols,N);