*.mat file size problem

Hi, I am currently processing an image using tiled-based approach. I save each tile after it is sampled from image and some features are extracted out of it. The types of features and image size is the same for all tiles but the size of *.mat file for each tile varies considerably going from 200 MB to 133 GB! Does anybody know about where things go wrong?

[Walter Roberson]

Please show your code.

It is expected that the size of .mat files might vary a little for data of exactly the same original size, but not by that much.

[Mehdi Ravanbakhsh]

The code is large so not suitable for this post bu what I am doing is to define a empty vector as:

b=[]; and then pass on this variable into a large number of sub-functions which perform convolution and aggregate to this variable like the code below

[ b] = feature_creation_ver2(b,imt,itr,band_dim_rgb,1,@stdfilt,NHOOD);

and here is the content of this function as follows

            for j=1:band_dim_rgb
                for k=1:numel(NHOOD)
                    NHOOD_temp=getnhood(strel('disk',NHOOD(k)));
                    b_temp=func_h(imt(:,:,j),NHOOD_temp);
                    b=[b double(b_temp(:))];
                end
            end

I hope this is helpful.

[Walter Roberson]

With the information you have given us, we cannot determine that b is being reset to [] between images. And we do not know what you are saving to disk.

I suggest that you go to your code that is doing the saving, and you disp() the size() of the variables you are saving.

Also if you are using the save --append flag, or if you are using matFile to do the saving, we need to know.

[Mehdi Ravanbakhsh]

Actually "b" is reset within two nested for loops where I access image tiles. 'b' is reset by b=[]; This image show the variability in file size although all are the same size.

[Walter Roberson]

Please use "whos" with the -file option to check the contents of the .mat files.

whos -file mosaic_small1_els_predict_3_15.mat
whos -file mosaic_small1_els_predict_4_12.mat

[Mehdi Ravanbakhsh]

Here is my testing on two different files in size as you see above but still not clear. Please see the result below

Answer 1

[Walter Roberson]

MATLAB does data compression for -v7 files and in some cases for -v7.3 files. The success of the compression is going to depend upon how much pattern the compressor can find. A matrix of all the same value would compress to much much smaller than a matrix that looked random to the compressor.

In the list you showed, none of the files are over 278 megabytes. Your X_obstrn variable would take 330 megabytes by itself if there were no compression. The conclusion would be that some of your matrices compress well and others do not.

[Mehdi Ravanbakhsh]

That is right. Those that are smallest are mainly filled with zeros and those bigger aren't uniform.