MatFile grows exponentially larger in an empty table replacement loop?

First create a MatFile:

a=matfile("Test.mat");

Then create an empty table:

a.Data=table('Size',[0 1],'VariableTypes',"uint8");

The Test.mat is 23.3 ㎅. Now we run the code again to replace the table with the same new one:

a.Data=table('Size',[0 1],'VariableTypes',"uint8");

Note the file becomes 51.6 ㎅, about 2.2 times larger. Again, 91.6 ㎅, 1.8 times larger; 150 ㎅, 1.6 times larger; 223㎅, 1.5 times larger; 308㎅, 1.4 times larger; 407 ㎅, 1.3 times larger … What a horrible increase!

Answer 1

[Asvin Kumar]

This is a known issue in MAT files saved with version 7.3 based on HDF5 which does not manage freespace as effectively as it should. Frequently rewriting or deleting a dataset can result in freespace and cause the file to grow unnecessarily large.

One workaround is to avoid changing a single variable in the matfile handle. Instead, load the variables and save it into a new MAT file. The other workaround is to choose the format 7.0 by using '-v7' argument of save. Please refer to the following section:

https://www.mathworks.com/help/matlab/ref/save.html#inputarg_version

Some useful links containing the detailed explanation:

https://stackoverflow.com/questions/41526334/matlab-matfile-increases-in-size-when-overwriting-cell-data

https://support.hdfgroup.org/HDF5/doc/H5.user/Performance.html