Speed up loading a single variable from a large mat file

I was trying to load a single variable 'data' from a large .mat file (>2GB) using the following command in MATLAB R2016b on Mac OS.

data = load('db.mat','-mat','data');

However, it was too slow to load the single variable. Note that the variable 'data' is not a matrix but a structure. I already refer to the previous discussion but it didn't give solution. Could anyone please suggest an alternative way to speed up loading a single variable from a large mat file? Thank you for your help in advance.

[per isakson]

• ">2GB" means version '-v7.3', I assume.
• '-nocompression' improves speed, but not that much
• AFAIK: '-v7.3' is slow. (I didn't test that since R2013a.)
• Try HDF5 files with 'ChunkSize' Default: Not chunked

Answer 1

[Jan]

Using a large MAT file and extracting a single variable takes time. There is no magic acceleration. The compression of the data of v7.3 files requires to decompress the complete or at least a large part of the data to extract you variable.

The clean solution would be not to use MAT files for this job.

[Rahimeh Rouhi]

I have the same problem. You are right, using Matfile is time-consuming. How can I fix the problem?

Thank you for your guidance.

[Walter Roberson]

With -v7.3 mat files there is now a -nocompression option

https://www.mathworks.com/help/matlab/ref/save.html

This applies only for -v7.3 .

It is possible to have a -v7 (not -v7.3) .mat file that is over 2 gigabytes as long as any one variable is not over 2 gigabytes. However, compression would happen automatically for -v7 mat files as well.

Sometimes what helps is to use matfile with -v7.3 files . This can help in accessing members of a struct array. Reading between the lines, it might be able to load a single field of a struct array, but anything below that would be loaded as a whole.

For pure numeric arrays sometimes it makes sense to write them as pure binary files and memmapfile() . This can also be done for struct arrays, but in each case the data written per entry must be fixed size.