function who seems slow

Hello,

I am using the MATLAB function 'who' to assess which variables are present in a .mat file. Basically something like this:

fileInfo = who('-file','filename.mat'); 

To return if the variable is inside the file or not, I use something like this:

ismember('my_variable',fileInfo)

My .mat file is about 11Gb in size and the 'who' function takes a very long time to evaluate (about 5min). Is there a faster way to get the variable names from a .mat file?

The .mat files are v7.3.

Regards,

Kenny

Answer 1

[David Saidman]

So its a bit messy, but I just tried this and its pretty much instant regardless of size using low level H5 libraries. Hope it works for you. I've tried on windows 2017b, havent tried in linux or other releases.

Note: This solution uses low level HDF5 libraries that are already built into matlab, so this method assumes your mat file is HDF5 (-v7.3). Otherwise it will not work.

I've tried checking 15 variables in a 3Gb file, worked basically instantly (didnt time it, fast enough for my needs).

You can be sure is a valid hdf5 file by doing this:

isValidHDF = H5F.is_hdf5('my_file.mat');

Then, if ValidHDF is true, check the variable is in the file by:

isThere = false; %Initialize as default value of false
fid = H5F.open('myfile.mat') % Use low level H5F builtin to open
try % Never use try/catch when possible but this is a good for when its ok, maybe someone can suggest alternative?
     % Try to open the h5 group. 
     % Will error and catch to report back false if the variable isnt there
     % Otherwise the variable exists
     gid = H5G.open(fid,'/data_info'); % Note the "/". It required for H5 syntax and its OS independent (never "\" even in windows)

     % I think this makes sure the variable isnt empty if the group opened successfully, 
     % you might not need this bit but has been working for me. Otherwise
     % can just do if gid > 0
     hInfo = H5G.get_info(gid); 
     isThere = hInfo.nlinks > 0; 
     H5G.close(gid);
end
H5F.close(fid);

[Kenny Nona]

Works like a charm! With 'who' I got an execution time of 32sec, while with this code, using a for loop over all variables to check: 0.03sec! Thank you!

Answer 2

[Cameron B]

Not sure, but maybe one of these is faster

load('filename.mat')
exist('my_variable')

or

load('filename.mat','my_variable')

[Kenny Nona]

I did a quick test on a (smaller) file. Running the 'who' command:

tic
variableInfo = who('-file', 'filename.mat');
toc

Elapsed time is 47.116613 seconds.

While with load:

tic 
load('filename.mat','my_variable')
toc

Elapsed time is 46.814521 seconds.

So, the load command is indeed (a bit) faster in my case, but still not nearly as fast as when you click on a mat file in the folder browser and within seconds you see its contents.