Loading multiple (e.g. ~ 3000) mat files into workspace and merging them in a table

I have many mat files (no particular sequence such as file1.mat, file2.mat etc etc) in a folder. I want to load all of them into a workspace and then merge all the mat files in a table.

Any efficient way to do so other than loading all of them manually (which can be done if I select all the files using CTRL+A then press enter, no issues of RAM as I have 16 gigs and the files will take at best 3 gigs of RAM) and then using the table(names of variables,..........)? My main issue is i will need to list all 3000 variables from workspace to table function manually which I don't want to do.

Answer 1

[Stephen23]

" i will need to list all 3000 variables from workspace.."

That is a good example of why load-ing directly into the workspace is a bad way to write code.

Preallocate a table (exactly how depends on how you want to join the tables).

Loop over the files. Load each file into a structure, i.e.:

S = load(...)

Convert the structure to a table using struct2table:

T = struct2table(S);

Use one of the table join functions to add the new table to the preallocated table:

https://www.mathworks.com/help/matlab/tables.html?s_tid=CRUX_lftnav

[Swapnil Sayan Saha]

Certainly a better way.

Answer 2

[Swapnil Sayan Saha]

Ok, let the question be here as this problem is faced by many. Solution (works only if you have sufficient RAM)

1. Select all the mat files in the folder using CTRL + A, press enter to load all variables in workspace.
2. Open a new script
3. Select all variables from workspace using CTRL + A, copy (CTRL + C) and paste them inside the braces of table() function. For example: x = table(var1, var2, ..... varN);

If anyone comes up with a better solution (such as using datastores and tall functions without having to load all mat files in primary memory), you are welcome to share.