How to access the variables with similar name in different mat files in sub-folders

Hi all: I have a sequence of six hourly time series data sets per each day (ws722.mat....ws1017.mat) in 88 subfolders (each subfolder has wsxxx.mat) in the home directory ('C:\DST_Duminda\field_data'). Each wsxxx.mat file (for example ws722.mat) has the four variable names (sw12am,sw6am,sw12pm,and sw6pm). each variable is a 128*6*3 3D matrix. I want to start with sw12am (ws722.mat) which is 128*6*3 and keep adding all the variables (sw6am,sw12pm like wise) sequentially in all the subfolders to generate 128*6*3*352 - which is a 4D matrix. I would appreciate if someone can help me with this. I need to automate the process starting from 1st subfolder to the end. Thanks in advance

[Stephen23]

This would be easier if you load into a variable, instead of straight into the workspace. See:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236127

Answer 1

[Steven Lord]

You should call load with an output argument then retrieve the appropriate data from the struct array load returns. For example, using one of the demo files included with MATLAB:

% I'll explain why I created a variable named map at the end of this example.
map = 1:10;

% Load the variables from the file into a struct array named data
data = load('durer.mat');

% What variables were in durer.mat (each became a field of the struct array data)
variablesInDurer = fieldnames(data);

% Extract the variable named map from the loaded data using dynamic field names
valueOfMapFromDurer = data.('map');
valueOfMapFromDurer2 = data.(variablesInDurer{2});

% Look at the properties of the variable extracted from the struct
whos valueOfMapFromDurer valueOfMapFromDurer2

% Compare with the properties of the variables stored in the MAT-file
whos -file durer.mat

% And look at the properties of the variable I created at the start of the example
whos map

The two variables valueOfMapFromDurer and valueOfMapFromDurer2 are the same size as the map variable in the file.

When you use this technique, when you load the data into the struct array you don't overwrite any variables in the workspace that have the same name as a variable in the file. In the example code, even though the map variable in the MAT-file was of size [128 3], the map variable in the workspace (that I created at the start of the example) is still of size [1 10]. The load call didn't overwrite it.

Answer 2

[Chris Perkins]

Hi Duminda,

Here's a quick example of loading files as you describe and combining the data into one new array.

I scaled-down some of the sizes so it was easier to work with, but you should be able to expand this example to fit your specific problem.

% Final 4D Matrix, updated throughout
finalMatrix = ones(10,6,3,1);

% Counter to track how many elements we have added to the final array so far
counter = 1;

% Generate a list of the files and folders in your current directory
subFolders = dir;
subFolders = {subFolders(:).name};
% remove '.' & '..' if they exist
subFolders(ismember(subFolders,{'.','..'})) = []; 

% Loop through all folders
for i = 1:numel(subFolders) 
     % Get the current folder name
    currentFolder = subFolders{i};

      % If the current item is a file, not a folder, skip it
      if ~isfolder(currentFolder)
          continue;
      end

      % List folder contents
      d = dir(currentFolder); 

      % Extract list of file names from folder
      fileNames = {d(:).name}'; 
      % remove '.' & '..' if they exist
      fileNames(ismember(fileNames,{'.','..'})) = []; 

      % Loop through each MAT file in the current sub-folder 
      for j = 1:numel(fileNames)
          % Load single MAT file
          fullFileName = strcat(currentFolder,'/',char(fileNames(j)));
          load(fullFileName);

          % Add Data to final matrix
          finalMatrix(:,:,:,counter) = sw12am;
          counter = counter +1;
          finalMatrix(:,:,:,counter) = sw6am;
          counter = counter +1;
          finalMatrix(:,:,:,counter) = sw12pm;
          counter = counter +1;
          finalMatrix(:,:,:,counter) = sw6pm;
          counter = counter +1;
      end
  end

Note: This assumes that each MAT file will have all 4 variables as you describe them. If 'sw12am', 'sw6am', 'sw12pm', and 's26pm' are not in every single file, you will need to adjust the logic when adding the data to the final matrix.

Good luck!

[Duminda Vidana Gamage]

Hi Chris Thank you for answering the question. i need to clarify one thing. is the code loading a single mat file or all mat files in a subfolder? because i have several .mat files in a subfolder but need to load only 'swxxx.mat' which contains the 4 variables (sw12am.....sw6pm). For example in subfolder 1 have sw722.mat and subfolder 2 'sw723.mat'. i would appreaciate if you clarify this. thanks Duminda