Extract data using variable names

Hello Matalab Community.

Let me ask your help to find out a way to improve porcessing my data.

I have files with lots of variables, data Type is structure.

I want to analyze data, let's say I want to review VSPD so I used below commadns:

%extract data, convert to double 
V_Time = VSPD.time;
V_VSPD = VSPD.signals.values;

Now I can easly use double data to plot it or do something else.

Becasue I got several files to go through and saveral variables, I'm trying to create a code to make this process faster:

clear all;              %   Clears all memory
clc;                    %   Clears the command window screen

fprintf('select folder containing data files\n');
currentpath=pwd; %%Identify current folder

[FileName,PathName] = uigetfile('@*.mat','Select the .dat-file(s)','MultiSelect','on');

if (PathName == 0)
    fprintf('Data directory was not selected...script will be terminated\n\n');
    return
end

cd(PathName); 
files=dir('@*.mat'); 
num_files=length(files(:,1)); 

for i=1:num_files
    fprintf('Progress: %d/%d\n\n',i, num_files)
    load(files(i,1).name)
end

FileName = files.name;
PathName = files.folder;

 if isnumeric(FileName)
    return
end
FileName = cellstr(FileName);
V = { ...
'VSPD'...
    };
N = numel(FileName);

my goal is to create a list of the variables that I want to use(extract) for my analysis in this case V

Then use V to somehow create V_Time = VSPD.time and V_VSPD = VSPD.signals.values;

A = cell2mat(V);
B= '.time';
C= '.signals.values';

AA = [A C];
BB = convertCharsToStrings(C);
CC = BB;

%%V_Time = A + C   ?????

any feedback will be highly appreciated

Answer 1

[Stephen23]

https://www.mathworks.com/matlabcentral/discussions/tips/849901-tutorial-why-variables-should-not-be-named-dynamically-eval

The most important change is to always LOAD into an output variable (which is a scalar structure):

S = load(..);

Also avoid CD in code (using absolute/relative filenames is more robust and much more efficient).

Why are you using UIGETFILE to select files... which you then completely ignore by using DIR to match all filenames? It makes your intent very unclear. Using UIGETDIR makes the intent much clearer.

P = '.'; % uigetdir();
if isnumeric(P)
    fprintf('Data directory was not selected...script will be terminated\n\n');
    return
end
S = dir(fullfile(P,'@*.mat'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    D = load(F);
    C = struct2cell(D);
    S(k).meta = fieldnames(D);
    S(k).data = vertcat(C{:});
end

The file data are all stored in the structure S which is trivial to access in a loop, for example the 1st file:

S(1).name; % filename
S(1).meta; % VSPD, etc.
S(1).data; % file data

If all of the imported data structures have the same fields, then you can easily concatenate them together:

C = vertcat(S.meta)

C = 8x1 cell array
    {'VSBART'      }
    {'VSBART_RT'   }
    {'VSBART_RT_PD'}
    {'VSBART_RT_PU'}
    {'VSFDC'       }
    {'VSPD'        }
    {'VS_RATE'     }
    {'VS_RATEPH'   }

D = vertcat(S.data)

D = 8x1 struct array with fields:
    Channelname
    Channeltype
    unit
    timeunit
    period
    device
    time
    signals
    min
    max

which makes accessing your data trivial a loop, for example the 1st data structure:

T = D(1).time;
V = D(1).signals.values;
plot(T,V)

The best approach by far would be to design the data better, without any meta-data in the variable names.

Forcing meta-data into variable names makes processing data slow and fragile. Best avoided.

[Rodrigo]

I didn't know I can do it this way, still leraning....

Thank you for taking time to answer and for sahring,

[Rodrigo]

hello Stephen23.

let me ask one more thing.... what if I want to use a loop so I can grab all data from different files and put them in to a single vector.

I made a copy of the data preiously shared and I processed them with the feedback you gave me.

D will include VSPD signal which is a 22555x1 double data.

and because we have same data twice D will be 45 110 struct which include VSPD 22555x1 double data twice.

my goal now is to create a loop so I can extract data from each data file and combine it into a single vector so I did this:

for ii = 1:length(D)
    if D(ii).Channelname == "VSPD";
    VV = D(ii).signals.values;
    end
end

the loop is working but i'm getting only half data

VV = 22 555x1 double

any recoomendation?

as always any feedback will be higly appreacited

[Stephen23]

P = '.'; % uigetdir();
S = dir(fullfile(P,'@*.mat'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    D = load(F);
    C = struct2cell(D);
    S(k).meta = fieldnames(D);
    S(k).data = vertcat(C{:});
end
C = vertcat(S.meta)

C = 16x1 cell array
    {'VSBART'      }
    {'VSBART_RT'   }
    {'VSBART_RT_PD'}
    {'VSBART_RT_PU'}
    {'VSFDC'       }
    {'VSPD'        }
    {'VS_RATE'     }
    {'VS_RATEPH'   }
    {'VSBART'      }
    {'VSBART_RT'   }
    {'VSBART_RT_PD'}
    {'VSBART_RT_PU'}
    {'VSFDC'       }
    {'VSPD'        }
    {'VS_RATE'     }
    {'VS_RATEPH'   }

D = vertcat(S.data)

D = 16x1 struct array with fields:
    Channelname
    Channeltype
    unit
    timeunit
    period
    device
    time
    signals
    min
    max

X = strcmpi({D.Channelname},'VSPD');
Y = [D(X).signals];
Z = vertcat(Y.values)

Z = 45110×1

0 0 0 0 0 0 0 0 0 0

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Rodrigo]

great, thanks a lot