Can I create a code to avoid updating the struct every time I use a different struct?

Hi all, and thanks in advance. I have 1x1 struct files which have 65 fields, some fields contain a single number, and others can be opened up and has about 80 cells worth of data. These are files produced by a piece of research equipment, and the name relates to the time the readings were taken, and are unable to be changed.

Is there a way I can easily extract data from specific fields? In every copy of the file the field names remain the same (i,e, Pressure, Compliance etc). Some of these will be single figures, others will be the ones with cells worth of data.

I have hobbled together something which works, but trust me...its bad, I have no idea what I am doing.

While it kind of works, I have to change the struct name in the code every time I open a new file. Is there a way I can avoid having to update the struct code every time? (180+ times a day).

Thanks in advance.

Matt

[KSSV]

Why you have to change struct name? There is no necessity according to my knowledge as of now. How you have the data? In a mat file? Each structure into one file? or all structures in one file?

[Stephen23]

@Matthew Wilson: most likely you just need to load your .mat files (if that is what you have, your explanation was not very clear) into an output variable. Then you can easily access its fields.

If you want more help with this then please upload some sample files by clicking the paperclip button.

[Matthew Wilson]

The files our equipment spits out are '.m', a 'MATLAB Code' file if that helps. So all the files I have that I wish to use/mentioned above are .m files.

[Stephen23]

"These are files produced by a piece of research equipment,..."

"The file is a '.m', a 'MATLAB Code' file if that helps."

Seriously, some special equipment writes code files to store data? Why would anyone do that, when there are plenty of perfectly good data file standard in existence?

Oh well, if this equipment's code really is so badly designed and implemented then so be it. Please upload some example files by clicking the paperclip button. Otherwise we have absolutely no idea how that code is structured, or even if it is a script or a function or a class ...

[Matthew Wilson]

So the piece of equipment has a GUI and printout that provides all the information clinicians need, and then some! It does what its designed to do in a fantastic way. Patient data/results seem to be stored within the program I think, as when you remove the .m file the results are still viewable.

The .m file is produced for research purposes only. From a research perspective this is the only way we can get the raw data; so it is designed from a clinical perspective, less so a research perspective.

I have attached a file :) (dummy patient)

Thanks again everyone.

The sections I am interested in are something like: EQUIVALENT_EAR_VOLUME, RESONANCE_FREQUENCY, PEAK_PRESSURE, COMPLIANCE_226_Hz, GRADIENT_226_Hz, PEAK_PRESSURE_226_Hz (all in one row if possible)

then in another row ABS_PEAK and in another row ABS_AMBIENT

[Walter Roberson]

In the code I posted you would use

filename = 'File_WB3DT_2018_10_12__16_10_32.mat'

and you would change the list of character vectors to include EQUIVALENT_EAR_VOLUME, RESONANCE_FREQUENCY, PEAK_PRESSURE, COMPLIANCE_226_Hz, GRADIENT_226_Hz, PEAK_PRESSURE_226_Hz and so on.

The code I posted creates a struct with just those fields. You could easily put the values into a cell array instead. If the values are all numeric and all the same size you could put them into a numeric array. The disadvantage of the cell array or numeric array is that you have to plan out the order you want the fields in, and since the fields are about quantities that are not just values related in time, it might make more sense to use them from a struct so you can extract by name with clear references instead of code that just has to know which offset corresponds to which value.

[Matthew Wilson]

Would I need to change the filename = code for each file? As every patient produces four files (two different types of test, and left/right ears tested).

[Walter Roberson]

Use dir to find the files.

dinfo = dir('FILE*.mat');
filenames = {dinfo.name};
for K =1 :length(dinfo)
 filename = filenames{K};
....
end

[Stephen23]

@Matthew Wilson: the data is not saved in a .m file, when you look at the file you supplied the very first code line is clearly load-ing the data from a .mat file:

saveVarsMat = load('File_WB3DT_2018_10_12__16_10_32.mat');

"Patient data/results seem to be stored within the program I think"

Once the data are load-ed from the .mat file they are stored in MATLAB's memory, just like every other variable in MATLAB's base and function workspaces:

https://www.mathworks.com/help/matlab/matlab_prog/base-and-function-workspaces.html

Read this to load the data from a sequence of .mat files:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

[Matthew Wilson]

Anything I've done thus far has kind of worked because I saved my one file as a .mat file, but it won't pick up any of the files I have saved as .m In reality they'll likely be batched together per patient and I've worked out how to save them all within one .mat file (attached an example of a dummy patient). Would it make it any easier if it was saved like this? Maybe then the code suggested on this page using .mat would pick up the files too :) I know the code says to open the .mat but it doesn't pick them up while they're in .m format it seems? (Clueless, so I may be wrong).

Matt

[Stephen23]

"Would it make it any easier if it was saved like this?"

It depends on what you mean by "easier". It certainly won't make your code or your processing any simpler, because your either have to loop over filenames or fieldnames, and both of those require about the same amount of code. If the separate .mat files already exist, then I would not bother combining them into one.

" Maybe then the code suggested on this page using .mat would pick up the files too"

What code you referring to? You have received several answers. The code I wrote, tested, and showed in my answer already loads your .mat files, so it is completely unclear what you mean.

" I know the code says to open the .mat but it doesn't pick them up while they're in .m format it seems?"

A .mat file cannot be "in .m format", whatever that means. Your data are saved in .mat files. That is what the code in my answer loads. What do .m files have to do with anything?

If by "but it doesn't pick them up" you mean that the data is not being loaded from the file into MATLAB (your terminology is very vague) then the best solution is to find out why:

• how are you checking if the file data has been loaded?
• are the data files saved in the current directory?
• what does exist return when you try to locate one of the .mat files?

Please show us the exact code that you tried using.

[Matthew Wilson]

S = dir('*.mat');
for k = 1:numel(S)
    S(k).data = load(S(k).name);
end

>> S = dir('*.mat');
for k = 1:numel(S)
    T = load(S(k).name);
    T = struct2cell(T);
    S(k).data = T{1};
end
>> S(1).data.PRESSURE
Reference to non-existent field 'data'.

and in the workspace it says k [] S 0x1 struct

If I have a .mat file in the folder (if I type save for instance, which gave me the file I attached recently) then I get:

k 1 S 1x1 struct T 2x1 cell

That is why I was wondering if it was picking them up or not (more likely though, I've missed a step somewhere).

Matt

[Stephen23]

@Matthew Wilson: it means that the dir command did not find any .mat files in the current directory. You can fix this by telling your code which directory to look in:

D = 'path to where your .mat files are saved';
S = dir(fullfile(D,'*.mat'));
for k = 1:numel(S)
    F = fullfile(D,S(k).name);
    T = load(F);
    T = struct2cell(T);
    S(k).data = T{1};
end

Note that if no files are found then S will not contain the data field and it will be empty, just as you have. The number of elements in S is simply how many files were matched by dir (empty means no files found). Only when the files are load-ed in the loop is the data field added to the structure.

You can either write the folder D yourself, if you know where the files are saved, or browse to the folder and copy-paste from the path from your OS file browser. It can be an absolute or relative path.

[Matthew Wilson]

I still get S 0x1 struct with that, but if I change *.mat to *.m then I get S 4x1.

All the files are the .m format of the very first file I uploaded if that helps.

Again thank you for your help and your patients (and Walter too!)

Matt.

[Stephen23]

"All the files are the .m format of the very first file I uploaded if that helps."

If that is true then if you look inside every single one of those .m files you will find that they all load some .mat file/s. Those .mat files are where you data are actually saved.

Find those .mat files. Their location is what you need to find out. Where are they on your computer?

[Matthew Wilson]

We only take the .m files from the computer. If you drag and drop the .m file I posted earlier into the command window does it open? Thats what we have been doing to get to the data.

[Stephen23]

@Matthew Wilson: you have said several times that you use those .m files to load your data. What I am trying to do is show you how to load your data from the .mat files where it is actually saved.

Your question asked "Is there a way I can avoid having to update the struct code every time?", and that is what I showed you in my answer and comments. To make it work you will have to find the folder where those .mat files are saved. They are saved on your computer, not mine. All OS's have search functions, so I am sure that you can locate them.

[Matthew Wilson]

This file here, if you drag and drop it to the command window does it show anything?

We don't take anything but the .m file from the computer used to run the tests, so the computer I use for Matlab only has a USB stick full of these .m files.

[Walter Roberson]

That is an executable file that can be used to create data. But it is completely different than the earlier .m you posted which consists of doing a load() of a .mat . The ones that do the load() need the mat files, but the one like you just posted do not.

[Matthew Wilson]

Oh OK, not sure why the first was different, it had been opened in Matlab previously but this one has not, so this one here is probably the right type then. Apologies for any confusion and trouble. Are these similar to .mat files to work with, just need to modify some of the code? Or quite different.

[Matthew Wilson]

Would it be best to start a new question now that I have more of an idea of what I need/what I have and what to ask? This has all become quite messy and confusing for me :(

[Stephen23]

"Are these similar to .mat files to work with, just need to modify some of the code? Or quite different."

The .m file you posted earlier is totally unrelated to everything that you have uploaded previously on this thread. Answering a question is very difficult when the information keeps changing.

"Would it be best to start a new question now that I have more of an idea "

You could, but based on your most recent .m file the solution will not be particularly simple or neat. Basically the tool that you are using is very badly written because its author has forced meta-data (the date) into the structure's name. Forcing meta-data into a variable name makes accessing that variable slow, complex, and buggy. Read this to know why:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

The best solution would be to fix the tool that you are using. Otherwise you will have to run that file, then magically try to access the variable that it creates:

save('temp.mat','-regexp','File_.*_\d{4}_.*')
T = load('temp.mat');
C = struct2cell(T);
A = C{1};
A.PRESSURE
A.FREQ
A.ABSORBANCE
... etc

I recommend getting the tool fixed. If you upload it here or give us a link we might be able to help with that.

Answer 1

[Walter Roberson]

datastruct = load(filename) ;
FN = fieldnames(datastruct) ;
selected_data = SelectData(datastruct.(FN{1});

function selected = SelectData(S)
   fns = {'Pressure', 'Compliance', 'etc'} ;
    for K = 1 :length(fns)
      fn = fns{K};
      selected.(fn) = S.(fn);
     end

Answer 2

[Stephen23]

If each mat file contains only one variable (but its name changes and you don't know the name in advance) then you could just use struct2cell to get the data of that variable. If there are multiple variables in the .mat file then this method can be modified, but what modifications are most suitable depends on how the variables are named, which you have told us nothing about.

Here is a simple method that will load your .mat file/s (which is actually how your data is saved), without knowing the name of the structure inside the .mat file/s.

One Single File

T = load('WB3DT_2018_10_12__16_10_31.mat');
C = struct2cell(T);
A = C{1};
A.PRESSURE
A.FREQ
A.ABSORBANCE
... etc.

Multiple Files

S = dir('*.mat');
for k = 1:numel(S)
    T = load(S(k).name);
    C = struct2cell(T);
    S(k).data = C{1};
end

And then you can access the imported structure simply like this, e.g. here is the data for the first file:

S(1).data.PRESSURE
S(1).data.FREQ
S(1).data.ABSORBANCE
... etc.

[Stephen23]

@ Matthew Wilson: now that you have actually uploaded a sample data file, it was easy to test my code. Here are some of the values that it imported:

>> S(1).data.RESONANCE_FREQUENCY
ans =  483
>> S(1).data.PEAK_PRESSURE
ans = -5
>> S(1).data.PRESSURE(:)
ans =
   178
   178
   157
   148
   136
   123
   106
    90
    74
    59
    45
    31
    18
     5
    -5
   -16
   -27
  ... lots of rows here
  -330
  -333
  -335
  -338
  -340
  -344
  -349
  -355
  -361
  -367
  -372
  -375
  -379
  -384
  -389
  -395

Note how the code in my answer does not use the structure name anywhere (which is exactly what your question said that you wanted to acheive).

Answer 3

[Matthew Wilson]

It seems this is all unfortunately beyond my skill. I'll just continue to do extract the data manually. Thank you all for your help.

Matt

[Stephen23]

@Matthew Wilson: if you upload some sample .mat files then I can show you how to import them in a loop. Well, actually my answer already shows you that, and it only takes four lines of code.

[Matthew Wilson]

I've uploaded one of the files the equipment provides above, a .m file, unfortunately I only have a different test (so different measurements), these are the only dummy files I have that can be uploaded.

[Stephen23]

Either:

• crate a .mat file that has the same properties as the one loaded by your .m file, but using some fake data, and upload that.
• clearly describe how the .mat file, e.g. how many variables it contains, etc. For this you can use whos and simply show us what is displayed.

[Matthew Wilson]

This one here (the one I loaded previously I think) is what the device gives us. They will all be similar to this, hope that helps.

So far I have been dragging/dropping it into MATLAB and then going into each field and copy/paste the data from there.

[Stephen23]

@Matthew Wilson: please open that .m file. Look at the first line of code. It looks like this:

saveVarsMat = load('WB3DT_2018_10_12__16_10_31.mat');

That load command imports data from a .mat file into MATLAB. That .mat file is where your data is saved. The .mat file's name is WB3DT_2018_10_12__16_10_31.mat. I asked for that .mat file, or something like it, or a description of it (e.g. using whos). Please provide the requested information.

[Matthew Wilson]

Hoping this is what you mean :)

[Stephen23]

"Hoping this is what you mean :)"

Yes, that is the data file.

Note that my answer shows you how to import and access the data in your .mat files without using the structure name, which is exactly what you asked for in your question: "I have to change the struct name in the code every time I open a new file. Is there a way I can avoid having to update the struct code every time?"

Try the code in my answer. When I tested it, it does what you requested. If you have any questions, please make a comment to my answer.

[Matthew Wilson]

Thank you so much, all of you! I feel like I'm so close to something which will be great! Is it possible to call on multiple fields, say for instance:

ABS_PEAK, ABS_AMBIENT, EQUIVALENT_EAR_VOLUME, RESONANCE_FREQUENCY

and have these come up in one spreadsheet page (like when you double click on one of the box things in the work space) so that way I can copy rows or columns into their respective fields in a spreadsheet?

Again can't thank you guys enough, this will help us with our research more than I can even imagine.

Matt

[Walter Roberson]

If you use the code I posted, adjusting for the fields you want, then afterwards you can use struct2table and then writetable