how to average multiple folders with multiple files. I am attaching three folders with two files each. I need a help to find an average of two files from three folders. My code is showing an error"Dot indexing is not supported for variables "

close all
clear all
D = 'D:\filename\';
S = dir(fullfile(D,'*'));
N = setdiff({S([S.isdir]).name},{'.','..'}); % list of subfolders of D.
for ii = 1:numel(N)
    T = dir(fullfile(D,N{ii},'*')); % improve by specifying the file extension.
    C = {T(~[T.isdir]).name}; % files in subfolder.
    for jj = 1:numel(C)
        F = fullfile(D,N{ii},C{jj})
                fprintf('test%s\n',F);
     S(ii).data = F(:);
    end
end
Y = cat(3,S(:))%this line is wrong
out = mean(Y,3)

[Ameer Hamza]

You dataset contains several columns. How do you want to average them?

[MS]

Thanks. I need two files(test_001.txt, test_002,txt) from all three folders(out of six files) finally. let me know if you need further clarifications.

[MS]

code through N(here 3 folders) different folders with M(two files here) different.txt files(text​001.txt...​..text002.​txt) with (O rows and P columns). and find the average(te​xt001.txt.​....text002.txt)of M different files from the N folders.

Answer 1

[Ameer Hamza]

Create a script in the same directory where you have folders T1, T2, and T3. Then paste the following code in that script and run it

files = dir('**/*.txt');

data = cell(1, numel(files));
for i=1:numel(files)
    filename = fullfile(files(i).folder, files(i).name);
    data{i} = readmatrix(filename);
end

M = cat(3, data{:});
average_val = mean(M, 3);

[MS]

Thanks for the answer. Your code does the average of all six files all together. But, I need you to help me to get avearge of test_001.txt from all three folders and text_002.txt for all three folders and give it as two diffrent files(avg_value1.txt, averagevalue2.txt.).

[Ameer Hamza]

In that case, try this

files = dir('**/*.txt'); % get names of all the files recursively.
% get the location of current directory to exclude all the files in it
current_dir = pwd;
idx = strcmp(current_dir, {files.folder});
files(idx) = []; % remove the files in current directory from the list
% place all the files with same name in a group
grps = findgroups({files.name});
filegrps = splitapply(@(x) {x}, files, grps'); 
avg_mat = cell(1,numel(filegrps));
for i=1:numel(filegrps)
    grp = filegrps{i};
    data = [];
    for j=1:numel(grp)
        filename = fullfile(grp(j).folder, grp(j).name); % construct the full path of the files
        if isempty(data)
            data = readmatrix(filename); 
        else
            data = data + readmatrix(filename); % read the matrix and add to previously available values
        end
    end
    avg_mat{i} = data/numel(grp); % calculate the aerage.
end
for i=1:numel(avg_mat)
    figure();
    axes();
    hold on
    view(3)
    x = avg_mat{i}(:,1);
    y = avg_mat{i}(:,2);
    u = avg_mat{i}(:,3);
    v = avg_mat{i}(:,4);
    plot3(x, y, u, '+');
    plot3(x, y, v, '*');
    legend({'u', 'v'});
    writematrix(avg_mat{i}, ['avg_val_' num2str(i)]);
end

[Ameer Hamza]

OP's comment moved here:

Many thanks. This code is givng me four files instead of two files for some reason, Can you please check this.

[Ameer Hamza]

Check the update code. It will only create 4 files after you run the code 2nd time, because new there are two new txt file. I have updated the code to exclude those files.

[MS]

Thank you very much. your code is working well.

[Ameer Hamza]

Glad to be of help.

[MS]

thanks. is there way to save all the average files output as mat files. Sorry, i am asking too much. kindly let me know if you can.

[Ameer Hamza]

Yes it can be done by just changing the last for loop

for i=1:numel(avg_mat)
    Avg_mat = avg_mat{i};
    save(['avg_val_' num2str(i)], 'Avg_mat');
end

[MS]

Thanks, can you suggest me a loop to save as any type of files eg., .dat, etc.,

[Ameer Hamza]

You can add as many line as you want like this:

for i=1:numel(avg_mat)
    Avg_mat = avg_mat{i};
    name = ['avg_val_' num2str(i)];
    
    writematrix(Avg_mat, name); % .txt
    save(name, 'Avg_mat'); % .mat
    dlmwrite([name '.dat'], Avg_mat); % .dat
end

[MS]

you are awesome. I request you to sugget me a book or any resources to learn these basics eg., looping, handling various type of files, manipulating the variables etc.,. I am a begineer to any programming.

[Ameer Hamza]

Maybe try this: https://www.mathworks.com/learn/tutorials/matlab-onramp.html. This is a tutorial created by Mathworks and gives a brief overview of basics on MATLAB.

[MS]

Thank you.

[MS]

@Ammer Hamza, I request you to please check and modify the code. The current code is giving the answer as first folder files instead of averaging the three folder files.

[Ameer Hamza]

There were a few mistakes in my code. Please check the updated code in the second comment of this answer.

[MS]

Many Thanks. you have been really helpful to me. Now, the code works fine. I request you to comment the code. It will be helpful for everyone to learn from your thought process. if you can Please add a code to plot the average output files.

[Ameer Hamza]

Please see the updated code. there are four column in your matrix. It will plot each column as a seperate line.

[MS]

Thanks for commenting the code . The first two columns of the files are x,y axis and the next two columns are the data. is there a way to mention it in the code and plot it.

[Ameer Hamza]

Can you show an example of how you want to plot your columns?

[MS]

Thanks, plot the figure as mentioned below

first columsn = x axis

second columsn = y axis

third column = u(x,y)

fourth column = v(x,y)

[Ameer Hamza]

See the updated code.

[MS]

Thanks. Sorry for the late resposne.