Store .csv data and calculate average value

Goodmorning to everybody.

Can someone help me to understand how I can save in matlab 10 .csv files, select only the columns in which I am interested and get as output a final plot in which I have the average value of the y coloumns and standard deviation of y axes? I am not so good in matlab and so I kindly ask if someone to help me to solve this question.

Thanks in advance.

Answer 1

[Walter Roberson]

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

Roughly,

which_column = 7;   %for example
for K = 1 : 10
   filedata = csvread(FileNames{K})
   columndata(:,K) = filedata(:,which_column);
end
columnavg = mean(columndata,2);
columnstd = std(columndata,2);
xvals = 1 : size(columnavg,1);
plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');

This would plot the average and would also plot the positions that are 1 standard deviation above and below the average.

[Snapshot83]

I get this error:

>> Untitled

Undefined variable "FileNames" or class "FileNames".

Error in Untitled (line 3)
   filedata = csvread(FileNames{K})

Another question is: In "which_column" I need to add my y axe? The coloumns I am interested are the 4th (x axe) and the 5th (Y axe)

[Walter Roberson]

You would store the names of the csv files as a cell array of strings in FileNames . Those names might possibly be found by using dir() like is shown in the link. For example,

FileNames = {'first.csv', 'File02.csv', 'data3good.csv', '004-very-important.txt'};

or

dirstats = dir('*.csv');
FileNames = {dirstats.Name};

Are you x axis values (column 4) the same for all of the files? If not then there is more work to do in order to form the average. In the code above you would assign which_column=5 to average the Y values.

[Snapshot83]

X axis values are the same for all the files. Can kindly tell me where I need to put this two lines? Just after the declaration of k? (after K = 1 : 10 ?)

If I write the code on this way

which_column = 5;   
for K = 308:317
   dirstats = dir('*.csv');
   FileNames = {dirstats.Name};
   filedata = csvread(FileNames(K))
   columndata(:,K) = filedata(:,which_column);
end
columnavg = mean(columndata,2);
columnstd = std(columndata,2);
xvals = 1 : size(columnavg,1);
plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');

I obtain this error:

>> Untitled
Reference to non-existent field 'Name'.
  Error in Untitled (line 4)
     FileNames = {dirstats.Name};

[Walter Roberson]

Assuming that you are implying that the names are 308.csv 309.csv up to 317.csv

x_column = 4;
y_column = 5;
file_range = 308:317;
for K = 1 : length(file_range)
   filename = sprintf('%d.csv', file_range(K));
   filedata = csvread(filename)
   columndata(:,K) = filedata(:,y_column);
end
columnavg = mean(columndata,2);
columnstd = std(columndata,2);

%since the x are all the same we can use the data from the last file we read
xvals = filedata(:,x_column);

plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');

[Snapshot83]

Hi. Yesterday was able to running the following code but today I have this error and I don0t know how I can solve: The error is:

Attempted to access num(:,3); index out of bounds because size(num)=[0,0].

Error in csvBatchRead (line 10)
   col3=num(:,3);

Instead, the code I used is:

clear all;
clc;
which_column = 5; 
dirstats = dir('*.csv');
col3Complete=0;
col4Complete=0;
for K = 1:length(dirstats)

     [num,txt,raw] = xlsread(dirstats(K).name);
     col3=num(:,3);
     col4=num(:,4);
     col3Complete=[col3Complete;col3];
     col4Complete=[col4Complete;col4];
     avgVal(K)=mean(col4(:));
  end
  col3Complete(1)=[];
  col4Complete(1)=[];
  %columnavg = mean(col4Complete);
  %columnstd = std(col4Complete);
  % xvals = 1 : size(columnavg,1);
  % plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');

B = reshape(col4Complete,[5000,K]);
m=mean(B,2);
C = reshape (col4Complete,[5000,K]);
S=std(C,0,2);

[Walter Roberson]

One of your csv files has no numeric values.

[Snapshot83]

I Checked, all the files have numeric values

[Walter Roberson]

At the MATLAB command line, give the command

dbstop if error

then run the program and wait for it to fail. When it fails, look at dirstats(K).name which will be the name of the current file. You can look at size(num), size(txt), size(raw)

I speculate that one of your .csv files is empty.

[Snapshot83]

Actually If I open the matrix K, it has only one value (1).

[Walter Roberson]

K is not intended to be a matrix. K is a loop counter. You are having problems on the first file. Show us

dirstats(K).name 
dirstats(K).bytes
size(num)
size(txt)
size(raw)

[Snapshot83]

Now it works, thanks. My basic error was that I didn't delete an old file inside the folder so I didn't process properly my data.

I have a second problem. How I can copy this data in an excel file properly? I used the function xlswrite but maybe something is wrong. If I write

xlswrite('file1.xlsx', m),

for exemple, I am not able to see the file created inside the folder.