MATLAB load 2nd column of all files into matrix

I have a folder with a lot of files of the form: defects_*.txt

Assuming the path to the directory containing these files is C:/Path, how can I load all the files into MATLAB, and create a matrix that contains the 2nd column of each file in the directory.

So basically, I don't care about column 1, but I want to load in the files sequentially, defects_002.txt, defects_005.txt, etc. (no constant increment), and then take the 2nd column from each of these txt files, and paste them as separate columns in a new matrix. So I will have a new matix where column 1 is the 2nd column of defects_0.002.txt, column 2 is the 2nd column of defects_0.005.txt etc. Is this possible?

[Christopher Wallace]

What is contained in the defect_*.txt files?

[Benjamin Cowen]

@ChristopherWallace Just 2 columns of data, with 1 header row. I just care about the 2nd column of data, not the header row though

Answer 1

[Walter Roberson]

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

Reading only the second column can be done in a number of different ways, depending upon the format of the data. One possible way is to use csvread with the r and c parameters both set to 1 -- the r being 1 to skip 1 header row, and the c being 1 to skip 1 column.

[Benjamin Cowen]

unfortunately i tried using the code from that site already but never could get it to work

[Christopher Wallace]

Please post a sample of the data you're trying to parse.

[Benjamin Cowen]

example attached

[Walter Roberson]

projectdir = 'C:/Path';
dinfo = dir( fullfile(projectdir, 'defects_*.txt') );
filenames = fullfile(projectdir, {dinfo.name});
nfiles = length(filenames);

data = [];
for K = 1  : nfiles
   thisdata = csvread( filenames{K}, 1, 1);  %skip 1 row and 1 column
   data(1:length(thisdata), K) = thisdata;
end

Note: if the files do not have an equal number of columns, then this will zero pad each short row. Other paddings such as nan are possible with a few more lines of code.

[Benjamin Cowen]

hmmm, i get an error

   Index exceeds matrix dimensions.

Error in dlmread (line 152)
result = result{1};

Error in csvread (line 48)
    m=dlmread(filename, ',', r, c);

Error in defects_v2 (line 7)
   thisdata = csvread( filenames{K}, 1, 1);  %skip 1 row and 1 column

[Walter Roberson]

projectdir = 'C:/Path';
dinfo = dir( fullfile(projectdir, 'defects_*.txt') );
filenames = fullfile(projectdir, {dinfo.name});
nfiles = length(filenames);

data = [];
for K = 1  : nfiles
   fid = fopen(filenames{K}, 'rt');
   thisdata = cell2mat( textscan(fid, '%*f%f', 'HeaderLines', 1) );
   fclose(fid);
   data(1:length(thisdata), K) = thisdata;
end