Importing a foler of CSV files into matlab

Hi, I am trying to import a folder (groupB) which contains a large number of csv files named group01.csv, group02.csv and so on but the code I am inputting just returns a blank matrix. Here is my code:

JumpFiles = dir('E:\groupB/*.CSV');

Any help on why this won't return the files?

Thanks

Answer 1

[Luke Perry]

The easiest way I've found on doing this would be something like the following:

%Get information about what's inside your folder.
myfiles = dir('C:\The_file_location_folder');

%Get the filenames and folders of all files and folders inside the folder
%of your choice.
filenames={myfiles(:).name}';
filefolders={myfiles(:).folder}';

%Get only those files that have a csv extension and their corresponding
%folders.
csvfiles=filenames(endsWith(filenames,'.csv'));
csvfolders=filefolders(endsWith(filenames,'.csv'));

%Make a cell array of strings containing the full file locations of the
%files.
files=fullfile(csvfolders,csvfiles);

Now after this you can use whatever you'd like to loop through those files and open them, whether it be xlsread() or fopen() with textscan().

Personally, I prefer using textscan() to open .csv files because it takes quite a bit less time, and it sounds as though you may have a large number of files. However, textscan does have its drawbacks, and if the files were created in Excel, it may not always format whatever's inside the files properly, not allowing you to actually get the information you desire out of it.

See these examples for textscan() and xlsread().

textscan():

Specific Data Formats: Data Format

Some problems you may encounter: Problem 1 Problem 2

xlsread()

Online Documentation: Mathworks Documentation

You may also find other resources on how to read in csv files quickly, such as csvread(). However, I've always preferred to design my own routines to read in the information I am looking for.

[Jan]

contains(filenames,'.csv')

This catches "myfile.csv.pdf" also. Use endsWith instead.

Use fullfile instead of strcat, because this considers e.g. 'C:\' without inserting a double file separator.

[Luke Perry]

Thank you, Jan. Good catch. I've adjusted my answer with both of your suggestions.

Answer 2

[Sean de Wolski]

If they all have that naming scheme how about using a for loop:

for ii = 1:10
  fn = strcat('E:\group',num2str(ii,'%02i'),'.csv')
end

[Jan]

As far as I understand, the folder is called "groupB" and only the file names have numbers.

[Seth]

Yes I have tried using a loop, however you're example and numerous others I have tried haven't worked. Using this all it returns is a value of 'E:\group10.csv' for fn in the variable editor/workspace

[Sean de Wolski]

Well that's because nothing else happens inside the loop.

You would have to include the csvread or whatever command at each iteration in order to read it in while fn is equal to that name...

FAQ

[Seth]

Any time I use csvread to import the whole folder it says the variable doesn't exist. Using csvread allows me to import single files inside the folder 'groupB01.csv' for instance but the folder 'groupB' doesn't return anything

Answer 3

[Jan]

Are you working under the case-sensitive Linux system?

When dir('E:\groupB\*.csv') does not find any files, there are no such files. So please explain, why you expect the files to exist.

[Seth]

the files are downloaded and saved in a folder named groupB which has all the csv files in it but they do not appear when I try to read them in as a group. I can import indivudual csv files from the group by writing:

csvread('groupB01.CSV',6,1);

But I have not been able to do import the whole folder, any hints?

[Ken Atwell]

I believe CSVREAD can only read one file. You would need to create a loop to read all of the files. Something like (this written from memory):

 cd <folder/where/files/are>
 files = dir('*.csv');
 outs = cell(numel(files),1);
 for i = 1:numel(files)  % Could be parfor
    out{i} = csvread(files(i).name, 6.1)
 end
 % Merge the outputs into one bigger matrix
 allouts = vertcat(out{:});

Answer 4

[David Sanchez]

Are your CSVs in the working directory? my_csvs = dir('*.CSV') should return a struct array with a field called "name", hosting the name of the *.csv file in each case. If your *.csv files are somewhere else, try to retrieve the path with

my_csv_dir = uigetdir;

what will prompt a search directory window and return the directory of your choice. Then:

my_full_path = horzcat(my_csv_dir,'\group*.CSV')
my_csv_files_struct = dir(my_full_path);

[Seth]

Yes the folder with the CSV files is in the working directory yet it does not import them all in