Opening files with randomly varying file names

Hello,

I am using Matlab to read in csv files generated during data collection. The files all have the format:

PR-2230A_YYYY-MM-DD_00-08-*.csv

where the * is a randomly changing number from 0 to 9. There is no pattern to the number and I have A LOT of data files to open so I need to loop through them. Typically I would declare a filename and use strcat() with variables for any part of the filename that changed, but since I have no idea what the numbers will be I can't do this. The last number is non essential there is only one file per day.

Is there anyway to read in the files without knowing the last number?

i.e. filename = strcat('PR-2230A_',YYYY,'-',MM,'-',DD,'_00-08-',~,'.csv')

I would be greatful for any help!

Answer 1

[Oleg Komarov]

I propose a different approach:

EDIT: forgot about the wildcard

% Retrieve all the files in a directory
names = dir('C:\Users\Oleg\Desktop\Nuova cartella\PR-2230A_YYYY-MM-DD_00-08-*.csv');
names = {names.name};

Now names will contain only the files which begin with root and you can loop through all of them and load one by one.

Oleg

[Jacki]

Worked GREAT thanks!!!!!!!!

[Zhendong Huang]

Thanks so much!!!

[Candice Cooper]

I can't figure out why when I try to do this I get the error "Dot indexing is not supported for variables of this type."

[Walter Roberson]

@Candice Cooper

Typically now when you use dir() and the location does not exist, then what you get back is an empty struct. But sometimes now (not sure what circumstances), and historically, instead of getting back an empty struct, you could instead get back [] -- but you cannot do dot indexing of []

Any time you use dir() you should check isempty() before relying on the results.

[Candice Cooper]

I now recognize why the dot indexing isn't working, but I'm not getting an empty struct or []. It's returning a double for some reason (that I don't understand). I have a folder that contains various files that each differ only in a number. I first input:

n = dir('NDBC_winds_*.txt');
    % where the wildcard spans 1998:2016

It's returning a vector of 16 values, none of which I can make sense of.

Thank you for your help! If you have any suggestions please do let me know. Thanks!

[Walter Roberson]

@Candice Cooper

You accidentally created a variable named dir so dir('NDBC_winds_*.txt') is being treated as an indexing request, equivalent to

dir = randi(9, 1, 200);    %for example

indices = double('NDBC_winds_*.txt')

indices = 1×16

78 68 66 67 95 119 105 110 100 115 95 42 46 116 120 116

n = dir(indices)

n = 1×16

1 8 5 2 2 5 6 4 8 6 2 6 9 4 1 4

[Candice Cooper]

oh my gosh this makes total sense! thanks for helping me debug that!!

Answer 2

[Jim Hokanson]

Just in anyone else comes across this, use a wildcard instead:

d = dir('PR-2230A_YYYY-MM-DD_00-08-*.csv')
names = {d.name};

The trick is that the dir() function supports wildcards.

[Oleg Komarov]

Thanks, +1

Answer 3

[BHARGOB DEKA]

So, How do I loop it over several file names?

[Emma Birkett]

Here is an example which might help: I have a list of numbers that I need to find in a list of file names. The filenames here are in the format tapsDataP_RandN where P = participant and RandN is a random number. So my list of P numbers is:

    actualPNos = [1:4,7:12,14,16:23,25:38,40,43:48,50:52,54:57,59:60];

Get the length of this:

    nParts = length(actualPNos);

Set up a matrix where I want to put the data from each of these files (they're quite big data sets) - there will be a new 'sheet' for each dataset:

    DataMatrix = nan(175000,15,nParts);

I loop round this loop to get each filename, look up the data in that file and put it in my matrix. Here I have to state the file name, using the number string relating to the current participant number and a wildcard(*) (for the RandN). Then the filename contains a wilcard (csvread doesn't accept this), so use dir to find the entry with that filename and index into that dir struct using .name. This gives a string which I turn into a character vector using char. This can then be used for csvread.

for n = 1:nParts

         pNo = actualPNos(1,n);

         filename = (['tapsData',num2str(pNo),'_*.mat']);

       d = dir(filename);

       file = {d.name};

       name = char(file);

       M = csvread(name);

       DataMatrix(:,:,n) = M;
end

Probably not the most elegant, but it works!