Reading multiple text files

I am very thankful to mathwork community to promptly answers the queries asked by me. I am very new to matlab, wrote my first code to compute runoff using antecedent Soil Moisture Method and it is working well.

The only problem that I am facing now, is to extract data from text files. I asked the same question previously but was not able to solve the problem. Now rephrasing the question. My input data is in the form of text files, each file having daily rainfall data of specific year. I simply need to translate information into single file having 2 columns (1st column to represent date, mm/dd/yyyy and 2nd column to represent rainfall. I have 50 text files of a gauging station and requires to create a single output file from all these text files. The input file is attached for your reference.

Thanks for your cooperation

Kind regards,

[per isakson]

See

• http://www.mathworks.com/matlabcentral/answers/158467-help-with-operation-of-multiple-files#answer_155014
• http://www.mathworks.com/matlabcentral/answers/154540-run-code-on-multiple-files-within-a-directory#answer_151509

Answer 1

[Walter Roberson]

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

You say that in the result, column 1 should represent the data and column 2 should represent the rainfall, but you have not indicated where the rainfall is coming from.

Is it possible that you want column 1 to represent the date and column 2 to represent the the data that you extracted from those 50 files? If so then what information would you like stored for the date? Is the year information part of the file name?

[Saleem Sarwar]

Column 1 to represent dates including year, month and day and column to represent the rainfall of that particular day. The format will be like this

File 1

1/1/1950 0
1/2/1950 2
1/3/1950 3
.....
''''
'''' 
12/31/1950 4

File 2

1/1/1951 0
1/2/1951 2
1/3/1951 3
.....
''''
'''' 
12/31/1950 4

File 50

/1/2000 0
1/2/2000 2
1/3/2000 3
.....
''''
'''' 
12/31/2000 4

Regards, Saleem

[Walter Roberson]

I am not sure where the 31 X 12 comes into this then?

[Saleem Sarwar]

31 rows and 12 columns for 1 year File attached for your review. I have 50 text files in this format. The output file, required will be 2 column file. 1st column to represent datae i.e mmyydd format and second column to represent rainfall.

[Walter Roberson]

Yes, that can be done. It requires fixed-width columns, though. Looking I see that the decimal points are 9 characters apart, so the missing data (because the day does not exist) can be determined by a sequence of 9 spaces. Because of that the easiest way to handle the missing entries would probably to use regexprep(LinesCell, '\s{9}', ' X') and then to regexp() 'split' the result on '\s+', vertcat() the resulting cells to get a rectangular array of cells, then str2double() to get numeric values.

The year is probably most easily pulled out of the second line by using regexp() of '\d{4}'

Anyhow, it is my bedtime, so I will leave this for you (or someone else) to implement.

[Saleem Sarwar]

Hi all, I am very new to the matlab. I am not too expert to write a code for reading multiple text files and yet I have to do this for my assignment. Can any body please help me in this regard? I simply need to translate information into single file having 2 columns (1st column to represent date, mm/dd/yyyy and 2nd column to represent rainfall. The output file format is shown below Day Rainfall(mm)from input 50 text files( file attached for reference)

Answer 2

[per isakson]

Try cssm() it outputs to the screen

01/01/2013      0.0
01/02/2013      0.0
...
02/27/2013     14.6
02/28/2013       TR
03/01/2013      0.0
03/02/2013      0.0
...
12/30/2013      0.0
12/31/2013      0.0

where

function  cssm()
    filespec = 'h:\m\cssm\JHMRRR13.TXT';
    fid = fopen( filespec );
    hd  = textscan( fid, '%s', 5, 'Delimiter', '\n' );
    cac = textscan( fid, ['%f',repmat('%9s',[1,12])], 'WhiteSpace','' );
    fclose( fid );
    yy  = regexp( hd{1}{2}, '\d{4}\s*$', 'match', 'once' );
    y   = str2double(yy);
    d   = cac{1};
    for mm = 1 : 12 
        data = cac{mm+1};
        for jj = 1 : length( data )
            if not( strcmp( data{jj}, '         ' ) )
                str = datestr( datenum([ y, mm, d(jj) ]), 'mm/dd/yyyy' );
                fprintf( '%s%s\n', str, data{jj} )
            end
        end
    end
end

&nbsp

Following Walter's advice and use '%9c' (see comment below)

function  cssm()
    filespec = 'h:\m\cssm\JHMRRR13.TXT';
    fid = fopen( filespec );
    hd  = textscan( fid, '%s', 5, 'Delimiter', '\n' );
    cac = textscan( fid, ['%f',repmat('%9c',[1,12])], 'WhiteSpace','' );
    fclose( fid );
    yy  = regexp( hd{1}{2}, '\d{4}\s*$', 'match', 'once' );
    y   = str2double(yy);
    d   = cac{1};
    for mm = 1 : 12 
        data = cac{mm+1};
        for jj = 1 : length( data )
            if not( strcmp( data(jj,:), '         ' ) )
                str = datestr( datenum([ y, mm, d(jj) ]), 'mm/dd/yyyy' );
                fprintf( '%s%s\n', str, data(jj,:) )
            end
        end
    end
end

&nbsp

So far one file. To read 50 files more information is needed. Remember my questions?

[Walter Roberson]

Careful, %9s does not start until the first non-blank. %9c starts in the current position.

[Saleem Sarwar]

It worked. Thanks Walter.

[Walter Roberson]

per deserves the credit for this code

[per isakson]

Walter, thanks for your comments

"Fixed width" &nbsp The Matlab documentation fails me, and I don't have the ten years of experience with C needed to get it :-(

A little experiment with R2013a

>> cac = textscan( '  12', '%2s%2s', 'WhiteSpace','' )
cac = 
    {1x1 cell}    {1x1 cell}
>> cac{:}
ans = 
    '  '
ans = 
    '12'
>> cac = textscan( '  12', '%2c%2c', 'WhiteSpace','' )
cac = 
    '  '    '12'
>> cac = textscan( '  12', '%2c%2c' )
cac = 
    '12'    [0x2 char]

Excerpt from R2015b documentation (same wording in R2013a)

When the field width operator is used with single characters (%c), textscan also reads delimiter, white-space, and end-of-line characters.

cac = textscan(' 12','%2c%2c') doesn't honor the documentation the way I interpret it.

[Walter Roberson]

An even simpler example:

textscan(' 1','%c')

This would be expected to return cell with a 2 x 1 char, but instead it returns a cell with a 1 x 1 char. That is not what I would expect from the definition of %c