How to skip lines when reading a text file that start with specific input?

Hi -

I'm working on reading in a text file of accumulated data, but the headerlines are always printed with the data. For example, the beginning of each line in the text file looks like:

"TOA5","CapeBird","CR1000",
"TIMESTAMP","RECORD",
"TS","RN","","","","",
"","","Smp","Smp","Smp"
"2014-10-01 00:10:00", 
"2014-10-01 00:20:00",
"2014-10-01 00:30:00", 
"TOA5","CapeBird","CR1000",
"TIMESTAMP","RECORD",
"TS","RN","","","","",
"","","Smp","Smp","Smp"
"2014-10-01 00:40:00", 
"2014-10-01 00:50:00",
"2014-10-01 01:00:00",

.....

I've found examples where you can specify to not read in strings through the "textscan" function, but as you can see, everything is a string. The data I want read in starts with the dates "2014-10-01..." and I want to eliminate the 4 lines that have words or just empty double quotes. The header will always be exactly the same... it's a very long line of text, so I'm not sure if that complicates this, or if I can just find the first phrase/word and eliminate the entire line. Also, the data is not always consistent and will not always come in lines of 3 (sometimes more, sometimes less), so I can't just skip 4 lines, read in 3, skip 4 lines, etc.

If anyone knows how I can delimit or just eliminate these 4 header lines by somehow checking for a word/phrase/entire line, that would be incredibly helpful. Long story short, when the line is NOT one of the header lines, I need this to work:

textscan(file,'%s %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f', ...
        'delimiter',',','EmptyValue',nan, 'headerLines',4,'TreatAsEmpty','"NAN"')

Thanks!

Answer 1

[Andrew Reibold]

Hi Marian,

Maybe this can get you started. I made a textfile of the example you provided, and a sample code is attached to read it.

What this is doing is reading each line, checking if it has the right date, and then if it does - perform 'some action' . Right now, as an example I have it just saving the desired strings to a new cell called Clean_Data. If you wanted, I'm sure you could replace my example action with your textscan command here. :)

Let me know if you have a question. I did not incorporate your specific textscan line because the example data is not a full sample with the right format.

Edit: Will not be available this weekend. Best wishes though

[Marian]

Thanks so much! I had to generalize the code slightly (So that it can work for all months, not just Oct, and all dates, not just the 1st), but this definitely solved my problem!

[Andrew Reibold]

Oops. Yeah, hard coding the date was pretty dumb! Good job!