how to skip incomplete and missing lines when reading from Text file?

Hi everyone. I have some trouble. I try reading from a text file and sample data is as shown

16.Mrz.14 10:12:34

16.Mrz.14 10:12:37 -01,22522E-09

16.Mrz.14 10:12:40 -00,90903E-09

16.Mrz.14 10:12:42 -00,72633E-09

16.Mrz.14 10:12:45 -00,59084E-09

16.Mrz.14 10:12:48 -00,50685E-09

16.Mrz.14 10:12:50 -00,42215E-09

16.Mrz.14 10:13:00 -00,25394E-09

16.Mrz.14 10:13:03 +9,900000E+37OADC,+1629586.578736secs,[phone]RDNG#

16.Mrz.14 10:13:22

16.Mrz.14 10:13:31 -000,1151E-09

This is the code for reading from text file

%% Open the text file.

fileID = fopen(fullPath,'r');

%% Read columns of data according to format string.

dataArray = textscan(fileID, formatSpec, 'Delimiter','', 'WhiteSpace', '', 'ReturnOnError', false);

%% Close the text file.

fclose(fileID);

I used text scan and it gives me an error. Sometimes there are also empty lines btw these lines. I want to skip these incomplete lines and missing lines.how to do it? I would be glad if someone helps me.

Answer 1

[dpb]

dataArray = textscan(fileID, formatSpec, 'Delimiter','', 'WhiteSpace', '', ...

A) You don't show formatSpec

B) The delimiter isn't white space, it's both blank_ and comma-delimited.

There's no way to read a constant-width field data file with missing values in C other than parsing the lines by character position. See recent thread for a complete discussion of the issues involved and solutions for a given problem that will illustrate for others.

http://www.mathworks.com/matlabcentral/answers/120549-extract-value-from-txt-weird-lay-out

If at all possible, change the generation mechanism to write delimited files with empty fields demarcated. If that's not possible, then you'll have to process the file line-by-line and parse each line individually. fgetl is your friend here as well as perhaps regexp

ADDENDUM:

B) The delimiter isn't white space, it's both blank and comma-delimited.

Actually, it appears even worse than that on closer inspection --

16.Mrz.14 10:13:03 +9,900000E+37OADC,+1629586.578736secs,[phone]RDNG#

What is the above supposed to translate to? It appears that it uses comma as both decimal point and field separator? Matlab doesn't know how to handle the comma as decimal indicator and I wouldn't think there'd be any standard language that can handle both usages simultaneously.

And what's the value of

+9,900000E+37OADC

? Is it REALLY 9.9E370 (or even 9.9E+37 if assume the '0ADC' is some label rather than 3-digit exponent)? Either way, that's an awfully big number and the E370 if so will overflow Matlab default double and return Inf. The largest double is on the order of E308.

[Snehalatha]

Hi dpb, Thank you for your reply. Sorry I did not specify the formatspec. Matlab doesn't recognise comma as decimal separator but i wrote a script that would convert comma to decimal but I had trouble reading incomplete lines and lines like what you mentioned above. It is actually not a value but an error and it is such lines I wanted to skip. I processed it line by line as you mentioned and it worked.

thanks a lot!

Answer 2

[Jos (10584)]

Empty lines are skipped. The real trouble is reading the incomplete lines. I suggest to read each line in as a string, and then parse those strings in a loop.