How to extract rows from a text file with a specific start?

$GNRMC,083059.00,A,2948.16655,N,12133.72003,E,0.018,,280217,,,D*69
$GNVTG,,T,,M,0.018,N,0.034,K,D*36
$GNGGA,083059.00,2948.16655,N,12133.72003,E,4,12,0.90,5.0,M,10.9,M,1.0,0790*64
$GNGSA,A,3,23,27,16,14,26,31,,,,,,,2.00,0.90,1.78*19
$GNGSA,A,3,80,73,82,83,70,81,71,,,,,,2.00,0.90,1.78*1C

For example I have this kind of text file (it continues on for very long) and I want to extract only the rows starting with $GNGGA and write them all to another text file

Answer 1

[Jan]

% Parameter:
FileName = 'C:\Your data file.txt';
Key      = '$GNGGA';
NewFile  = 'C:\Fixed.txt';

% Import text file and select lines starting with the Key string:
Str   = fileread(FileName);
CStr  = strsplit(Str, '\n');
Match = strncmp(CStr, Key, length(Key));
CStr  = CStr(Match);

% Create new file and write matching lines:
fid = fopen(NewFile, 'w');
if fid == -1
  error('Cannot create new file: %s', NewFile);
end
fprintf(fid, '%s\n', CStr{:});
fclose(fid);

[Michael Lim]

it worked! thank you very much

[Ali Jam]

How about more than one key? for example, all lines starting with 'X' and 'Y' and 'Z' or more?

[dpb]

Keep or remove? Following removes; reverse sense of contains logic test to keep--

keys=cellstr(['X':'Z'].');
fid1=fopen('yourfile');
fid2=fopen('newfile','w');
while ~feof(fid)
  l=fgets(fid);
  if contains(l(1),keys); continue, end  % not found, go on
  fwrite(fid2,l)                         % found, write 
end
fid1=fclose(fid1);
fid2=fclose(fid2);

[Andreas Voldstad]

Hi!

I have a very similar problem, trying to find lines in a txt file starting with 'TuneOnset', which also contains onset times at the end of the line in milliseconds.

I tried this solution as a start, but somehow I can't get it to work.

The code works and I get no errors, the file is split up into cells based on lines in the text, and the cells look like I expect, some of them starting with 'TuneOnset', but strncmp does not match it and outputs only logical 0s.

Does any one have any idea why this would not work for me, or an alternative solution for extracting a number at the end of lines starting with some specific string?

Answer 2

[dpb]

"Dead-ahead" solution...

fid1=fopen('yourfile');
fid2=fopen('newfile','w');
while ~feof(fid)
  l=fgets(fid);
  if isempty(strfind(l,'$GNGGA')), continue, end  % not found, go on
  fwrite(fid2,l)                         % found, write 
end
fid1=fclose(fid1);
fid2=fclose(fid2);

Alternatively, read full file into memory and do the search there if not too big...

[Jan]

@dpb: isempty(l,'$GNGGA')? Better: ~strncmp(l, '$GNGGA', 6)

[dpb]

good catch; edit typo for air-code fixup...initially forgot to save the record for the subsequent write and elided more than needed/shoulda'...

[Jan]

@dpb: It is a good idea to avoid buffering the complete file. Your code can process a 500GB file on the fly. +1

[dpb]

Indeed, simply a filter works well altho if subsequently going to process anyway, simply deleting unwanted lines from an array in memory may be reasonable as well. Of course, is GB file and only a few K are left after processing, maybe "not so much". ;)

The hangup that it seems like we talked over once before is that the record-by-record reading of the file at one time could almost be guaranteed to be quite slow; it appears from what I recall of some timings you did earlier that TMW and/or the OS now has buffering done well enough there was little penalty at all...

[Satya Gopal]

Hi, I've been trying to do the same thing. Your code works well. I just had a problem with it. If I have items like alpha, alpha/all, beta, beta/all in the first row and i put the strfind input for 'alpha', it copies alpha and alpha/all rows as well. Can anyone help with extract string match?

[dpb]

strcmp and strncmp are exact matches...you have to define a unique string as the pattern that allows you to segregate as desired. With multiple patterns, depending upon what you're actually trying to accomplish you may need to do a second compare for the 'all' pattern having found the record containing the other key word.

Or, switch to using regular expressions for more general pattern matching.