remove duplicate rows from text file
조회 수: 6 (최근 30일)
이전 댓글 표시
I'd be glad to have some advise on the fastest way to read a text file which contains duplicate rows, remove all the duplicates and write it back.
Matlab version R14 SP3
댓글 수: 2
답변 (1개)
Guillaume
2018년 7월 19일
lines = strsplit(fileread(yourtextfile), '\n')); %read file and split into lines
lines = unique(lines, 'stable'); %remove duplicate lines
fid = fopen(newfilename, 'w'); %open file for writing
fwrite(fid, strjoin(lines, '\n'), 'char'); %merge lines and write
fclose(fid);
댓글 수: 4
Guillaume
2018년 7월 19일
Matlab version R14 SP3
Wow! You're only 25 versions behind the current version, soon to be 26.
While it's probably fairly easy to write a version of strsplit using regexp that would work with your version and strjoin is probably not too complicated either, you'd also have to write your own unique function since the 'stable' option didn't even exist back then. That's a more complicated task particularly as I'd have to pore over the online documentation to find out what was possible or not back then.
I'm sorry you're on your own there. At some point you've got to let go of legacy code.
Walter Roberson
2018년 7월 20일
lines = strsplit(fileread(yourtextfile), '\n')); %read file and split into lines
[ulines, ia] = unique(lines);
lines = ulines(sort(ia));
fid = fopen(newfilename, 'w'); %open file for writing
fprintf(fid, '%s\n', lines{:});
fclose(fid)
Together with
function S = fileread(filename)
fid = fopen(filename, 'rt');
if fid < 0
error('file does not exist');
end
S = fread(fid, [1 inf], '*char');
fclose(fid);
function C = strsplit(S, delim)
C = regexp(S, delim, 'split');
참고 항목
카테고리
Help Center 및 File Exchange에서 Data Import and Export에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!