using datastore does not generate exact data files (extra rows or rows missing)
조회 수: 1 (최근 30일)
이전 댓글 표시
Hi there,
I have been trying to use datastore and apply it to the file attached (the file is an extracted section of one of our data files). The script I use to set up the datastore setting is as follows:
ds = datastore(data_file_loc_ds);
ds.ReadSize = 1000000;
ds.NumHeaderLines = 0;
ds.Delimiter = {'\t',' '};
% ds.Whitespace = {' \b\t'};
ds.MultipleDelimitersAsOne = 1;
ds.CommentStyle = {'%'}
ds.TreatAsMissing = {'-1.#IO'};
ds.TextscanFormats = {'%{dd/MM/yyyy}D','%{HH:mm:ss}D',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f',' %f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f'};
ds.ReadVariableNames = 1;
ds.VariableNames(1:2) = {'date','time'};
preview(ds)
The created datastore seems to do different things for slightly different files. It generally tends to skip the first row (or a few) of the files. At times it repeats a section of the data file and hence for instance I end up with several extra files and sometimes it skips rows. I have been trying different combinations of the "TabularTextDatastore" properties to get the rows exactly as they are but it seems to be impossible.
댓글 수: 0
답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Large Files and Big Data에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!