CSV file reading and applying loop to plot graphs

조회 수: 30 (최근 30일)
Arshey Dhangekar
Arshey Dhangekar 2021년 6월 2일
편집: Arshey Dhangekar 2021년 6월 15일
Hello
How can I do it for multiple csv file name of "WT_201119.csv" ,"WT_201120.csv" contain 52 main header with column. All headers are constant in csv files. Above csv read data correctly using readtable command but only for few files. where I have to do changes? similar to above one csv file Because I got error- Where I have to do changes always so that it can detect the total number of header and rows which we wan b "Error using matlab.io.internal.shared.TextInputs/set.DataLines (line 101) 'DataLines' must be specified as a positive integer or a 2-column matrix of positive integers."y
clc
clear all
d=dir('WT_201119.csv'); % "*" will detect the sequence of your file
opts = detectImportOptions(d(1).name,"Delimiter",","); % already know it's csv, avoid detecting it
T = {};
for i=1:numel(d)
fid = fopen(d(i).name,'r');
nhdr = 8 + cell2mat(textscan(fid,'Total Channels:%f',1,'HeaderLines',5,'Delimiter',','));
fid = fclose(fid);
if nhdr == 8 % skip no data in file cases if that occurs (dpb conjecture)
warning(['No data. Skipped file: ' d(i).name])
continue
end
opts.DataLines = [nhdr+2 inf];
opts.VariableNamesLine = nhdr+1;
T{i} = readtable(d(i).name,opts)
end
See more (1)
  댓글 수: 5
Mathieu NOE
Mathieu NOE 2021년 6월 2일
hello
sorry I don't understand the meaning of your second sentence... what do you want to retrieve or not ? and finally what was the issue in first place because now I am bit lost...
Arshey Dhangekar
Arshey Dhangekar 2021년 6월 2일
"Error using matlab.io.internal.shared.TextInputs/set.DataLines (line 101) 'DataLines' must be specified as a positive integer or a 2-column matrix of positive integers."
getting error below the line
opts.DataLines = [nhdr+2 inf];

댓글을 달려면 로그인하십시오.

채택된 답변

Jeremy Hughes
Jeremy Hughes 2021년 6월 3일
Without an example file, it's impossible to say why the code doesn't work as expected. I'm just working backwards from the error message and what I know about the returns of textscan.
I see the example uploaded in a comment for a different answer, but that file works as expected with the code you included.
  댓글 수: 5
LO
LO 2021년 6월 3일
편집: LO 2021년 6월 3일
I agree, that's what I meant.
I think before writing any code about this, Arshey should have clear in mind where are the data files coming from, how are they formatted and standardize the format as much as possible.
If you scroll though the CSV files there are multiple columns containing several errors and the "tabulation" of the data is also incoherent (besides the different file format).
Arshey Dhangekar
Arshey Dhangekar 2021년 6월 15일
편집: Arshey Dhangekar 2021년 6월 15일
Hello Jeremy,
I have two csv file WT and both have fixed have 52 header. I tried to use below command to read however header is as Var1,Var2..to Var52 instead of Store no,Date and Time. How can I fix the header problem with importing all the datas from those files?
table = readtable('WT_201120.csv');
table1 = readtable('WT_.csv');

댓글을 달려면 로그인하십시오.

추가 답변 (2개)

LO
LO 2021년 6월 2일
편집: LO 2021년 6월 3일
first import one single CSV file and select coumns and rows of what you want usually to import.
Generate the code by clicking the 'generate code' button, in the import menu (right side of the screen, up)
Then use this code and put it where IMPORT CODE is written here below
[logfname, pathname] = uigetfile('*.csv','Pick a log file'); %select a csv file
cd(pathname); % move to that folder
file_list = dir ('*.csv') ; % create a list of csv files based on folder content
conversion = struct2cell(file_list);
conversion1 = transpose(conversion);
name_list = (conversion1(:,1));
% these two limits give you the option to select only a certain file range
% but you could actually remove them and simply set the k in the for loop
% from 1 to Nr of files
first= 1; % set first file to analyze
last= 10; % set last file to analyze
for k= first:last
filename = [pathname name_list{k}];
delimiter = ',';
startRow = 5;
endRow = 11831; % you can remove this or set it to a level that would fit all your data logs
formatSpec = '%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%[^\n\r]';
fileID = fopen(filename,'r','n','UTF-8');
fseek(fileID, 2, 'bof');
dataArray = textscan(fileID, formatSpec, endRow-startRow+1, 'Delimiter', delimiter, 'TextType', 'string', 'EmptyValue', NaN, 'HeaderLines', startRow-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
fclose(fileID);
data_table = table(dataArray{1:end-1}, 'VariableNames', {'Name','x1','x2','x3','x4','x5','x6','x7','x8','x9','x10','x11','x12','x13','x14','x15','x16'});
data_table = data_table(22:end,:); % this crops away the header part of your file (somehow the file format is not properly detected by matlab and there is some problem with formatting
clearvars filename delimiter startRow endRow formatSpec fileID dataArray ans;
size_t = size(data_table);
plot_size = sqrt(size_t(2));
if plot_size > floor(plot_size)
plot_size=plot_size+1;
end
for i = 1:size_t(2)
subplot(round(plot_size),round(plot_size)-1,i)
% pre allocate var space
time_vector = zeros(numel(data_table(:,i)),1);
var_vector = zeros(numel(data_table(:,i)),1);
for j = 1:numel(data_table(:,i))
time = table2array(data_table(j,1));
time(time == ' ') = [];
time_vector(j) = str2double(time);
var = table2array(data_table(j,i));
var=strrep(var, ' ', '');
var_vector(j) = str2double(var);
end
plot(time_vector,var_vector)
title('title here')
xlabel('label')
ylabel('label')
end
end
  댓글 수: 6
Arshey Dhangekar
Arshey Dhangekar 2021년 6월 2일
Hello,
I uploaded file. I ´want to time vs every M variable in csv using loop
LO
LO 2021년 6월 2일
편집: LO 2021년 6월 3일
I have updated the script, it works. However there is an issue with the text formatting of your data: due to the presence of whitespaces between characters the variables are converted into NaNs so you cannot see the data. I have tried to eliminate them using the function (var is one of the values in your data array)
strip(var,char(133)) or the other char codes you can see below but it does not remove them.
char(133) Next line
char(160) Nonbreaking space
char(8199) Figure space
char(8239) Narrow no-break space
the strip function could replace the line "var=strrep(var, ' ', '');", which is meant to have analogous results (although strrep does not affect special whitespace characters I suppose).
The problem may be in the formatting of the text and how MATLAB reads it. Do you have by chance the older file you posted ?
Apart from this, it should work.

댓글을 달려면 로그인하십시오.


Jeremy Hughes
Jeremy Hughes 2021년 6월 3일
You should look at what nhdr is when you see the error. It's likely an empty value which makes nhdr+2 empty, and [nhdr+2 inf] just [inf].

카테고리

Help CenterFile Exchange에서 Spreadsheets에 대해 자세히 알아보기

제품


릴리스

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by