How to import and read several large csv in matlab
조회 수: 10 (최근 30일)
이전 댓글 표시
Dear all,
I have a huge amont of large csv files (about 170Mb each).
I need to open them and use the information in two of the columns to plot, let's say collumn D and E (see the attachment).
This is what I did so far, however I found that it is not very efficient.
%Load directory
directorio=uigetdir;
x=dir([directorio,'/','*.csv']);
K=length(x);
Atotal=[];
tabletotal=[];
fprintf('Leitura: \n')
tic;
for k=1:K
FILENAME = strcat(x(k).folder,'/', x(k).name);
TotalData=readtable(FILENAME);
table=TotalData(:,[4 7 18 19 27]); %4-vessel name; %7-vessel type; 18-longitude; 19-latitude
fprintf('File_%s: %s \n', num2str(k), FILENAME);
tabletotal=[table; tabletotal];
%Coluna 2= longitude
%Coluna 3= latitude
end
loadAIS=toc;
fprintf('Tempo de carregamento do AIS: %s \n', num2str(loadAIS) );
clear table;
clear TotalData;
A = table2array(tabletotal(:,2:5));
Thank you in advance
채택된 답변
Sulaymon Eshkabilov
2022년 6월 21일
If your data file names are sequential, then you may consider using this type of approach here:
FILE = fullfile('C:\Users\...', '*.csv'); % Directory where the files are residing.
LIST = dir(FILE);
N = length(LIST); % N = number of files
D = zeros(???, ???, N); % Memory allocation to speed up the process of data storing
for ii = 1 : N
FFName = fullfile(LIST(ii).folder, LIST(ii).name);
DATA = readmatrix(FFName);
D(:, :, ii) = DATA; % NOW D contains all data from 10000 files
end
... % Select and process the part of D that is necessary
An alternative way might be this one:
P=pwd; % If your files are located in the current directory
P = fullfile('C:\...\...', '*.csv'); % Specify: directory where the files are residing.
S = dir(fullfile(P, '*.csv')); % Select the file extension to suit your data files
M = 0;
for k = 1:numel(S)
F = fullfile(P, S(k).name);
D = D + readmatrix(F);
end
...
댓글 수: 2
Sulaymon Eshkabilov
2022년 6월 22일
readmatrix() and readtable() are recommended ones due to their efficiency instead of xlsread() or csvread(), etc.
추가 답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Data Import and Analysis에 대해 자세히 알아보기
제품
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!