MATLAB Answers

large file slow running

조회 수: 12(최근 30일)
Joydeb Saha
Joydeb Saha 2021년 9월 20일 8:25
답변: Jan 2021년 9월 20일 15:36
latmin=-90; %chooce your location
latmax=90;
longmin=-180; a=1;
longmax=180;
data=zeros(90,90);
for b=1:length(x(:,1))
if (x(b,8)>=longmin && x(b,8)<longmax && x(b,7)>=latmin && x(b,7)<latmax);
for col=1:13
data(a,col)=x(b,col);
end
a=a+1;
end
end
theres a file called x.mat (601760x13). What might be the reason it runs too slow
  댓글 수: 1
KSSV
KSSV 2021년 9월 20일 8:47
What you are trying to achieve? It can be achived without loop. You need to give an explanation of your problem. What does x.mat have?

댓글을 달려면 로그인하십시오.

답변(1개)

Jan
Jan 2021년 9월 20일 15:36
I guess that the iterative growing of the array data slows down the processing.
Remember that the growing of an array required to create a new array and to copy the foermer contents.
x = [];
for k = 1:1e6
x(k) = rand;
end
This creates a new array in each iteration. Although the final array needs 8MB only (8 bytes per double), Matlab has to allocate and to copy sum(1:1e6)*8 bytes, which are more than 4 TB ! The solution is easy: Allocate with the final size:
x = zeros(1, 1e6); % Pre-allocation
for k = 1:1e6
x(k) = rand;
end
In your case you do not know the needed size in advance. You tried to pre-allocate with data=zeros(90,90), but this is not the final size. You could determin the final size:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = zeros(sum(index), 13);
a = 1;
for b = 1:size(x, 1) % nicer and faster than: length(x(:,1))
if index(b)
for col = 1:13
data(a, col) = x(b, col);
end
a = a + 1;
end
end
But it is easier, nicer and faster to perform the copy directly without a loop:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = x(index, 1:13);

태그

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by