script running for 13 hours. should I be worried?
조회 수: 2 (최근 30일)
이전 댓글 표시
I have a function going through 9 excel files with about 1.7mil cells. It imports them, puts everything into one column then goes through and removes all the 0's and saves the output of about 300k cells. Its been running for about 13 hours of so now and I'm concerned that I may have messed up the code so I was hoping someone would help me out a fresh set of eyes.
it does look like its working, maybe? the output files look with theyve been edited, but I can't really be sure. this is the code i'm using at the moment.
function [ m ] = combine( input_args )
source = 'C:\Users\aoneil\Desktop\ODEN\trimmed'; %input the directory to choose the source folder
dest_dir = 'C:\Users\aoneil\Desktop\ODEN\graphs'; %choose where to output the resulting file
files = dir(fullfile(source, '*.CSV'));
for j = 1:length(files) %loops through all the file in the directory folder
a = xlsread(fullfile(source, files(j).name));
for x = 1 : size(a,1) %steps through the matrix
d((1+((x-1)*size(a,2))):(size(a,2)*x), 1) = a(x,1:end); %saves all the values of original to matrix "d"
end
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
csvwrite(fullfile(dest_dir, files(j).name), m); %outputs matrix "d" to the file name/directory chosen
end
end
댓글 수: 0
채택된 답변
Jan
2015년 2월 5일
You urgently require a "pre-allocation". Letting arrays grow iteratively is extremely expensive. Example:
x = [];
for k = 1:1e5
x[k] = rand;
end
Now in each iteration a completely new vector x is created and the older values are copied. This means, that the OS has to reserve sum(1:1e5) * 8 bytes in total and copy almost the same amount of data in addition.
A vectorization solves the pre-allocation implicitly:
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
Better without loop and IF:
m = d(d > 0);
댓글 수: 0
추가 답변 (1개)
Alessandro Masullo
2015년 2월 5일
You can use a task manager to check if matlab is still using the CPU. If not, it probably got stuck.
댓글 수: 0
참고 항목
카테고리
Help Center 및 File Exchange에서 Data Import from MATLAB에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!