How to run a program on an csv file data in batches
조회 수: 1 (최근 30일)
이전 댓글 표시
Hello
I am trying to import data from csv file with size (1046520 1). Using the importdata function.
K = load('caamel.csv');
It is taking over an hour to compute my program.
How do I run my code on the excel file in batches instead of one whole data which is taking forever to see result
I will appreciate your help
thanks
Tino
댓글 수: 12
Walter Roberson
2019년 6월 20일
You implied that the reason that your code is taking so long is that import (but really load) is slow. I showed timing on a file 50 times larger than yours could take about 20 minutes to load() so your file would take about half a minute to load(). If you were to switch to textscan then it should take roughly 250/50 = 5 seconds. At that point you might as well read in all the data and loop over portions of it.
textscan also permits you to to specify a count to read, if you find that easier than loading everything and looping indexing.
답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Large Files and Big Data에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!