read file variables methods
이전 댓글 표시
I got a large data matrix (3Gb) of signal data which I saved to a .mat file. Since the data size is large I figured out it will be wiser to access the file and read each time a block of data, process it and move to the next block instead of loading the complete file. I'm wondering should I use the matfile method, fopen and fread or should I use a datastore and read command. What is faster from the processing point of view and easier to work with? Or any other considerations? Thanks
댓글 수: 3
per isakson
2016년 11월 11일
"data set"   is that one numerical array?
"fairly large data set"   how large compared to the physical memory?
Talmon Alexandri
2016년 11월 12일
per isakson
2016년 11월 14일
Given
- "large data matrix (3Gb) of signal data which I saved to a .mat file"
- "Total RAM size - 3G[B]"
- "[one] plain numerical array 10 million lines by 48 columns"
I assume the mat-file is version 7.3, because that's needed to store an array of that size.
I would say this a a perfect case for matfile, Access and change variables directly in MAT-files, without loading into memory.
It's the easiest to work with and I don't think there is any other way that is significantly faster.
답변 (1개)
Walter Roberson
2016년 11월 11일
0 개 추천
카테고리
도움말 센터 및 File Exchange에서 Large Files and Big Data에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!