Out of memory issue
조회 수: 5 (최근 30일)
이전 댓글 표시
I have a sparse matrix which has a size of, (2031616x4096 sparse double).
I want to make 10 subset of this matrix and need to convert this matrix in to "full". So, I have made 10 subset and while processing it was able to covert 5 subset into " full" and for the rest of the five subset it is showing me the message : "out of memory".
One thing I have done that is, when I saved the each subset separately into a external harddisk may be at that time it will be possible but here again the problem appears during the importing the subset matrix. I know that it is not a solution .
I will appreciate if anyone can suggest some helpful solution to tackle this problem.
댓글 수: 3
답변 (1개)
Steven Lord
2021년 9월 20일
sz = [196608 4096];
numberOfElements = prod(sz);
bytes = 8*numberOfElements;
gb = bytes/1024^3
Each subset is 6 GB in size (assuming real doubles, 12 if complex doubles) when converted into a full array. Depending on how much memory you have available it's quite possible you couldn't fit all ten subsets in memory at once.
Can you avoid the conversion to a full matrix? MATLAB can save sparse matrices to MAT-files and load them from those files perfectly well. Alternately if you need to store them in a format some other program can read perhaps write a coordinate list (which you can generate using the find function) to the disk?
Alternately clear-ing each subset once it has been written to disk may allow MATLAB to reuse that memory for the next subset.
참고 항목
카테고리
Help Center 및 File Exchange에서 Data Type Conversion에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!