Out of memory issue

조회 수: 5 (최근 30일)
Amit Chakraborty
Amit Chakraborty 2021년 9월 20일
댓글: Amit Chakraborty 2021년 9월 21일
I have a sparse matrix which has a size of, (2031616x4096 sparse double).
I want to make 10 subset of this matrix and need to convert this matrix in to "full". So, I have made 10 subset and while processing it was able to covert 5 subset into " full" and for the rest of the five subset it is showing me the message : "out of memory".
One thing I have done that is, when I saved the each subset separately into a external harddisk may be at that time it will be possible but here again the problem appears during the importing the subset matrix. I know that it is not a solution .
I will appreciate if anyone can suggest some helpful solution to tackle this problem.
  댓글 수: 3
Walter Roberson
Walter Roberson 2021년 9월 20일
It sounds as if you will have multiple subsets in memory ?
Amit Chakraborty
Amit Chakraborty 2021년 9월 20일
The size of each subset is : 196608x4096 double

댓글을 달려면 로그인하십시오.

답변 (1개)

Steven Lord
Steven Lord 2021년 9월 20일
sz = [196608 4096];
numberOfElements = prod(sz);
bytes = 8*numberOfElements;
gb = bytes/1024^3
gb = 6
Each subset is 6 GB in size (assuming real doubles, 12 if complex doubles) when converted into a full array. Depending on how much memory you have available it's quite possible you couldn't fit all ten subsets in memory at once.
Can you avoid the conversion to a full matrix? MATLAB can save sparse matrices to MAT-files and load them from those files perfectly well. Alternately if you need to store them in a format some other program can read perhaps write a coordinate list (which you can generate using the find function) to the disk?
Alternately clear-ing each subset once it has been written to disk may allow MATLAB to reuse that memory for the next subset.
  댓글 수: 1
Amit Chakraborty
Amit Chakraborty 2021년 9월 21일
** All the subsets are with real value no complex value is there.
** I need to convert the sparse matrix into "full" because i need this for further calculation.
** If it is possible for you to show me an example of writing coordinate list with "find" function?
** Lastly, yes I have been clearning the unwanted variable once I worked with it so that MATLAB can reuse the memory.
Thanks to Everyone !

댓글을 달려면 로그인하십시오.

카테고리

Help CenterFile Exchange에서 Data Type Conversion에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by