Problems with MaxPossibleArrayBytes dropping while overall memory use does not change

I a have a code where a data is processed in a long loop. Code is implemented a OOP class and all the memory is preallocated. Processing is going well until I suddenly I run out of memory. I started monitoring memory usage using "memory" function user.MemUsedMATLAB and class size never changes, but user.MaxPossibleArrayBytes seems to oscillate a lot with occasional spikes where minimum MaxPossibleArrayBytes goes to fraction of the median value. During one of such spikes my code crashes when minimum goes to 0.01% of the median value.

I am not sure how to debug this kind of issue or what might be causing it. I am running on a machine with 24 cores and 32 Gb of memory.

Answer 1

[Walter Roberson]

Generally speaking, MaxPossibleArrayBytes can reduce as a result of memory fragmentation. This is a significant problem with 32 bit MATLAB, but it is not usually a problem with 64 bit MATLAB unless you are using very large arrays.

[Jaroslaw Tuszynski]

I am using 64 bit system and I do keep around some big arrays (11k x 300 x 8), but I do not do much with them other than read and write to them. What seems to get me in trouble is "waterfall plot" that performs:

 im_wf = circshift(im_wf,[1 0 0]);  % shift all the lines down
 im_wf(1,:,:) = newline; % overwrite the first line

or

 im_wf = cat(1,newline, im_wf(1:end-1,:,:));

operations on much smaller array (700 x 300 x 3). I wonder if those operations are somehow more prone to defragment memory.