JVM out of memory when running parfor on cluster

I'm running parfor on cluster from matlabpool open and getting JVM out of memory error heap size. The same parfor works perfect when I open matlabpool with 'local' profile and run it just on one machine.

MDCS is installed properly and passes validation.

For sure its not lack of memory on remote cluster PC - it has plenty of memory left and system is win7 64. Both PCs has 16G of memory,one is 8 core,another 4 core.

Java heap size is set to 2048MB, java.opts is set to 4096MB and heap size in java application is also set high. All those settings works well either for serial or for local parfor.

Krzysztof

Answer 1

[Thomas Ibbotson]

It's possible that the worker JVMs are not picking up the java.opts file. Workers have a different startup folder to the client MATLAB, so in order to get them to pick up the java.opts I would recommend putting it in the matlab/bin/<arch> folder.

You can check the java heap memory on the workers by running the following command with a pool open:

pctRunOnAll('r = java.lang.Runtime.getRuntime(); r.maxMemory / (1024*1024)')

The first result will be the memory on the client machine followed by the memory on each worker in MB.

[Krzysztof Fajst]

Yes it was it !!! When I open matlabpool on cluster heap memory was set to default. So I put java.opts kind of everywhere to /bin, /bin/w64 and /MATLAB, it was still not OK but after a reboot heap was set higher.

Thanks, Krzysztof