I have problem with clear GPU memory

After executing this code, the GPU memory is use by 2 GB. Only the D matrix in GPU memory...

A=fix(gpuArray(rand(1,1000))*99)+1;
B=fix(gpuArray(rand(1,1000))*99)+1;
C=gpuArray(rand(100000,100));
E=C(:,A);
F=C(:,B);
D=E.*F;
clear E F C A B

However, if I execute this code.

D=gpuArray(rand(100000,1000));

There we see D matrix (same size) in GPU memory, but now it only use 1 GB of GPU memory. Why is there a difference? and how to clear the memory in the first variant?

[Raymond Norris]

Hi Vitaly,

I don't have a GPU at my disposal. Your code, A..D, consumes 2.37 GB. D alone is 763 MB. So I think that aligns with what you're seeing, except I'm not sure why you think only D is in the GPU memory.

Have you called reset to clear the GPU?

reset(gpuDevice)

Are you using nvidia-smi to check if the memory goes up and down?

Raymond

[Vitaly Bur]

Hi, Raymond!

If add in first code a clear D - the GPU memory is reset to 0.

I see the difference between the variants under the same conditions - before starting the program, of course, I reset memory to 0 and everything is cleared to 0.

When the program ends or when I interrupt its execution, everything is reset to 0. The problem is that along with D, a copy of it is stored in memory, or something else that I don't need.

I using the task Manager: GPU memory.

If I use CPU such problem is not present.

look at the charts. You can see that clear works on the CPU and clear memory. But the clear command doesn't work on the GPU. On the GPU, memory is cleared when all variables are cleared. So it shouldn't be?

Answer 1

[Joss Knight]

MATLAB does not clear all GPU memory unless all variables are released because allocating memory is a performance bottleneck. So MATLAB instead pools memory for later allocations.

To force the GPU to release all memory, you can call reset(gpuDevice).

[Vitaly Bur]

Thanks for the answer.

The problem is that I don't need to clear ALL my GPU memory. I need to clear up memory from data that will not be used, but that takes up expensive memory space.

[Joss Knight]

You can force MATLAB to stop pooling memory with this feature command:

feature('GpuAllocPoolSizeKb',0);

[Vitaly Bur]

The effect appeared, but the memory is not cleared. something is happening (look for the red arrow), but not that.

You can see on CPU is all ok.

[Joss Knight]

What does the red arrow represent? You calling clear E F C A B?

[Vitaly Bur]

Yes. The red arrow is the moment when everything is cleared except D. (clear E F C A B).

To the Right of this arrow is the result of cleaning.

The Blue arrow is when D is cleared.

The Yellow arrow is when an array of D is created again (for to see the size of the array D).

gpu=gpuDevice();
reset(gpu);
gpu=gpuDevice();
disp(gpu)

feature('GpuAllocPoolSizeKb',0);
 
A=fix(gpuArray(rand(1,1000))*99)+1;
B=fix(gpuArray(rand(1,1000))*99)+1;
C=gpuArray(rand(100000,100));
E=C(:,A);
F=C(:,B);
D=E.*F;

pause

clear E F C A B

pause

clear D
 
pause

D=gpuArray(rand(100000,1000));

pause

[Joss Knight]

Oh I see now. This is just because D hasn't been evaluated yet, which means E and F are still being held in memory, as is C because E and F are just slices of C which means it's being held in memory until we attempt to modify them. We use a lazy evaluation optimization and you haven't actually required D to be used or displayed. Insert gather(D) or wait(gpu) to force evaluation, and the memory will be freed.

[Vitaly Bur]

Thanks.A miracle happened - the use of wait(gpu) solved the problem.

gpu=gpuDevice();
reset(gpu);
gpu=gpuDevice();
disp(gpu)

feature('GpuAllocPoolSizeKb',0);
 
A=fix(gpuArray(rand(1,1000))*99)+1;
B=fix(gpuArray(rand(1,1000))*99)+1;
C=gpuArray(rand(100000,100));
E=C(:,A);
F=C(:,B);
D=E.*F;
wait(gpu)
 
pause

clear E F C A B

pause

clear D

pause

D=gpuArray(rand(100000,1000));

pause