Memory increases on GPU while performing modification inplace

When I allocate squeeze a 3D array to output a 2D array, my memory is increasing while I think I performing the modification inplace. That way I would expect that since the number of elements in the array does not incease, the memory used on the GPU does not increase.

However, I see something different in the task manager. What is going on?

• MATLAB Version: 9.8.0.1538580 (R2020a) Update 6
• Operating System: Microsoft Windows 10 Enterprise Version 10.0 (Build 18363)
• NVidia GeForce RTX 3090

data = zeros(5632000, 128, 2, 'int16');
testSO(data); 

function testSO(RF)
    whos RF % int16 2.75 GB
    size(RF); % 5632000x128x2  
    % GPU memory at beginning 1.2 GB
    RF = gpuArray(RF); % 3.9 GB
    RF = squeezingSuperFrames(RF); % 6.6 GB
end

function RF = squeezingSuperFrames(RF)
    % concatenate pages of 3D array to create tall RF array
    RF = reshape(permute(RF, [1 3 2]), [], size(RF, 2), 1);
end

Answer 1

[Joss Knight]

MATLAB cannot perform this operation in place because data is a workspace variable. MATLAB has no way of knowing there won't be an error or user interrupt (Ctrl-C) during execution, so it takes a copy of data to ensure your workspace would not be corrupted.

[rinkert]

@Joss Knight Thanks Joss for the reply. I get that Matlab makes a copy of data, but the operation on the variable RF is actually not related the workspace variable data anymore? I mean I get that there might be a copy of the data on RAM, but not an additional copy in graphics memory, right?

Anyway, is there a solution in this case, that would allow me to not use twice as much GPU memory after the 'squeezing'? I expected that since I am performing this operation in a local function workspace, that I would surpass the risk of a corrupted base workspace.

[Joss Knight]

Good point, I wasn't paying proper attention to when you were moving data to the GPU. As Matt points out, your culprit is permute.

MATLAB may be still holding onto the memory, but only because it is pooled, it is not preventing its use. This is a performance optimization. You can see that this memory is available for use by other MATLAB variables and operations by inspecting the AvailableMemory property in the output of gpuDevice.

[rinkert]

Thanks for the pointer to AvailableMemory, that indeed showed that there is still memory available while nvidia-smi shows it up as in use.

Answer 2

[Matt J]

reshape() does not result in data copying, but permute() does. It cannot be done in place, because it is reordering the data in memory.

[rinkert]

I see, thanks @Matt J, what would be the solution in this case to free up the memory after the permute operation.

[Matt J]

The memory should be freed after the function call is complete. On my GPU (GTX 1080 Ti), this is what occurs. Maybe you should reset your GPU or even reboot the system to make sure your GPU didn't get stuck in some weird state.

[rinkert]

yes indeed the memory is freed after the function call is completed. Seems like the memory is already available before, if I look at g=gpuDevice(); g.AvailableMemory before the function exits.