How does Matlab allocate GPU memory ?

I am using Matlab 2017b in conjunction with 2 NVIDIA Pascal type GPUs. I have been writing a mex File that uses "mxGPUCreateGPUArray" and "mxGPUCreateFromMxArray" to allocate Memory on the GPU and make it accessible from Matlab. Then I have been using the CUDA function "cudaMemcpyAsync" to copy data from system/host Memory to GPU Memory.

Now, the new NVIDIA Pascal architecture has all these fancy Features like for example "Unified Memory". Question: How can I use the corresponding Memory allocation function "cudaMallocManaged" for allocating unified memory under Matlab ?

Or other way around: what CUDA functions exactly (if any) does Matlab use in "mxGPUCreateGPUArray" and "mxGPUCreateFromMxArray" ?

I can see in the NVIDIA API there exist 3 different memory allocation functions "cudaMalloc" ("standard" way of allocating memory directly from GPU memory), "cudaMallocHost" (allocate page-locked memory on the host/in system memory for fast copy operation from host memory to GPU memory), "cudaMallocManaged" (allocate "unified" Memory, transparently visible under same address from the GPU as well as the CPU) but I don't see which of these functions Matlab uses and how it uses them.

Answer 1

[Joss Knight]

MATLAB allocates memory when necessary using the driver API. But it maintains a memory pool and avoids raw allocation if possible. This considerably increases performance. So, to answer your question, most of the time when you call "mxGPUCreateGPUArray" and "mxGPUCreateFromMxArray", no allocations will happen at all.

Unified memory doesn't replace data transfers, it just places data transfer between CPU and GPU in the hands of the CUDA runtime. MATLAB's model is to place data transfer in the hands of the user.

Future versions of MATLAB may seek to leverage Unified Memory to extend the memory capacity of the GPU, although this will probably have to wait until the minimum supported architecture is Pascal, since that is the first with efficient paging.

Of course, the GPU MEX API is intended to help the user pass GPU data to and from the MATLAB workspace in the form of gpuArray variables. If you don't intend to use them, you can allocate memory however you like in a MEX function.