- WDDM2 releases 90% of available memory to CUDA.
- A single application is only allowed to allocate 90% of that 90%, i.e. 81% of total memory.
- NVIDIA are working with Microsoft to tighten this bound, and/or remove it on non-display cards.
Matlab is able to utilize only a part of actual available VRAM.
조회 수: 9 (최근 30일)
이전 댓글 표시
It is known that Windows10 with WDDM2.2 reports less than actual VRAM present. For instance with a GTX-1080-TI having 11GB of memory is being reported as only 9GB of free available. However Microsoft claims that as soon as an application claims more memory, VRAM becomes available, so the entire 11GB of VRAM is actually available to the user.
Just to note that the GTX-1080-TI is the 2nd GPU, there is another low profile GPU that has the monitors attached to it as dedicated for display (Set from NVIDIA control panel)
Now, Matlab reports 9.02GB of available VRAM using the 'gpudevice' command. However, this becomes a hard-limit within Matlab. Meaning I cannot declare a GPU variable larger than the available 9.02GB.
What can be done to use the additional 2GB of actual physical memory that is available on the GPU but not accessible to Matlab. This is a very expensive resource and very useful and would like to use it. Can Matlab not claim the memory from the OS (Windows10) dynamically? I am using Matlab R2017a and system spec is 128GB main memory with 2 Xeon CPUs each with 6 cores running at 2.6GHz.
댓글 수: 0
채택된 답변
Joss Knight
2018년 9월 12일
NVIDIA have responded to confirm that this is expected behaviour. In summary:
댓글 수: 11
Joss Knight
2018년 9월 12일
It would benefit other users of the forum if you could accept this answer in both your posts. Thanks.
추가 답변 (5개)
Joss Knight
2018년 8월 5일
편집: Joss Knight
2018년 8월 5일
The division of GPU memory into compute and graphics is dictated by the driver - it isn't something that a CUDA application has any control over. I'm not really sure what you're referring to with regards more memory becoming available once requested. By the time you call gpuDevice, the CUDA runtime has been initialized and it takes a large chunk of memory, then MATLAB preallocates a bunch of slabs of GPU memory, so maybe you've lost about 1GB to graphics and about 1GB to MATLAB and MATLAB's CUDA context. So 9GB sounds about right.
I suppose you could start MATLAB using softwareopengl - maybe that would save some video memory? Probably not though, given you're not actually driving the display with this card.
댓글 수: 5
Joss Knight
2018년 8월 6일
Okay, I've put your question to NVIDIA for you. It would be nice if there were indeed some magic we could do to get Windows to give us back some memory; but to be honest, if there were, I would have expected to have heard of it by now.
J. McCabe
2018년 9월 8일
편집: J. McCabe
2018년 9월 8일
Pavel, I've been getting exactly the same numbers as you with my 1080ti, and similar numbers with my 1070. And Windows 10 Task Manager seems to agree that the maximum utilization I can get on my 1080ti is 9 GB with Matlab plus another 0.6 GB of other stuff on the system. However, when I go into Visual Studio and run the CUDA "cudaMemGetInfo" to obtain free and total GPU memory bytes right after I fill the memory with a 9 GB gpuArray it shows used memory is 10.7 GB, not 9.6 GB like Task Manager and Matlab show. cudaMemGetInfo reports free and total, and I take the difference to obtain "used". Below is the link to my posts on the subject:
Pavel Sinha
2018년 9월 9일
댓글 수: 1
J. McCabe
2018년 9월 9일
편집: J. McCabe
2018년 9월 9일
Yeah, the strange thing is that when you make a gpuArray in Matlab and you KNOW exactly the amount of VRAM it requires, and you KNOW the starting amount of VRAM used by the other processes (from GPUs in Task Manager), and that adds up to 9.6 GB, and if you increase the array size by a tiny amount and it crashes, you're pretty sure of the amount of memory that can be used by all the processes is 9.6 GB. Though maybe it holds 1 GB for a different process to come along and request it, and won't give it out to existing processes that are using VRAM. But then at the same time CUDA tells you you're actually using over 1 GB more than that (ie, 10.7 GB). I have a hard time believing that Microsoft or NVIDIA would deliberately hold out VRAM just to be nasty, or that they'd block existing processes from accessing VRAM that nobody else has asked for. It's all pretty strange, IMO. By the way, here's the CUDA code I used in Visual Studio:
int main()
{
cudaSetDevice(0);
// show memory usage of GPU
size_t free_byte;
size_t total_byte;
cudaError_t cuda_status = cudaMemGetInfo(&free_byte, &total_byte);
if (cudaSuccess != cuda_status) {
std::cout << "Error: cudaMemGetInfo fails, " << cudaGetErrorString(cuda_status) << std::endl;
exit(1);
}
double free_db = (double)free_byte;
double total_db = (double)total_byte;
double used_db = total_db - free_db;
std::cout << "GPU memory usage: used = " << used_db / 1024.0 / 1024.0 / 1024.0 <<
" GB, free = " << free_db / 1024.0 / 1024.0 / 1024.0 <<
" GB, total = " << total_db / 1024.0 / 1024.0 / 1024.0 << " GB" << std::endl;
return 0;
Pavel Sinha
2018년 9월 9일
댓글 수: 2
J. McCabe
2018년 9월 9일
편집: J. McCabe
2018년 9월 9일
Looks like I was able to utilize 10.8 GB out of my 1080ti's 11 GB. I started with my Matlab code that loaded the gpuArray up to what it saw as the max available, 9 GB. At the time there was also some system stuff using 0.6 GB. But then I opened a 3D rendering app which used about 2GB of GPU memory, and it loaded fine. In fact Task Manager showed 10.8 GB out of 11 GB of GPU memory "dedicated". So maybe what Microsoft is saying actually IS true. Maybe any one individual process can't access more than 9 GB, but if another process comes along and asks for VRAM it will get it. Now when I look in Task Manager under the Details tab, where you can see the Dedicated & Shared GPU Memory for each process, it shows Matlab taking 8.6 gB and the 3D rendering process taking 2.1 GB, which indeed totals 10.7 GB. BTW, both are also taking some "shared" GPU memory, which I believe is system RAM. There are also some other system processes taking GPU memory at the same time, amounting to maybe 0.2 GB or less. Interesting...so maybe that's the story. Maybe any one process can't access more than 9 GB on a 1080ti, but if someone else comes along it can grab whatever is remaining, so that the total utilization is just under 11 GB.
Joss Knight
2018년 9월 9일
I note that the second application only requires graphics memory, not compute. Have you tried starting a second MATLAB and seeing how much memory you can allocate between them?
Ultimately WDDM is a video driver and I suspect that any memory held back is only released for graphics.
Pavel Sinha
2018년 9월 9일
댓글 수: 1
J. McCabe
2018년 9월 9일
Pavel thats interesting that even the same application can request more and get it. Like I said I tried to increase the Matlab gpuArray size slightly above the available 9 GB and it crashed on "out of memory". Maybe it needs to be a separate process or request or something. Or like Joss suggests another instance of Matlab
J. McCabe
2018년 9월 11일
So I tried what Joss suggested, and ran two instances of Matlab. Both instances had the same code, which checks available GPU memory, and then fills a gpuArray with just enough elements to fill that amount of memory. Below is an image of Task Manager/Details, which shows dedicated and shared GPU memory for both processes. The first one (on top), grabbed about 8 GB, and the second one took 2 GB of GPU memory and almost 7 GB of shared memory. I believe that means it grabbed some system RAM in order to make both arrays?
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/195386/image.jpeg)
댓글 수: 2
Joss Knight
2018년 9월 11일
I can't really explain that; MATLAB doesn't deliberately allocate any unified memory (which is what the Task Manager could be showing), but the CUDA runtime and CUDA libraries might, so perhaps that's what that is, depending on what your script is doing; it could also be the behaviour of the WDDM driver which is allocating resources out of CPU memory.
If you look at dedicated GPU memory that now adds up to 10.4 GB, which is much closer to 11 GB than before; maybe this issue is resolved?
Given that MATLAB's way of measuring available memory depends on the driver, you may find a more reliable way of packing memory is to allocate arrays of larger and larger size until MATLAB throws an out-of-memory error.
J. McCabe
2018년 9월 11일
FWIW, here's the code I'm using to fill a gpuArray just under what is returned as "available":
g = gpuDevice(1);
bytestoGB = 1073741824.0;
fprintf(1,'Device %s has Total Installed Memory %.2f GB \n', g.Name,g.TotalMemory/bytestoGB)
fprintf(1,'Device %s has Available Memory %.2f GB \n', g.Name,g.AvailableMemory/bytestoGB)
% Determines the number of 8-byte ints that are needed to fill GPU VRAM
% based on how much is available. For 1080ti that is a 34809 x 34809 array,
% and for 1070 it's a 29693 x 29693 array
elements=uint64((sqrt(g.AvailableMemory/8))-1);
%
% % Creates an array of int64's on GPU
Array1=gpuArray(ones(elements, 'int64'));
%
% % Figure how much memory is used by the array
el = cast(elements,'double');
arrayMemory = (el*el)*8.0/bytestoGB;
fprintf(1,'Array has %i x %i 8-byte (int64) elements \n', elements, elements)
fprintf(1,'Array consumes: %.3f GB \n', arrayMemory)
fprintf(1,'Device %s has Available Memory %.4f MB after array fills GPU \n', g.Name,g.AvailableMemory/1024.0/1024.0)
fprintf(1,'Total GPU Memory minus Array Memory = %.2f GB \n', (g.TotalMemory/bytestoGB-arrayMemory))
참고 항목
카테고리
Help Center 및 File Exchange에서 Kernel Creation from MATLAB Code에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!