GPU computations slowing down

조회 수: 8 (최근 30일)
Peter
Peter 2017년 1월 5일
댓글: Joss Knight 2017년 1월 8일
Within a loop, my code carries out some iterative solving procedure on the GPU. With every loop iteration, the time taken by the solving procedure increases. There doesn't seem to be a memory leak: gpuDevice consistently reports approximately the following:
TotalMemory: 1.2885e+10
AvailableMemory: 5.6773e+09
However, when I pause the loop for a while by means of a debug stop, after relaunching again, computations are up to speed again, after which they start slowing down again...
I'm running matlab2016b on Windows 10, using a Nvidia Titan X (Pascal) GPU. Any suggestions for this problem?
  댓글 수: 3
Jan
Jan 2017년 1월 5일
Without seeing the code or knowing any details, it is hard to guess, what's going on. Did you control the temperature of the graphics board?
Peter
Peter 2017년 1월 6일
thanks Jan, I already started to be suspicious about temperature and monitored it. After launching the matlab script, temperature starts rising, GPU performance starts being capped by Temperature. (See performance monitor screen shot below) After reaching 95(!) degrees, the crash occurred again. So we do have strong indications for a cooling issue... What does this mean?
  • a) I got myself a wrong system set-up, need better cooling ?
  • b) this GPU card doesn't suit my needs, perhaps go for one of the Keplers (more expensive for similar memory)?

댓글을 달려면 로그인하십시오.

채택된 답변

Peter
Peter 2017년 1월 6일
Monitoring the GPU performance revealed that most probably the temperature is causing the issue: Slowing down of performance goes with rising of temperature and performance is capped by temperature.
Crash of the GPU occurred when GPU reached 95 degrees...
  댓글 수: 1
Joss Knight
Joss Knight 2017년 1월 8일
Most people who report issues like this are simply seeing an issue of asynchronous execution. They see fast times until the kernel execution queue is full, and then they start seeing the actual complete computation times.
To avoid this try using wait(gpuDevice) to add synchronisation points to your code.

댓글을 달려면 로그인하십시오.

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 GPU Computing에 대해 자세히 알아보기

제품

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by