gpu computing vs complex operation

Hi , I have a question, is there any difference between gpu and cpu while they are handling operation that involve with complex ?

Answer 1

[Walter Roberson]

Yes, there is.

1. NVIDIA GPUs do not adhere strictly to IEEE 754 standards, especially with respect to signalling NaN and gradual underflow and rounding modes and details of permitted error in the last few bits
2. The order of operations is not guaranteed to be the same between CPU and GPU. GPU kernel generation optimizes operations for efficiency, not for strict adherence to MATLAB or C or C++ order of operation rules
3. The high performance math libraries used on CPU are not the same as the libraries used on GPU

In short, you should expect that the results calculated on GPU for any floating point operations may differ from the results on CPUs.

[Kevin Lin]

Do you mean that the error might(for sure) be greater on gpu than cpu computing? but the computation is faster

[Walter Roberson]

If you are using MATLAB for anything other than very simple floating point operations, you have no control over whether MATLAB will be able to recognize the pattern of computation you are doing and transform it into calls to high performance floating point libraries that will have different error than doing the same thing according to strict MATLAB-defined order of operations in a loop would get you. For example, for sufficiently large arrays, MATLAB might end up transforming D = A + B.*X + C into temp = A+C, D = B.*X + temp in calls to high performance routines.

If you are concerned about least error, you probably should not be using MATLAB -- not unless you have very carefully coded a lot of calls to "compensated sum" routines such as https://www.mathworks.com/matlabcentral/fileexchange/26800-xsum and "high accuracy SVD" such as discussed at https://www.mathworks.com/matlabcentral/answers/331745-svd-high-accurancy-demmel-kahan-algorithm

It is not that Mathworks does not care about low error: Cleve is famous for designing some of the fundamental high accuracy algorithms. The problem is that MATLAB does not document accuracy concerns, and does not document how to enable or disable strict order of operations (as written in the code), and does not document how to change rounding modes or the circumstances under which rounding modes can get changed underneath you. Thus the error considerations in MATLAB are unreliable and uncontrolable . And remember that the high performance libraries do sometimes change between releases, and when they change can depened upon operating system and on whether you are using Intel or AMD CPUs.

What you can say about CPU vs GPU is not that error is greater on GPU, but rather that error is likely to be different on GPU. You would have to do a lot of work to determine the circumstances under which one was "better" than the other... and the result might differ with the next patch update.

[Kevin Lin]

Thanks for your answer.

So GPU does handle complex operation, but in a different way that CPU does

[Walter Roberson]

Yes.

Also, the exact result you get out can depend upon the release of nvcc that you have installed.

[Edric Ellis]

@Walter JFTR, the version of nvcc installed by a user can influence only results computed by CUDAKernel or GPU MEX files compiled with their nvcc. The precise version of the CUDA driver can affect results of previously compiled code (e.g. MATLAB, PCT, and the CUDA libraries it ships).