Profile Total time vs timeit

I am studying the efficience of my code and to do that I was measuring the time spent to run the code. But when I use the function timeit to measure it the anwser is 0.15s and when I use the profile the total time is 0.5s. I dont know why the difference in time by measuring it those functions. Anoyone could help me to understand that?

[Bjorn Gustavsson]

Once uppon a time my guess would've been that it had something to do with the just-in-time compilation-optimization, because then there were some steps that were not used when profiling the code. Now I'm not sure that is an issue, and it seems to be a bit much to explain with timing-related overhead. It might also have to do with the total system-load of your computer, I can see that type of variations if my computer runs hot and starts to swap much to disk.

But, a question to you is: does this difference matter for this task? I've found the profiler suitable to reveal bottle-necks in my code rather effectively.

[Augusto Perboni]

Thank you for yor anwser Bjorn, yes it was great to find the bottle-neck of the code. I asked about that just to try to understand a bit better what was going on. I think you are right and it have some to deal with the compilation optmization.

I wish you the best, thank you for your time

[Bruno Luong]

Profiler is like a measurement device and your code is like a particle in quantum mechanic: you measure it then you perturb it.

Answer 1

[Jan]

The profiler disables (at least parts of) the JIT, the just in time optimization. This happens, because the JIT can reorder commands, if it improves the speed. But then the profiler cannot correlate the processed command with the code lines anymore. This means, that the detailed profiling impedes the processing speed and this is, what the profiler should observe. What a pity.

If the bottleneck is a simple loop with elementary commands, the JIT acceleration can be strong. Then a bottleneck found by the profiler must not be a bottleneck in the real processing.

timeit and tic/toc are less smart und more accurate to compare runtimes between different versions of a code.