Disparity between training and evaluation episodes

I tried to re-train this DDPG control example using exact same configuration (except the max training episode set to 500) and this is the result I got:

It is quite different compared to the provided screen capture (below), despite the random seed has been fixed reproducibility purpose.

Why are there huge disparities between training and evaluation episodes on my training? Eventhough my training process was successful (evaluation statistic hit 2000), the last training episodes still couldn't match evaluation episodes. How to properly interpret this result? Is this normal?

Answer 1

[Jacob Mathew]

Hey Ari,

I tried running the example on my end and got a different graph in the training compared to both what you have seen and the screenshot in the example guide. But my output was reproduceable across multiple runs of the example:

However, while the path taken during training does seem to vary, the results logged in the scope are inline with that the example expects, which showcases that the training was successful and predictions are in line: