DQN training getting worse

Hello all,

I am trying to train a very simple model with DQN. When I increase the trainning steps, the average/episode reward gest worse and worse. The agent gives deseriable result at a much earlier episode than a later ones.

This is what I had for average reward and epsidoe reward. I know for sure this amount of trainning is way more than sufficient.

This is the parameters I had for this training:

Can anyone help to explain why that happens and what can to do to avoid that from happening?

Thanks