I use reinforcement learning to optimize the controller parameters of the control system, but after a few rounds of training, the code will report an error, indicating that at

I use reinforcement learning to optimize the controller parameters of the control system, but after a few rounds of training, the code will report an error, indicating that at 500s, the state "1" is not differentiable, and there may be a singularity in the solution.

I have been troubled by this problem for a long time and I hope it will be resolved.