Computation Time Reinforcement Learning Toolbox

Hello everyone

Following up on my question here, I have managed to get the example of the ball balancing system running. Now when I try to do the training on my machine, it doesn't take a few minutes as specified, but rather a few hours or days, since the simulation progresses slower than real time. I tried using the parallel computing toolbox, but didn't get significantly better results. I am using a 6 core Intel i5-8500 CPU at 3 GHz. Is this only an issue of my machine, or are there other ways to speed up the simulation?

Cheers

Answer 1

[Ari Biswas]

Training the SAC agent in the ball balance example could take as long as a day, generally speaking. We are working on performance improvements for future releases but as of now you may only get minor performance benefits.

1. Disable the Simscape Mechanics Explorer as mentioned here. You will not be able to see the animation in this case. You can always turn it back on after training is finished.

2. Disable the training visualization by setting the 'Plots' property in the training options to 'none'. You can also set the 'Verbose' property to view progress in the command window instead.

The other (more involving) approach is to experiment with different reward functions or tuning the agent hyperparameters.

[green_ananas]

Thanks Ari, a day sounds about right to me. I was only wondering whether I did something wrong. After trying what you proposed, it seems that the limiting factor is indeed the simulation itself, especially the contact force and friction between the ball and the plate seems to be quite expensive.

[Ari Biswas]

Yes, the demo has high fidelity contact force modeling. You can experiment with different solver settings (solver type, absolute and relative tolerances) to check for any peformance improvements, but at the cost of simulation accuracy.