Reinforcement learning with Simulink and Simscape

Hi,

improving control of physical systems with Reinforcement learning is tempting and might lead to real improvements.

If we considere a physical domain like Thermal for example, reinforcement learning would require to build a model to achieve offline training.

Are there limitations that would prevent the use of Simulink+Simscape to do this (algebraic loop issues for example)?

BR

David

Answer 1

[Emmanouil Tzorakoleftherakis]

Even outside the thermal domain, you most likely need to start with a simulation model. RL does not need to build that model necessarily (e.g. model-free methods).

You can certainly use Reinforcement Learning Toolbox with Simulink and Simscape without issues, assuming you have a Simulinm model that simulates without errors outside of RL. You may still run into algebraic loops depending on the problem, but these do not prevent you from applying RL (quick fix is to try adding a delay).

[Karim Darwich]

Hello @Emmanouil Tzorakoleftherakis

I have that algebraic loop problem because my Agent specifies the mass flow but the initial condition of my unit delay changes the simulation it's like the initial condition specifies the mass flow not the agent calculation do you have any idea how to solve this kind of problem