Problem of using DDPG agent in external mode

I am working on Reinforcement Learning to control coupled-tank system. I created a simulink environment for Deep Deterministic Policy Gradient (DDPG) agent. When I run the model in normal mode it works well. But when I choose the external mode it gives the below error. I would be happy you to help in this problem. Best regards

MATLAB System block 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1/AgentWrapper' error occurred when invoking 'getSampleTime' method of 'AgentWrapper'. The error was thrown from '

'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 150

'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 210

'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 265

'C:\Program Files\Polyspace\R2019b\toolbox\rl\rl\simulink\libs\agentmaskinit.m' at line 11

'C:\Program Files\Polyspace\R2019b\toolbox\rtw\rtw\tlc_c.m' at line 199

'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 886

'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 218

'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\@ModelCodegenMgr\make_rtw.m' at line 8

'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder_core\make_rtw.m' at line 20

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\build_standalone_rtw_target.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\private\on_target_one_click.p' at line 0

'C:\Program Files\Polyspace\R2019b\toolbox\simulink\simulink\sl.m' at line 15

'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 119

'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 129

'C:\Program Files\Polyspace\R2019b\toolbox\coder\simulinkcoder\toolstripHW\m\+coder\+internal\+toolstripHW\CustomHardwareBoardHandler.m' at line 107

'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 84

'C:\Program Files\Polyspace\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 242'.

The RL Agent block only supports Normal and Accelerator simulation modes.

[Anirban]

The paths in the message seem not right to me. They refer to C:\Program Files\Polyspace\R2019b\ whereas the paths should be C:\Program Files\MATLAB\R2019b\.

Do you happen to have a Polyspace installation and did you explicitly addpath folders from the Polyspace installation in your MATLAB? If so, you can check your path variable, then rmpath those folders and see if the errors go away. The steps to integrate Polyspace and MATLAB are given in Integrate Polyspace with MATLAB and Simulink, and if you follow these steps, there should not be a problem.

[mehmet bozuyla]

I tried the method which you mentioned on another computer and similarly got the following error again.

Error in 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1': Initialization commands cannot be evaluated.

Caused by:

MATLAB System block 'rlwatertankControl/Coupled-Tank Simulation/RL Agent1/AgentWrapper' error occurred when invoking 'getSampleTime' method of 'AgentWrapper'. The error was thrown from '

'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 150

'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 210

'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\AgentWrapper.m' at line 265

'C:\Program Files\MATLAB\R2019b\toolbox\rl\rl\simulink\libs\agentmaskinit.m' at line 11

'C:\Program Files\MATLAB\R2019b\toolbox\rtw\rtw\tlc_c.m' at line 199

'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 886

'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\ModelBuilder.m' at line 218

'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\+coder\+internal\@ModelCodegenMgr\make_rtw.m' at line 8

'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder_core\make_rtw.m' at line 20

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_target.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\build_standalone_rtw_target.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\slbuild_private.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\private\on_target_one_click.p' at line 0

'C:\Program Files\MATLAB\R2019b\toolbox\simulink\simulink\sl.m' at line 15

'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 119

'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardHandler.m' at line 129

'C:\Program Files\MATLAB\R2019b\toolbox\coder\simulinkcoder\toolstripHW\m\+coder\+internal\+toolstripHW\CustomHardwareBoardHandler.m' at line 107

'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 84

'C:\Program Files\MATLAB\R2019b\toolbox\coder\sltoolstrip_base_hw\m\+coder\+internal\+toolstrip\HardwareBoardContext.m' at line 242'.

The RL Agent block only supports Normal and Accelerator simulation modes.

Answer 1

[Emmanouil Tzorakoleftherakis]

The root of the error is likely on Reinforcement Learning Toolbox side, not Polyspace. As the last error line mentions,

"The RL Agent block only supports Normal and Accelerator simulation modes."