필터 지우기
필터 지우기

Deployment of a RL Agent

조회 수: 17 (최근 30일)
Matt
Matt 2021년 7월 2일
답변: Hariprasad Ravishankar 2022년 12월 26일
Hello!
I'm using Matlab 2021a to design a reinforcement learning agent. I am in the process of training a series of agents and I am (hopefully) nearing deployment. However, I have a few questions that I have not been able to find direct answers to.
First, can I deploy a RL agent in Simulink Real-Time? I have a Speedgoat available and I have done significant work to close the hardware loop around my agent in Matlab, but I haven't seen anything that says in a straightforward manner that I can deploy it in Simulink. I saw here that you can deploy CUDA code in Simulink but I don't really understand the process. I'm not sure if it applies to this situation or if there is a more straigtforward way to deploy an agent in Simulink.
Second, if I cannot deploy a RL agent in Simulink, what other platforms can I deploy a RL agent in? I would prefer to deploy the agent in Matlab or Simulink since I've done a lot of the hardware work in Matlab, but I'm not sure what to do if Simulink isn't an option. I know that I can hyptothetically generate C and CUDA code, but I'm nout sure what the next step is.
And lastly, I clearly am lacking some vital knowlege on this front - it turns out I'm an electrical engineer, not a computer scientist! What resources should I use to further my understanding of this? I'd rather understand what I need to do than ask others for help. I don't really know anything about CUDA or deploying programs on actual hardware and I'm not sure where to look... and it turns out that there's a pretty steep learning curve.
Thanks!

채택된 답변

Drew Davis
Drew Davis 2021년 8월 2일
Hi Matthew
The RL toolbox has a feature: generatePolicyFunction - which will generate a static MATLAB policy function from an RL agent object. This function can be placed inside a MATLAB Function block for code generation.
Note 1, the Speedgoat platform does not support compiling against any third party deep learning libraries, so the model parameter DLTargetLibrary is restricted to be "none". This limits the which layers can be used for deployment. You can check which built-in layers are supported via the following command:
coder.getDeepLearningLayers("TargetLibrary", "none")
Note 2, custom layers (such as the RL scalingLayer) are supported by the "none" target library.
Hope this helps
Drew
  댓글 수: 2
Drew Davis
Drew Davis 2021년 8월 5일
Correction - the RL scalingLayer is not supported in 21a. Note:
supportedLayers = coder.getDeepLearningLayers
Will list ALL shipping layers that are supported by code generation.
Filip Poloczek
Filip Poloczek 2022년 3월 6일
Hi,
I'm trying to use an trained agent on a hardware. The training went well and I have an action space between -9 and 9, but if I use generatePolicyFunction() then the resulting evaluatePolicy() returns only values between -1 and 1. Do you have a solution for this?

댓글을 달려면 로그인하십시오.

추가 답변 (1개)

Hariprasad Ravishankar
Hariprasad Ravishankar 2022년 12월 26일

카테고리

Help CenterFile Exchange에서 Get Started with GPU Coder에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by