Saving simulation data during training process of RL agents

조회 수: 21 (최근 30일)
Nima Mathwork
Nima Mathwork 2021년 1월 25일
댓글: awcii 2023년 7월 30일
Hello everyone
Using the Logging tab of the scope blocks inside the Simulink, I managed to save the data to the MATLAB workspace and then save them to a text file. I need to do the same thing for every iteration (episode) during an RL agent's training process. I wonder how I can similarly save the log data for further analysis and visualization.

답변 (2개)

Ari Biswas
Ari Biswas 2021년 5월 19일
Elaborating on Emmanouil's suggestion:
There are two ways to log and visualize data during training.
Option 1 is to use the training statistics structure (output of the train function). You must enable the Signal Logging checkbox under Model Settings > Data Import/Export. Then specify the signals to log in the Simulink model. This can be easily done by right clicking a signal and selecting Log Selected Signals. After the training is complete you can access this data from the SimulationInfo field in the training statistics struct.
Option 2 is to use the Simulation Data Inspector (SDI). With this option you can visualize the data as the simulations are completed during training. You would need to enable the Record logged workspace data in SDI option under Model Settings > Data Import/Export in the Simulink model, then specify all signals to log in the model. If training in parallel, execute the following command prior to starting training for making the SDI compatible with parallel simulations. Parallel workers will send logged data to the SDI as the simulations are completed.
Simulink.sdi.enablePCTSupport('local') % if the parallel workers are from the local cluster
The SDI also has options to export logged data to the MATLAB workspace.
  댓글 수: 1
awcii
awcii 2023년 7월 30일
how can find the SimulationInfo ? Should it be in workspace ?

댓글을 달려면 로그인하십시오.


Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2021년 1월 28일
Have you tried logging the data with Simulation Data Inspector? Make sure to pick only signals you actually need since depending on duration of training, you might end up saving a lot of data.
  댓글 수: 2
Nima Mathwork
Nima Mathwork 2021년 2월 1일
Thank you @Emmanouil Tzorakoleftherakis for your answer. I think using Simulation Data Inspector is a proper solution. However, I could not find an appropriate example in the documentation. Now, I managed to use the cell array, which is suggested here. I would be grateful if you can indicate an example for using Simulation Data Inspector. As a reminder, I need the raw data to draw my custom plots.
Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2021년 2월 1일
Maybe this page and link included are helpful. I don't think there is an example specifically for reinforcement learning in the documentation.

댓글을 달려면 로그인하십시오.

카테고리

Help CenterFile Exchange에서 Sequence and Numeric Feature Data Workflows에 대해 자세히 알아보기

제품


릴리스

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by