Setting up a 'pause - resume' reinforcement learning training using RL Toolbox

I wanted to leverage the reinforcement learning toolbox for training on an actual scaled vehicle. What I am trying to do is as follows:

1) After end of each episode the the training should pause.

2) I will manually reset my physical robot to start location

3) Once I am ready, I will resume the training for next episodes

4) Keep on repeating these steps for ~100 episodes

I am trying tu use the DDPG agent for this task. Is there any way I can achiev this. I am open to using other agents (continuous states - continuous actions) if required.

Answer 1

[Anurag]

Hi Ameya,

I understand that you want to be able to pause and resume your training process at the end of each epoch, and then be able to repeat the steps for 100 episodes. Follow the below steps for being able to do the same:

To implement the pausing and resuming of training, you'll need to create a custom training loop.

Inside your custom training loop, you can implement a mechanism to pause and resume training. You can use a combination of pause, save, and load MATLAB commands to save and load agent checkpoints or training data.

Implement a way to reset the environment or the vehicle to its initial state. You can create a custom function or method that performs the reset operation.

Create a loop that iterates for a specified number of episodes

Sample code attached below for your reference:

% Create your custom training loop 
numEpisodes = 100; 
agent = trainDDPGAgent();  % Create and configure your DDPG agent 
for episode = 1:numEpisodes 
    
    % Perform training for one episode 
    trainingData = trainOneEpisode(agent, episode); 
    % Pause training and manually reset your robot 
    % Implement the logic to save the training data if needed     
    pause;  % Manually reset your robot 

    % Resume training for the next episode 
    if episode < numEpisodes 
        % Load the agent and continue training 
        agent = loadAgent();  % Implement the loading logic 
        resume(agent, trainingData);  % Implement the resuming logic 
    end 
end 

Please refer to the following link for more information:

• https://in.mathworks.com/help/reinforcement-learning/ug/ddpg-agents.html

Hope this helped.

Regards,

Anurag