Set gpu option for rlPPOAgent actor

Hi,

Is there a way to set the gpu option for an rlPPOAgent actor?

agent = rlPPOAgent(obsInfo,actInfo,agentOpts);

critic = getCritic(agent);

critic.Options.LearnRate = 1e-3;

critic.Options.UseDevice = 'gpu';

agent = setCritic(agent,critic);

I was able to set the critic to gpu but need to know how to set it for the actor. agentOpts is rlPPOAgentOptions, which doesn't take UseDevice.

Thanks!

Answer 1

[Emmanouil Tzorakoleftherakis]

What you have specified is sufficient for the critic. If you do the same for the actor you are all set - there is no additional param for the agent overall (actor and critic are two key components of the agent after all)

[John Doe]

Sorry, I meant this question for the actor, not the agent.

[John Doe]

How would I do the same for the actor?

[John Doe]

The confusion is because there is no actor definition at all in this:

agentOpts = rlPPOAgentOptions(...

'ExperienceHorizon',2000,...

'ClipFactor',0.2,... % how much change is allowed each step

'EntropyLossWeight',0.01,...

'MiniBatchSize',1000,...

'NumEpoch',3,...

'AdvantageEstimateMethod','gae',...

'GAEFactor',0.95,...

'SampleTime',0.1,...

'DiscountFactor',0.9);

agent = rlPPOAgent(obsInfo,actInfo,agentOpts);

critic = getCritic(agent);

critic.Options.LearnRate = 0.1;

critic.Options.UseDevice = 'gpu';

agent = setCritic(agent,critic);

[John Doe]

this is what it's showing during training:

[John Doe]

if this isn't possible, can we generate the actor net definition and modify things in that ourselves? I'd like to change the learn rate for it also, which might be another buried option.

[John Doe]

I could use some guidance here to run fully on GPU. My env runs pretty slow right now

[Emmanouil Tzorakoleftherakis]

I believe I already answered this question - for the actor you need to do the same thing you did for the critic.

actor = getActor(agent);
actor.Options.UseDevice = 'gpu';
agent  = setCritic(agent,actor);

If the environment itse;f is simulating slowly GPUs will not help as much. You need to check your model and identify what's slow