why the continuous gaussian actor function is not working?

why the continuous gaussian actor function is not working and giving me this bug "Unrecognized field name "obsInfo""?

[KSSV]

Show us the full code and specify the error.

[Ehab]

sure,

env = RocketLander;

actionInfo = getActionInfo(env);

obsInfo = getobsInfo(env);

numObs = obsInfo.Dimension(1);

numAct = numel(actionInfo.Elements);

Ts = 0.1;

rng(0)

criticLayerSizes = [400 300];

actorLayerSizes = [400 300];

criticNetwork = [

featureInputLayer(numObs,'Normalization','none','Name','observation')

fullyConnectedLayer(criticLayerSizes(1),'Name','CriticFC1', ...

'Weights',sqrt(2/numObs)*(rand(criticLayerSizes(1),numObs)-0.5), ...

'Bias',1e-3*ones(criticLayerSizes(1),1))

reluLayer('Name','CriticRelu1')

fullyConnectedLayer(criticLayerSizes(2),'Name','CriticFC2', ...

'Weights',sqrt(2/criticLayerSizes(1))*(rand(criticLayerSizes(2),criticLayerSizes(1))-0.5), ...

'Bias',1e-3*ones(criticLayerSizes(2),1))

reluLayer('Name','CriticRelu2')

fullyConnectedLayer(1,'Name','CriticOutput', ...

'Weights',sqrt(2/criticLayerSizes(2))*(rand(1,criticLayerSizes(2))-0.5), ...

'Bias',1e-3)];

criticNetwork = dlnetwork(criticNetwork);

criticOpts = rlOptimizerOptions('LearnRate',1e-4);

critic = rlValueFunction(criticNetwork,observationInfo);

actorNetwork = [featureInputLayer(numObs,'Normalization','none','Name','observation')

fullyConnectedLayer(actorLayerSizes(1),'Name','ActorFC1', ...

'Weights',sqrt(2/numObs)*(rand(actorLayerSizes(1),numObs)-0.5), ...

'Bias',1e-3*ones(actorLayerSizes(1),1))

reluLayer('Name','ActorRelu1')

fullyConnectedLayer(actorLayerSizes(2),'Name','ActorFC2', ...

'Weights',sqrt(2/actorLayerSizes(1))*(rand(actorLayerSizes(2),actorLayerSizes(1))-0.5), ...

'Bias',1e-3*ones(actorLayerSizes(2),1))

reluLayer('Name', 'ActorRelu2')

fullyConnectedLayer(numAct,'Name','Action', ...

'Weights',sqrt(2/actorLayerSizes(2))*(rand(numAct,actorLayerSizes(2))-0.5), ...

'Bias',1e-3*ones(numAct,1))

softmaxLayer('Name','actionProb')];

actorNetwork = dlnetwork(actorNetwork);

actorOpts = rlOptimizerOptions('LearnRate',1e-4);

actor = rlContinuousGaussianActor(actorNetwork,obsInfo,actionInfo);

agentOpts = rlPPOAgentOptions(...

'ExperienceHorizon',600,...

'ClipFactor',0.02,...

'EntropyLossWeight',0.01,...

'ActorOptimizerOptions',actorOpts,...

'CriticOptimizerOptions',criticOpts,...

'NumEpoch',3,...

'AdvantageEstimateMethod','gae',...

'GAEFactor',0.95,...

'SampleTime',Ts,...

'DiscountFactor',0.997);

agent = rlPPOAgent(actor,critic,agentOpts);

trainOpts = rlTrainingOptions(...

'MaxEpisodes',20000,...

'MaxStepsPerEpisode',600,...

'Plots','training-progress',...

'StopTrainingCriteria','AverageReward',...

'StopTrainingValue',430,...

'ScoreAveragingWindowLength',100,...

'SaveAgentCriteria',"EpisodeReward",...

'SaveAgentValue',700);

doTraining = true;

if doTraining

trainingStats = train(agent,env,trainOpts);

else

load('rocketLanderAgent.mat');

end

[Ehab]

The error is Unrecognized field name "obsInfo""

Answer 1

[Ayush]

Hey Ehab,

I understand that while using the continuous gaussian actor function “rlContinuousGaussianActor”, you are encountering an error “Unrecognized field name "obsInfo"”.

The error occurs because the "rlContinuousGaussianActor" function expects the third input to be the “actionInfo”, not “obsInfo”. In your code, you have mistakenly passed “obsInfo” as the third input instead of “actionInfo”.

To resolve this issue, you need to update the creation of the “rlContinuousGaussianActor” as follows:

actor = rlContinuousGaussianActor(actorNetwork, actionInfo, obsInfo);

For more information on "rlContinuousGaussianActor" function, refer to MathWorks documentation link below:

https://in.mathworks.com/help/reinforcement-learning/ref/rl.function.rlcontinuousgaussianactor.html

Hope above provided information helps!

Regards,

Ayush Goyal