Problems importing Farama Gymnasiums (previously Open AI gym) continuous environments in MATLAB to use RL toolbox
조회 수: 8 (최근 30일)
이전 댓글 표시
Alberto Tellaeche
2023년 2월 20일
답변: Emmanouil Tzorakoleftherakis
2023년 2월 21일
Dear all,
I am using RL toolbox in Matlab to solve Farama Gymnasium environments (https://gymnasium.farama.org/)
While in environments with Discrete action spaces, the procedure works well, in Environments with CONTINUOUS action spaces gives an error in the step function inside the needed wrapper class, in the second line.
The lines that throws the error are:
ActionSingle = single(Action);
result = this.open_env.step(ActionSingle);
The error I get is :
Error using rl.train.SeriesTrainer/run
There was an error executing the environment's step method.
Caused by:
Error using continuous_mountain_car>step
Python Error: TypeError: 'float' object is not subscriptable
I think this error has something to do with the integration of Python ndarrays in Matlab.
The same issue is reported here:
https://es.mathworks.com/matlabcentral/answers/568278-environment-for-reinforcement-learning-project?s_tid=srchtitle
How can I solve this problem so that I can train environments with continuous action spaces?
Thank you in advance,
Alberto
CODE FOR THE WRAPPER CLASS
classdef MountainCarContinuousEnv < rl.env.MATLABEnvironment
%MYCARTPOLEENV: custom environment in MATLAB.
%% Properties (set properties' attributes accordingly)
properties
open_env = py.gymnasium.make("MountainCarContinuous-v0");
end
%% Necessary Methods
methods
function this = MountainCarContinuousEnv()
ObservationInfo = rlNumericSpec([2 1],'LowerLimit',[-1.2 -0.07]', 'UpperLimit', [0.6 0.07]');
ObservationInfo.Name = 'MountainCarObservation';
ObservationInfo.Description = 'Position, Velocity';
ActionInfo = rlNumericSpec([1 1],'LowerLimit',-1, 'UpperLimit', 1); %CONTINUOUS ACTION SPACE !!!!!
%ActionInfo = setDataType(ActionInfo,'single');
ActionInfo.Name = 'DirectionalForce';
this = this@rl.env.MATLABEnvironment(ObservationInfo, ActionInfo);
end
function [Observation,Reward,IsDone,LoggedSignals] = step(this,Action)
ActionSingle = single(Action);
result = this.open_env.step(ActionSingle); % PROBLEM HERE !!!!!
Observation = double(result{1})';
Reward = result{2};
IsDone = result{3};
LoggedSignals = [];
end
function InitialObservation = reset(this)
result = this.open_env.reset();
InitialObservation = double(result{1})';
end
function result =render(this)
result = this.open_env.render();
end
function result =close(this)
result = this.open_env.close();
end
end
end
댓글 수: 0
채택된 답변
Emmanouil Tzorakoleftherakis
2023년 2월 21일
Hi Alberto,
In the post you are mentioning, I recommended a 3rd party tool to use OpenAI Gym with Reinforcement Learning Toolbox. For issues with this tool, I would normally recommend talking to the developer of the tool.
Nevertheless, in this case it seems to be a data conversion error. I think if you replace
result = this.open_env.step(ActionSingle);
with
result = this.open_env.step(py.numpy.array(ActionSingle,pyargs('ndmin',int32(1))));
the error will go away.
Also, as I mentioned in the other post, MathWorks is working on making available an interface from Reinforcement Learning Toolbox to OpenAI Gym, but I don't have any specific dates I can share yet.
Hope this helps
댓글 수: 0
추가 답변 (0개)
참고 항목
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!