I believe the RL environment template creator has an error in the reset function but I'm not sure

when using rlCreateEnvTemplate("MyEnvironment") to create a custom template I came across this line in the reset function:

% Line 123
InitialObservation = [T0;Td0;X0;Xd0];

Initial states seem to be reversed, I believe they should be [X0;Xd0;T0;Td0]?

the same seems to apply to the example loaded with openExample('rl/MATLABCartPoleDQNExample'), although I cannot see the reset function, the example gives the same results as the template and when I tried validating these two against the environments created with openExample('rl/CreateMATLABEnvironmentUsingCustomFunctionsExample') they yield different initial states.

but I'm still kinda new to this and I'm afraid to be missing something here. Could you please clarify?

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

You are correct the order is wrong. That being said, the order of states depends on your dynamics and how you set up the state vector in the step function.

Hope that helps

[Cecilia S.]

Excellent, thank you for clarifying. So, I believe in this case the starting angle would be constant throughout the episodes and the resetting randomization would be only of the X variable

Thanks!!