Can I add the condition on State in Reinforcement learning environment?

I want to add the condition on the State like below

When the State setting is

obsInfo=rlNumericSpec([4 1],'LowerLimit',[10;10;10;10],'UpperLimit',[100;100;200;200]);

I want to add the below condition

State(1)<State(3) and State(2)<State(4)

Can I do this ?

and additionally,

I have applied below reset function,

function [InitialObservation,LoggedSignal] = reset(this)

LoggedSignal.State=[this.a1 this.a2 this.a3 this.a4];

InitialObservation=LoggedSignal.State;

this.State = LoggedSignal.State;

end

but when the reset is performed, it is not applied to 'LowerLimit' and 'UpperLimit' on State.

How can it be maintained this condition when reset is performed?

Answer 1

[Ayush Aniket]

Hi Hyoung-Taek Lee,

As per my understanding you want to impose a custom condition on the states defined in your code. However, the “rlNumericSpec()” function does not have any attributes to apply this condition.

A way to enforce this could be by implementing it in your custom environment logic. You should implement these conditions in custom environment ‘step’ function. If a state does not satisfy these conditions, you can either clip it to the nearest valid state, or define a high penalty in the reward function, or terminate the episode. You can read more about creating custom RL environments by referring to the following link:

https://in.mathworks.com/help/reinforcement-learning/ug/create-matlab-environments-using-custom-functions.html

For ensuring that the reset function applies the limit, the initial state ‘InitialObservation’ should be within the 'LowerLimit' and 'UpperLimit'. As this is mostly heuristic you can manually enforce this by clipping the value within the limits defined as shown below:

function [InitialObservation,LoggedSignal] = reset(this)
LoggedSignal.State=[this.a1 this.a2 this.a3 this.a4]; 
% Clip the state to the valid range 
LoggedSignal.State = max([10;10;10;10], min([100;100;200;200], LoggedSignal.State)); 
InitialObservation=LoggedSignal.State; 
this.State = LoggedSignal.State; 
end

This will ensure that after the reset, the initial state is within the limits. For reading more about defining custom reset functions, you can refer to the following link:

https://in.mathworks.com/help/reinforcement-learning/ug/create-matlab-environments-using-custom-functions.html#:~:text=and%20rlFiniteSetSpec.-,Define%20Environment%20Reset%20and%20Step%20Functions,-To%20define%20a

Hope it helps.