DRL, handing two or more observations in Step function

Hi, I am having a problem in RL environment. I have two types of input (observations). I efined it as:

obsInfo = [rlNumericSpec([1,total_obs])     rlNumericSpec([1,total_obs])]    ;

Now, I want to set them to some values in the step function. I do not know how to do that. I have one variable State as env. property. Does anyone knows how to manage it? Thanks

Answer 1

[Jack]

When you define multiple observations using an array of rlNumericSpec, your environment’s step function should return a cell array containing each observation. For example, if you have defined:

obsInfo = [rlNumericSpec([1, total_obs]) rlNumericSpec([1, total_obs])];

Then in your step function you need to split your state (or otherwise create two observation vectors) and return them as a cell array. For instance, if your environment property State is a vector that combines both observation parts, you could do something like:

function [Observation, Reward, IsDone, LoggedSignals] = stepFunction(this, Action)
    % Update the state based on Action, etc.
    % Assume this.State is a 1-by-(2*total_obs) vector:
    obs1 = this.State(1:total_obs);
    obs2 = this.State(total_obs+1:end);

    % Return observations as a cell array matching obsInfo:
    Observation = {obs1, obs2};

    % Compute reward, check if done, and assign logged signals
    Reward = ...;
    IsDone = ...;
    LoggedSignals = ...;
end

This way, MATLAB’s RL framework will correctly map each cell to its corresponding rlNumericSpec from obsInfo.

Follow me so you can message me anytime with future questions. If this helps, please accept the answer and upvote it as well.