How to set block parameters in a Matlab RL reset function

I am using a reset function as a part of the rlSimulinkEnv and calling it in the main script. In the reset function, when I try to reset a constant block, this works as expected. However, when I try to reset variable found in a simulink block like a revolute joint, the value will be reset at the beginning of the first episode, but then never again.

agentBlk = [mdl '/RL Agent'];
observationInfo = rlNumericSpec([23 1]);
actionInfo = rlNumericSpec([6 1]);
observationInfo.Name = 'observation';
actionInfo.Name = 'action';
env = rlSimulinkEnv(mdl,agentBlk, observationInfo, actionInfo);
env.ResetFcn = @(in)handResetFcn(in,sep,yL_end,l);

Here is the reset function. In this case, the 'goal_x' and 'goal_y' variables correspond to constant blocks and reset every episode. The joint angles will not reset every time. The will be set to the first episode's random value for every episode.

function in = handResetFcn(in,sep,yL_end)
    x = sep+.1*(rand-.5);
    y = yL_end+.05*rand;
    in = in.setVariable('goal_x',x);
    in = in.setVariable('goal_y',y);
    tl1_0 = pi/2 + .1*(rand-.5);
    tl2_0 = pi/2 + .1*(rand-.5);
    in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint','PositionTargetValue',num2str(tl1_0));
    in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint1','PositionTargetValue',num2str(tl2_0));
end

[ajshank]

I can confirm this is the case for Matlab R2021b. Other tricks such as assigning a workspace variable myVar for the block/mask parameter, and calling assignin("base", "myVar", val) in the reset function don't work either.

Answer 1

[Steve Miller]

Hi Carson,

I suspect the reason this did not work is because you have not configured the target position to be a run-time parameter. If you do not configure this to be a run-time parameter, your sweep may not recognize that the value has changed from run to run. You can see in example Quadruped Robot Locomotion Using DDPG Agent that the joint targets are configured as run-time parameters.

https://www.mathworks.com/help/reinforcement-learning/ug/quadruped-robot-locomotion-using-ddpg-gent.html

Below is a screenshot showing the setting you need to change on the blocks.

--Steve