Using environment reset function: Access Simulink workspace for if then tests

Question

0 개 추천

I'm attempting to code a PPO RL agent in which a glider is released and attempts to navigate to a target within "R997" distance. I've generated the model within Simulink including outputing the last distance value to the target ("D2Tgt") with a workspace block. For navigating to the target within a R997 value, it gets a reward. It gets other reward and penalties along the way as well. Some publications have mentioned such training cases can be speed up by holding the target location constant instead of randomizing it each training episode until the it starts getting close where you start randomizing it. This is done by starting with a large acceptable distance and decreasing it over the training time. This is what I've attempted to code below. The issue I'm having is pulling the "ans.D2Tgt" from the workspace during RL training even though I have a simulink workspace out block for this value. Any advice???

ans.D2Tgt=random initalization for now
env = rlSimulinkEnv('SimulinkModel','SimulinkModel/RL Agent',obsInfo,actInfo);
env.ResetFcn = @(in) localResetFcn(in, R997, ans);
function in = localResetFcn(in, R997, ans)
    persistent n
    if isempty(n)
        n=0
    end
    ans.D2Tgt=ans.D2Tgt
    DropHeight = randi([deleted values were here],1,1);
    blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
    in = setBlockParameter(in,blk,'xme_0',mat2str([0 0 -DropHeight]));
    ReleaseVelocity=randi([deleted values were here],1,1);
    blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
    in = setBlockParameter(in,blk,'Vm_0',mat2str([ReleaseVelocity 0 0]));
    if (ans.D2Tgt <= R997 * 5) && (n==0)
        n=n+1
        Target_X = randi([deleted values were here],1,1)
        Target_Y = randi([deleted values were here],1,1)
        blk = 'SimulinkModel/Constant5';
        in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0])); 
        blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
        in = setBlockParameter(in,blk,'const',mat2str(R997 * 4));
    elseif (ans.D2Tgt <= R997 * 4) && (n==1)
        n=n+1
        Target_X = randi([deleted values were here],1,1)
        Target_Y = randi([deleted values were here],1,1)
        blk = 'SimulinkModel/Constant5';
        in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
        blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
        in = setBlockParameter(in,blk,'const',mat2str(R997 * 3));
    elseif (ans.D2Tgt <= R997 * 3) && n==2
        n=n+1
        Target_X = randi([deleted values were here],1,1)
        Target_Y = randi([deleted values were here],1,1)
        blk = 'SimulinkModel/Constant5';
        in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
        blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
        in = setBlockParameter(in,blk,'const',mat2str(R997 * 2));
    elseif (ans.D2Tgt <= R997 * 2) && n==3
        n=n+1
        Target_X = randi([deleted values were here],1,1)
        Target_Y = randi([deleted values were here],1,1)
        blk = 'SimulinkModel/Constant5';
        in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
        blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
        in = setBlockParameter(in,blk,'const',mat2str(R997 * 1));
    elseif (ans.D2Tgt <= R997 * 1) && (n>=4)
        n=n+1
        Target_X = randi([deleted values were here],1,1)
        Target_Y = randi([deleted values were here],1,1)
        blk = 'SimulinkModel/Constant5';
        in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
    else
    end
end