i have unstable second order system and trying to make it follow square wave using reinforcement learning but the agent doesn't converge
조회 수: 1 (최근 30일)
이전 댓글 표시
i have unstable second oreder system with 2 poles at 2 ,3 and i have square wave as refrence and trying to make the system follow the ref. the observation are x1,x2 and error and the reward function is -rms(error) and the error signal is (ref-x2) as x2 is the output
the A matrix [0 -6;1 5] B=[10;1] C[0 1] D[0]
댓글 수: 6
Sam Chak
2022년 7월 27일
편집: Sam Chak
2022년 7월 27일
Hi @farouk, @farouk, Thinking from the mathematical perspective, what are the manipulated variables in RL that you think can be tuned (maybe with GA)?
Does the difficulty have something to do with Learning Rate of RL (which I read about in some documentation)?
What knowledge and theorem are required so that we can deterministically set the values to make the agent converge?
답변 (1개)
Sam Chak
2022년 7월 27일
I don't know how many iterations your RL will take to stabilize the system and track the square wave. But you can probably at least use the generated data for performance comparison purposes.
% 2nd-order system
A = [0 -6; 1 5]; B = [10; 1]; C = [0 1]; D = [0];
sys = ss(A,B,C,D);
Gs = tf(sys)
% Manipulated Variable (MV)
s = tf('s');
Gv = 20*(s^2 - 5*s + 6)/(s*(s + 10))
% Closed-loop system
Gcl = minreal(feedback(Gv*Gs, 1))
% Tracking a square wave reference signal
tau = 2;
[u, t] = gensig("square", tau, 4);
lsim(Gcl, u, t)
ylim([-1 2]), grid on
% Output signal of MV
Gu = minreal(feedback(Gv, Gs))
lsim(Gu, u, t), grid on
The system output can track the square wave reference input. It takes approximately 0.25 s to converge to the reference signal without overshoot.
댓글 수: 0
참고 항목
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!