# i have unstable second order system and trying to make it follow square wave using reinforcement learning but the agent doesn't converge

조회 수: 1(최근 30일)
farouk 2022년 7월 17일
편집: Sam Chak 2022년 7월 27일
i have unstable second oreder system with 2 poles at 2 ,3 and i have square wave as refrence and trying to make the system follow the ref. the observation are x1,x2 and error and the reward function is -rms(error) and the error signal is (ref-x2) as x2 is the output
the A matrix [0 -6;1 5] B=[10;1] C[0 1] D
##### 댓글 수: 6표시숨기기 이전 댓글 수: 5
Sam Chak 2022년 7월 27일
편집: Sam Chak 2022년 7월 27일
Hi @farouk, @farouk, Thinking from the mathematical perspective, what are the manipulated variables in RL that you think can be tuned (maybe with GA)?
Does the difficulty have something to do with Learning Rate of RL (which I read about in some documentation)?
What knowledge and theorem are required so that we can deterministically set the values to make the agent converge?

댓글을 달려면 로그인하십시오.

### 답변(1개)

Sam Chak 2022년 7월 27일
I don't know how many iterations your RL will take to stabilize the system and track the square wave. But you can probably at least use the generated data for performance comparison purposes.
% 2nd-order system
A = [0 -6; 1 5]; B = [10; 1]; C = [0 1]; D = ;
sys = ss(A,B,C,D);
Gs = tf(sys)
Gs = s + 10 ------------- s^2 - 5 s + 6 Continuous-time transfer function.
% Manipulated Variable (MV)
s = tf('s');
Gv = 20*(s^2 - 5*s + 6)/(s*(s + 10))
Gv = 20 s^2 - 100 s + 120 -------------------- s^2 + 10 s Continuous-time transfer function.
% Closed-loop system
Gcl = minreal(feedback(Gv*Gs, 1))
Gcl = 20 ------ s + 20 Continuous-time transfer function.
% Tracking a square wave reference signal
tau = 2;
[u, t] = gensig("square", tau, 4);
lsim(Gcl, u, t)
ylim([-1 2]), grid on % Output signal of MV
Gu = minreal(feedback(Gv, Gs))
Gu = 20 s^4 - 200 s^3 + 740 s^2 - 1200 s + 720 ----------------------------------------- s^4 + 25 s^3 + 56 s^2 - 820 s + 1200 Continuous-time transfer function.
lsim(Gu, u, t), grid on The system output can track the square wave reference input. It takes approximately 0.25 s to converge to the reference signal without overshoot.

댓글을 달려면 로그인하십시오.

R2022a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!