# Noise parameters in Reinforcement learning DDPG

조회 수: 44 (최근 30일)
Surya teja Tunuguntla 2019년 6월 14일
댓글: Atikah Surriani 2023년 5월 8일
What should be the values of Noise parameters (for agent) if my action range is between -0.5 to -5 in DDPG reinforcement learning I want to explore whole action range for each sample time? Also is there anyway to make the noise options (for agent) independent of sample time?

댓글을 달려면 로그인하십시오.

### 채택된 답변

Drew Davis 2019년 6월 19일
편집: Drew Davis 2019년 6월 19일
Hi Surya
It is fairly common to have Variance*sqrt(SampleTime) somewhere between 1 and 10% of your action range for Ornstein Uhlenbeck (OU) action noise. So in your case, the variance can be set between 4.5*0.01/sqrt(SampleTime) and 4.5*0.10/sqrt(SampleTime). The other important factor is the VarianceDecayRate, which will dictate how fast the variance will decay. You can calculate how many samples it will take for your variance to be halved by this simple formula:
halflife = log(0.5)/log(1-VarianceDecayRate)
It is critically important for your agent to explore while learning so keeping the VarianceDecayRate small (or even zero) is a good idea. The other noise parameters can usually be left as default.
You can check out this pendulum example which does a pretty good job of exploring during training.
The sample time of the noise options will be inherited by the agent, so it is not necessary to configure. By default, the noise model will be queried at the same rate as the agent.
Hope this helps
Drew
##### 댓글 수: 5이전 댓글 3개 표시이전 댓글 3개 숨기기
Drew Davis 2019년 12월 9일
You can derive this formula pretty easily:
decayfactor = 0.5 = (1 - decayrate)^(#steps)
Thank you Drew

댓글을 달려면 로그인하십시오.

### 추가 답변 (1개)

Atikah Surriani 2023년 4월 30일
can i change noise model of ddpg using matlab? for example, the original ddpg using OU noise, while my study tends to change it using gaussian?
##### 댓글 수: 3이전 댓글 1개 표시이전 댓글 1개 숨기기
Atikah Surriani 2023년 5월 8일
thank you for the answer, so we can change the noise option on DDPG using matlab?
for example:
rl.option.OrnsteinUhlenbeckActionNoise
we change as " rl.option.gaussianActionNoise or rl.option.anythingActionNoise "
or else
thankyou
Atikah Surriani 2023년 5월 8일
or do any modification to the noise?

댓글을 달려면 로그인하십시오.

### 카테고리

Help CenterFile Exchange에서 Reinforcement Learning에 대해 자세히 알아보기

R2019a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by