- Plot the rewards over time during the retraining process to observe the trend. This can help you understand if the rewards are converging or not.
- Experiment with different learning rates for the agent. A higher learning rate may lead to faster convergence but could also result in more variation initially.
- You can also try modifying the reward function to see if it reduces the variation in rewards.
freezing layers of actor and critic of RL agent
조회 수: 2 (최근 30일)
이전 댓글 표시
After training ,I have freezed every layer of my actor and crtitc network of my RL agent (by using setLearnRateFactor(neuralnet,'layers','parameters',0);) and then I am retraining my agent in same enviornment and I am getting rewards like as shown in image file.
My ques is is it normal to get rewards like this? (I mean shouldnt there should be no variation or very little variation in rewards.)
my reward function is 10 - e^2 (error).
댓글 수: 0
답변 (1개)
Karanjot
2024년 1월 30일
편집: Karanjot
2024년 1월 30일
Observing fluctuations in rewards is a common occurrence when retraining a reinforcement learning (RL) agent, despite having locked the parameters of both the actor and critic architectures. The agent continues its exploration and learning within the given environment, and the specified reward function significantly influences the reward outcomes.
The variation in rewards can be influenced by several factors, such as the exploration-exploitation trade-off, the complexity of the environment, and the learning rate of the agent. It is possible that the agent is still trying to optimize its policy and may encounter different states or actions that result in varying rewards.
The environment’s inherent stochasticity can lead to different state transitions and rewards for similar actions. Additionally, if there are other unfrozen parameters or noise processes involved in action selection, they could contribute to the observed variations
You may consider the following steps:
Keep in mind that RL training is inherently iterative, and achieving an optimal policy often requires multiple iterations. While some degree of reward variation is to be anticipated, it may indicate a need for further investigation or adjustments in your training setup.
댓글 수: 0
참고 항목
카테고리
Help Center 및 File Exchange에서 Environments에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!