Normalization process in Reinforcement Learning

Hi everyone, I have a question about 'Normalization', I'm asking that how do this process operate in ==>featureInputLayer?

The documentation says us that "Normalization" part is performed automatically. Is that true? Because when the featureInputLayer is created, 'Normalization' is defined "None" as following:

----------------------------------------------------------------------------------

featureInputLayer(numObservations,'Normalization','none','Name','observation')

----------------------------------------------------------------------------------

But, How it normalize observations/actions? Can you explain it clearly?

UASS

[H. M.]

Hi

did you get any answer yet

[Jyo Deshmukh]

In fact when I use normalization with any other option, I get an error:

Error using rl.internal.model.DLNetworkModel>localValidateLayer

Set 'Normalization' parameter

of featureInputLayer to 'none'.

[Unmanned Aerial and Space Systems]

yes, i agree with you, how can we use other options? the question is how we gonna change this option with other?

Answer 1

[Ayush Aniket]

The Normalization parameter in the featureInputLayer allows you to normalize input data directly within the layer. When you are working with Reinforcement Learning Toolbox, especially when defining networks for agents (like DQN, PPO, etc.), the toolbox manages normalization separately. That’s why you see this error:

Set 'Normalization' parameter of featureInputLayer to 'none'.

This is enforced by the internal validation logic of rl.internal.model.DLNetworkModel. The RL framework expects raw input data and applies its own preprocessing pipeline outside the layer definition.

In RL workflows, normalization is typically handled in one of these ways:

• Automatically by the agent: Some agents (like PPO or DDPG) internally normalize observations using running statistics.
• Manually by the user: You can preprocess your observations before feeding them into the environment or agent.
• Using custom layers: If you need specific normalization, you can insert a custom layer after the input layer.

You can not directly use normalization in featureInputLayer in RL agents. If you try to set 'Normalization','zscore' or anything else, the RL framework will reject it. Instead, you can:

1. Normalize data manually before passing it to the agent.
2. Insert a custom normalization layer in your network architecture.
3. Use observation preprocessing functions if supported by your agent/environment.
4. Use normalize function introduced in MATLAB R2024a - https://www.mathworks.com/help/reinforcement-learning/ref/rlnormalizer.normalize.html