Hidden Size in autoencoder

Hi everyone,

I am looking for a formula to determine the maximum allowable hidden size in an Autoencoder. I came across the following heuristic:

hiddenSize=round(n×numFeatures)

However, I haven't been able to find a specific reference to cite for this formula.

I would appreciate any guidance or references that could help me adjust the hidden size in an Autoencoder effectively.

Thanks in advance for your assistance.

Answer 1

[Malay Agarwal]

Hi @Jahetbe,

There is no rule of thumb for selecting the hidden size in an autoencoder. The best way to determine the hidden size of an autoencoder is via hyperparameter tuning. You can set up a grid search with a cross-validation loop to find the hidden size that yields the best results on your dataset. You can also use Bayesian optimization (https://en.wikipedia.org/wiki/Bayesian_optimization) to find the best value for the hidden size.

Please refer to the following link for an example on tuning hyperparameters using Bayesian optimization: https://www.mathworks.com/help/deeplearning/ug/deep-learning-using-bayesian-optimization.html.

Please note that you'll have to adapt the example according to your requirements. You can refer to the following link to learn more about the Bayesian optimization workflow and how you can adapt it to your requirements: https://www.mathworks.com/help/stats/bayesian-optimization-workflow.html.

Hope this helps!