sgdmupdate() is a single step or multiple steps until convergence?

Is calling sgdmupdate() once training a net over a single step or multiple steps until convergence? If the latter, how is convergence judged?

[robinho robinho]

?

Answer 1

[Akash]

Hi Robinho,

I understand that you are seeking clarification on whether calling "sgdmupdate" function during training involves a single step or multiple steps until convergence. Additionally, you would like to know how convergence is determined, specifically the criteria used for judging convergence.

The "sgdmupdate" function utilizes multiple steps or epochs to achieve convergence during training. Each step represents a single update of the parameters using "Stochastic gradient descent with momentum (SGDM)".

To determine convergence, you can specify the total number of training epochs using the "numEpochs" parameter. This parameter indicates the number of times the model goes through the entire training dataset. Convergence is typically judged based on the desired level of accuracy or a predefined stopping criterion.

Additionally, the convergence can also be determined by monitoring the "Stop" property of the "TrainingProgressMonitor" object. If the "Stop" property is set to true, it indicates that the training process should be stopped. This property can be triggered, for example, by clicking the "Stop" button during training.

To gain more information about "sgdmupdate" function, including its usage and convergence criteria, I recommend referring to the documentation provided at the below provided link.

Update parameters using stochastic gradient descent with momentum (SGDM) - MATLAB sgdmupdate - MathWorks India

Thanks,

Akash.

[feynman feynman]

thanks so much. So sgdmupdate runs multiple steps instead of 1?

[Ben]

Each call to sgdmupdate performs one step of the update algorithm specified at the bottom of the documentation page: https://uk.mathworks.com/help/deeplearning/ref/sgdmupdate.html

Typically you use sgdmupdate in custom training loops, which will typically end up calling sgdmupdate multiple times as the loop progresses, and control of convergence is up to the user to manage in the custom loop.