Marquardt adjustment parameter (trainbr)

In "trainbr" of neural network toolbox, the following lines have been used

% net.trainParam.mu 0.005 Marquardt adjustment parameter % net.trainParam.mu_dec 0.1 Decrease factor for mu % net.trainParam.mu_inc 10 Increase factor for mu % net.trainParam.mu_max 1e-10 Maximum value for mu

Could you please explain the meaning of "mu" and "mu_max"? what is their significance here?

thank you

Answer 1

[Greg Heath]

http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm

Answer 2

[Giovanni]

Hi Abul, the Levenberg-Marquardt algorithm uses an approximation to the Hessian matrix (using jacobian, so is not necessary to calculate the 2nd derivatives, only the 1rst, easier from computational point of view)

Backpropagation is used to calculate the Jacobian jX of performance perf with respect to the weight and bias variables X. Each variable is adjusted according to Levenberg-Marquardt,

jj = jX * jX je = jX * E dX = -(jj+I*mu) \ je where E is all errors and I is the identity matrix.

The adaptive value mu is increased by mu_inc until the change above results in a reduced performance value. The change is then made to the network and mu is decreased by mu_dec.

The parameter mem_reduc indicates how to use memory and speed to calculate the Jacobian jX. If mem_reduc is 1, then trainlm runs the fastest, but can require a lot of memory. Increasing mem_reduc to 2 cuts some of the memory required by a factor of two, but slows trainlm somewhat. Higher states continue to decrease the amount of memory needed and increase training times.

Training stops when any of these conditions occurs:

The maximum number of epochs (repetitions) is reached.

The maximum amount of time is exceeded.

Performance is minimized to the goal.

The performance gradient falls below min_grad.

mu exceeds mu_max.

Validation performance has increased more than max_fail times since the last time it decreased (when using validation).

[Abul Fujail]

Thank you Giovanni ... I want to know what is "mu" ?

Answer 3

[Luis]

Hi, anybody know how many percent use "trainbr" for training and how many for test, how change these percentages ??