Marquardt adjustment parameter (trainbr)

조회 수: 8 (최근 30일)
Abul Fujail
Abul Fujail 2011년 5월 16일
편집: Luis 2015년 4월 7일
In "trainbr" of neural network toolbox, the following lines have been used
% net.trainParam.mu 0.005 Marquardt adjustment parameter % net.trainParam.mu_dec 0.1 Decrease factor for mu % net.trainParam.mu_inc 10 Increase factor for mu % net.trainParam.mu_max 1e-10 Maximum value for mu
Could you please explain the meaning of "mu" and "mu_max"? what is their significance here?
thank you

채택된 답변

Greg Heath
Greg Heath 2013년 7월 31일

추가 답변 (2개)

Giovanni
Giovanni 2013년 7월 30일
Hi Abul, the Levenberg-Marquardt algorithm uses an approximation to the Hessian matrix (using jacobian, so is not necessary to calculate the 2nd derivatives, only the 1rst, easier from computational point of view)
Backpropagation is used to calculate the Jacobian jX of performance perf with respect to the weight and bias variables X. Each variable is adjusted according to Levenberg-Marquardt,
jj = jX * jX je = jX * E dX = -(jj+I*mu) \ je where E is all errors and I is the identity matrix.
The adaptive value mu is increased by mu_inc until the change above results in a reduced performance value. The change is then made to the network and mu is decreased by mu_dec.
The parameter mem_reduc indicates how to use memory and speed to calculate the Jacobian jX. If mem_reduc is 1, then trainlm runs the fastest, but can require a lot of memory. Increasing mem_reduc to 2 cuts some of the memory required by a factor of two, but slows trainlm somewhat. Higher states continue to decrease the amount of memory needed and increase training times.
Training stops when any of these conditions occurs:
The maximum number of epochs (repetitions) is reached.
The maximum amount of time is exceeded.
Performance is minimized to the goal.
The performance gradient falls below min_grad.
mu exceeds mu_max.
Validation performance has increased more than max_fail times since the last time it decreased (when using validation).
  댓글 수: 1
Abul Fujail
Abul Fujail 2013년 10월 25일
Thank you Giovanni ... I want to know what is "mu" ?

댓글을 달려면 로그인하십시오.


Luis
Luis 2015년 4월 7일
편집: Luis 2015년 4월 7일
Hi, anybody know how many percent use "trainbr" for training and how many for test, how change these percentages ??

태그

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by