Problem 44734. Solve the 2-norm Regularization Problem
In data fitting, regularization is a technique to stabilize an ill-posed problem. Ridge Regression (aka Tikhonov regularizaton) adds the 2-norm of the coefficient vector to the least squares problem. The cost function looks like this:
J=1/2*norm(A*x-b,2)^2+norm(alpha*x,2)^2 (MATLAB equation if image doesn't display)
where x is the coefficient vector, b corresponds to observations, A comes from evaluating terms of an equation, alpha corresponds to a chosen scalar weight value, and J is the cost. The minimization of this cost function is ridge regression. When alpha is zero and the least squares problem is ill-conditioned, this problem is unstable meaning that small changes in b cause large changes in x. Choosing alpha appropriately for a given problem stabilizes the problem causing small changes in b causes a small change in x.
This Cody problem is about solving the Ridge Regression problem. Implement the solution to the ridge regression problem. This is best done by solving a dual problem. Minimizes a different cost function yields the solution that solves the Ridge Regression problem:
J = 1/2*norm([A;alpha*eye(n)]*x-[b;zeros(1,n)],2)^2 (MATLAB equation if image doesn't display)
Therefore, write a function that minimum of the dual problem for a given A, alpha, and b.
Solution Stats
Problem Comments
-
2 Comments
The image links (for the equations) are broken.
The equation view is okay for me from a private browsing session. I am not sure exactly how to fix the image part. The hosting is supposed to be open always. Therefore, I added text versions of the equations in MATLAB language. I hope this helps.
Solution Comments
Show commentsProblem Recent Solvers4
Suggested Problems
-
Find common elements in matrix rows
2571 Solvers
-
Matrix indexing with two vectors of indices
709 Solvers
-
Compute a dot product of two vectors x and y
972 Solvers
-
461 Solvers
-
Solving Quadratic Equations (Version 1)
480 Solvers
Problem Tags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!