How wide range do optimalisation algorithms cover?

I'm currently useing fminsearch, fmincon and fminunc in my code, and I do not know, how wide the range they search, before giving a minimal place for a function. I want to make a valadation process, and I don't know how hould I change the initial point for the ''fmin'' functions. For example if my initial points are [1 1 1], how dose those functions fearch. Do they try every combination with definite steps from [-5 -5 -5 ] to [7 7 7] for example, or does it varys how they search for the minimum? How can one determine the initial point to try?

Answer 1

[Thiago Henrique Gomes Lobato]

All of them except fmincon are unconstrained optimization algorithms, which means they can have any possible value in the group of real numbers. In the case of fmincon you can decide which restrictions you want to have. All of them search for a solution not by moving in fixed steps (this would be brute force) but rather using some heuristic such as the gradient information of the function (fminunc and fmincon). I would advise you to try to view some lectures or videos about optimization before trying to understand the implementation of the matlab functions, so at least you have a general idea how they work. You can check the lectures in this site, for example: http://www.cs.cmu.edu/~ggordon/10725-F12/schedule.html. Matlab has some pages with information about the algorithms it uses, althought they may be too complex to start with: fminsearch , unconstrained optimization. Some topics you can also independently search and find short tutorials on youtube, for example, are: gradient descent, Newton's method, convex/non-convex optimization, Lagrange multipliers.