Auto differentiation vs finite differences in optimization toolbox

Is there a situation where finite differences is faster than automatic differentiation when using the "solve" function call in the optimization toolbox?

I'm using the optimization toolbox to solve an optimization problem with a complex loss funcation and relatively few optimization variables. I'm noticing a substantial speed up when changing the value of "ObjectiveDerivative" from "auto" to "finite-differences."

Any clarification would be greatly appreciated!

Answer 1

[Alan Weiss]

Yes, finite differences can be faster than AD. Typically, this occurs in situations like yours where the function or functions are complicated , and the resulting AD expressions are even more complex.

That said, sometimes you can help the solver by setting up your problem in a way that enables solve to operate efficiently. See Create Efficient Optimization Problems and, to a lesser extent for your problem, Separate Optimization Model from Data.

Alan Weiss

MATLAB mathematical toolbox documentation