fmincon (how to set square of a parameter in a constraint)

I am trying to build a multivariate BEKK on GARCH(1,1), where H(t) = kk' + a'e(t-1)e(t-1)'a + b'H(t-1)b.

One of the constraints is to set a^2 + b^2 < 1 for the fmincon.

How can I set up square of a parameter for A as in Ax = b?

Answer 1

[Bruno Luong]

You can't use Ax=b formulation. Your constraint is non-linear constraint so use the non-linear constraint formulation.

Answer 2

[Fara]

Thanks.

Haven't tried it before. Will check it out.