Nonlinear State Space System

See the ROC curve:

https://en.wikipedia.org/wiki/Receiver_operating_characteristic

[Utkarsh Anand]

What you are expecting cannot be done in Matlab as of today(R2018a). There are other ways though to represent NonLinear systems. You can refer the building of ARX and ARMAX models.