how can i solve a non-linear ode by ode45 or similar orders in matlab?

You won't be able to pose this as a problem for an ODE solver, since any standard ODE solver will require a finite domain over which to integrate. It is a boundary value problem anyway, so you will want to use a bvp solver. Tools like bvp4c use collocation, but you don't really care how they work, at least in terms of gaining a solution. I do think it highly unlikely that you will get an analytical solution, so you will need to provide explicit values for a and b.

So I'd first transform the problem into one with a finite domain. Thus, perhaps define a transformation that will map the infinite domain of x into a finite one. My first choice would be to use a tangent transformation.

x = tan(u)

So x lives on [-inf,inf]. But u lives on [-pi/2,pi/2].

Now, rewrite the ODE in terms of

z(u) = y(x(u))

We have y'=dy/dx. But what is dz/du? Chain rule gives us that.

dz/du = dy/dx*dx/du

And since x=tan(u), we know that dx/du=sec^2(u). Therefore we have

dy/dx = dz/du * (1/sec^2(u)) = dz/du * cos^2(u)

With a little effort, y'' should be doable too. So re-write the ODE in terms of z and u, substituting for x, y, y', y'' wherever they appear.

The boundary values will be unchanged.

z(-pi/2) = c
z(pi/2) = c

So, if you can solve the transformed boundary value problem over the new domain, that will be sufficient. If you don't like that choice of transformation, I'm sure there are others that would be viable. For example, an inverse logistic form would probably work, or you could use an erfinv transformation. The tangent transformation seems a very natural one here though.

Use one of the bvp solver tools. Although I have no experience with these solvers, it looks like bvp4c is designed to solve a general two point boundary value problem using collocation, so I'd start there. At least, that is how I'd attack the problem.