How to create a smooth curve through data points?

Hi, I used an ADV to gather the velocity of water at a depth. I have removed the noisy data and played around with the matlab curve fitting app. However i cannot seem to create a smooth curve going through the data points as one would in excel. I checked the forums aswell and the suggestions do not seem to work. Has anyone got any advice?

Thank you in advance.

Gerard.

Answer 1

[John D'Errico]

Just coming back to this, as I did not see the data provided. (Sorry to @Gerard Feeney)

But this is a difficult thing for multiple reasons. There clearly is noise in both variables, so an errors in variables problem. These are classically difficult problems to solve. Next, the two variables are scaled dramatically differently. So any smoothing in the x variable will overwhelm y.

x = [1536.5481	1566.4674	1501.5181	1531.5818	1631.3671	1631.0238	1614.7418	1618.1639	1589.9281	1589.6050	1605.2854	1581.2317	1625.7908	1618.1263	1545.4797	1549.7091	1438.2362	1557.2871	1525.0092	1472.9464	1373.4078	1425.7688	1312.1053	1239.4050	1210.2449	1171.1703	1165.6056	1074.7808	1065.6233	-248.93562	-167.40446	-164.76059	-126.13679	-120.86833	-166.72931	-134.69136	1045.9526	1005.8028	938.80090	-38.630116	-35.228153	12.393751	23.498005	103.81524	192.35429	245.67119	304.68271	313.25061	772.24829	1182.0425	1142.4915	1301.7898	1279.6500	1254.1230	1297.6682	1224.3103	1283.0277];
y = [0.123966942148760	0.115702479338843	0.107438016528926	0.102479338842975	0.100826446280992	0.0975206611570248	0.0892561983471074	0.0876033057851240	0.0859504132231405	0.0793388429752066	0.0776859504132231	0.0743801652892562	0.0727272727272727	0.0694214876033058	0.0677685950413223	0.0661157024793388	0.0628099173553719	0.0611570247933884	0.0595041322314050	0.0578512396694215	0.0528925619834711	0.0495867768595041	0.0479338842975207	0.0462809917355372	0.0363636363636364	0.0264462809917355	0.0247933884297521	0.0231404958677686	0.0214876033057851	0.0198347107438017	0.0181818181818182	0.266115702479339	0.264462809917355	0.262809917355372	0.261157024793388	0.259504132231405	0.257851239669422	0.254545454545455	0.0115702479338843	0.00991735537190083	0.00826446280991736	0.252892561983471	0.251239669421488	0.249586776859504	0.247933884297521	0.242975206611570	0.239669421487603	0.238016528925620	0.236363636363636	0.234710743801653	0.221487603305785	0.203305785123967	0.201652892561983	0.196694214876033	0.195041322314050	0.188429752066116	0.183471074380165	0.175206611570248	0.171900826446281];

whos x y

  Name      Size            Bytes  Class     Attributes

  x         1x57              456  double              
  y         1x59              472  double              

And, of course, the two vectors are not even the same lengths. Sigh. It looks like two vectors were chosen from different data sets, or something. So for the same of having something to look at, I'll zap out two points from y at random.

y([20 30]) = [];

Next, I would normalize the two variables to have roughly the same units. We can re-scale the fit at the end.

xscale = std(x);
yscale = std(y);

xhat = x/xscale;
yhat = y/yscale;
plot(xhat,yhat,'o')

Next, a real problem here is this is not a single-valued function of x, and barely so if we look at x(y), flipping the axes. So I'm going to do a coordinate change, moving to polar coordinates around a point somewhere in the middle.

x0 = 1;y0 = 1.5;

[theta,r] = cart2pol(xhat - x0,yhat - y0);

Next, plot the result, in the form of r(theta)

plot(theta,r,'o')

p = fit(theta',r','poly6')

p = 
     Linear model Poly6:
     p(x) = p1*x^6 + p2*x^5 + p3*x^4 + p4*x^3 + p5*x^2 + 
                    p6*x + p7
     Coefficients (with 95% confidence bounds):
       p1 =     0.07471  (0.04616, 0.1033)
       p2 =     -0.2063  (-0.2936, -0.119)
       p3 =    -0.08485  (-0.171, 0.001259)
       p4 =      0.5715  (0.3766, 0.7663)
       p5 =     -0.1036  (-0.2494, 0.04229)
       p6 =     -0.5676  (-0.6616, -0.4736)
       p7 =       1.352  (1.308, 1.396)

plot(p,theta,r)

That seems reasonable enough. A higher order polynomial would be too much. Note that I carefully scaled the data so that at least moderately high order polynomials are not an issue. 6 is not too bad here. Please don't get carried away with the idea, as degree 15 polynomials will almost always be a problem.

Now we can reconstruct the curve and plot it back in the original coordinate system.

thetapred = linspace(min(theta),max(theta),500)';
rpred = p(thetapred);
[xpred,ypred] = pol2cart(thetapred,rpred);
xpred = (xpred + x0)*xscale;
ypred = (ypred + y0)*yscale;

plot(x,y,'bo',xpred,ypred,'r-')

And that seems as good as we can hope for. With slightly more effort, I might have used a smoothing spline, or a regression spline of some sort, but this is surely adequate, as long as I did not overdo the polynomial degree.

Answer 2

[John D'Errico]

We don't have your data, so I cannot give any examples. As well, I don't know your definition of a smooth curve.

Do you want a curve that smooths out what appears to be noise, yet is fariy simple? In that case, I would use polyfit. Or if you just want a plot, the basic fitting tools on the plot will suffice. A quadratic or possibly a cubic polynomial (at most) fit will suffice. Do not go higher order than that, however, as you will start to get degeneracies, and over fitting.

Do you want a curve that interpolates the points exactly in some way, yet is still smooth. For this, you might want to use an interpolating spline. However it lookes like you MAY have repeated x values, with multiple distinct y values. That will cause problems. And an interpolating spline through data like that will surely look nasty.

Or do you want some sort of smooth curve that follows the points around in the order they are stored? Again, a spline can work here.

Do you want some sort of function you can write down? Or do you just want a nice plot? Do you need to be able to evaluate the function, but not need to write it down?

Regrdless, without your data and knowledge of what you really want out the end, it is difficult to much more. Otherwise I would be forced to write a small book.

[Gerard Feeney]

Hi John, thank you for getting back to me.

I have tried using polyfit, the basic fitting tools and curve fitting app but have had no luck.

I want a nice trend curve which finds the best fit through the data points. Something like this. Just purely for aesthetics.

X

1536.5481 1566.4674 1501.5181 1531.5818 1631.3671 1631.0238 1614.7418 1618.1639 1589.9281 1589.6050 1605.2854 1581.2317 1625.7908 1618.1263 1545.4797 1549.7091 1438.2362 1557.2871 1525.0092 1472.9464 1373.4078 1425.7688 1312.1053 1239.4050 1210.2449 1171.1703 1165.6056 1074.7808 1065.6233 -248.93562 -167.40446 -164.76059 -126.13679 -120.86833 -166.72931 -134.69136 1045.9526 1005.8028 938.80090 -38.630116 -35.228153 12.393751 23.498005 103.81524 192.35429 245.67119 304.68271 313.25061 772.24829 1182.0425 1142.4915 1301.7898 1279.6500 1254.1230 1297.6682 1224.3103 1283.0277

Y

0.123966942148760 0.115702479338843 0.107438016528926 0.102479338842975 0.100826446280992 0.0975206611570248 0.0892561983471074 0.0876033057851240 0.0859504132231405 0.0793388429752066 0.0776859504132231 0.0743801652892562 0.0727272727272727 0.0694214876033058 0.0677685950413223 0.0661157024793388 0.0628099173553719 0.0611570247933884 0.0595041322314050 0.0578512396694215 0.0528925619834711 0.0495867768595041 0.0479338842975207 0.0462809917355372 0.0363636363636364 0.0264462809917355 0.0247933884297521 0.0231404958677686 0.0214876033057851 0.0198347107438017 0.0181818181818182 0.266115702479339 0.264462809917355 0.262809917355372 0.261157024793388 0.259504132231405 0.257851239669422 0.254545454545455 0.0115702479338843 0.00991735537190083 0.00826446280991736 0.252892561983471 0.251239669421488 0.249586776859504 0.247933884297521 0.242975206611570 0.239669421487603 0.238016528925620 0.236363636363636 0.234710743801653 0.221487603305785 0.203305785123967 0.201652892561983 0.196694214876033 0.195041322314050 0.188429752066116 0.183471074380165 0.175206611570248 0.171900826446281

Answer 3

[David]

Since this problem still comes up a lot and I can't find good answers out there, I wanted to chip in even though this is clearly an old post. I think if you just separately smooth the x and y axis, you will get a nice smooth line.

x_smooth = smoothdata(X,'gaussian',10);
y_smooth = smoothdata(Y,'gaussian',10);
plot(x_smooth, y_smooth, 'r');

This might not handle the large gaps well, though. If the behavior over the gaps is weird, I suspect you could create a new variable, "path" which is the cumulative sum of the total distance traveled since the first point, and use this path variable as the x variable in the smoothing function.