I'm trying to compute the derivative of a time series. I attached its data to this question. I looked for same question in ‘Ask & Answer’ of Matlab central. First of all, I would summarize most of them here:
1. Forward or backward derivation may not be good enough. They could remain smooth, if f(Xi) on both side of Xi is equal.
2. Derivative over large interval (Delta-X) using Matlab-Function diff (), central derivation. This function has higher order of accuracy, but still dominated easily by noise.
3. Fit a function (model for curve which should be fitted) to the data and then derivate the function. The good choices of models are:
• Interpolating spline --> The Matlab command spline works in Matlab workspace as a cubic spline interpolation, which make a polynomial of 3. Order between two contiguous points. It involves a simply solved linear system of equation and therefore avoids oscillation. However, it can’t do the differentiation around a singularity point. Lagrange Interpolating Polynomial can do it poorly specially with higher order polynomials. But, the more data used in interpolation, the higher the degree of resulting polynomial, which leads to greater oscillation between data points. Specially when the interval (Delta-X) between two contiguous point is the same over the whole of data points. Therefore, use 3. Or 5. order of LIP. For noisily data lower order approximation will be the best.
• PCHIP --> If we want to avoid the high oscillation between each data point and the problem of differentiation around the singularity point, use PCHIP (as a spline which represent a curve with singularities). It is a far better approximation than simple cubic spline. However, both methods are not sensitive to noise. I know that differentiation amplifies any noise in my data. There are two Suggestions to avoiding noise:
* if worry about noise, use smoothing spline and then differentiate the spline model. If your data has noise, you need the kind of smoothing.
*
* Before using Spline-Toolbox, pre-filter (signal processing toolbox) the data. Use some filter design like Cremez, low pass filter (is known as moving average filtering in Matlab). Filtering of data the problem more complex.
4. Use mean over a period instead of original data (I didn’t still understand, what does it mean!!)
5. Savitsky-Golay are useful to build derivative estimates of time series. They are fast and efficient for long series of data, as they can be computed using the function filter.
6. Least-square approximation together with Sobolev-norm. Least-square fitting has the advantage of smoothing out or filtering the errors. Information regarding to the data:
• There isn’t any singularity in the data. • Data may have noise • Long series of data (size of vector= (9817*1)
However, I am still confused to use above mentioned methods! Should I use only smoothing spline through Curve fitting tool? Which command should be used to derivate the fitted model? If I use the command f=spline (x,y) (cubic spline interpolation), I could use the command ‘fnder’ to derivate f easily, but it isn’t sensitive to the noise. I don’t know at all, how to use filters such as Savitsky-Golay, Cremez. I’m sorry for the long History! I would be pleased, if anyone would help me!