Interpolating data from a sensor to get smooth graph

Hi, My data from a sensor had too much noises (or noise like data). It was fluctuating too much and I wanted it to be smoothen while keeping data as original as possible. I would like to interpolate the data so that I could get a smooth graph out of it I've tried few methods (interp1, spline, pchip) but all of these methods gave me an error . Could anyone help me..?

here's my code:

%% get data from excel file

[~, ~, raw] = xlsread('C:\Users\SK\Desktop\n.xlsx','sheet1');

raw(cellfun(@(x) ~isempty(x) && isnumeric(x) && isnan(x),raw)) = {''};

cellVectors = raw(:,1);

raw = raw(:,[2,3]);

R = cellfun(@(x) ~isnumeric(x) && ~islogical(x),raw);

raw® = {NaN};

data = reshape([raw{:}],size(raw));

Date = cellVectors(:,1);

wristTorque = data(:,1);

elbowTorque = data(:,2);

clearvars data raw cellVectors R;

data = xlsread('C:\Users\SK\Desktop\m.xlsx','Sheet1');

time = data(:,1);

elbowAngle = data(:,2);

wristAngle = data(:,3);

clearvars data raw;

%% get a interpolated plot

xq = [];

for i=1:length(time)-1

    xq = [xq; time(i)+(time(i+1)-time(i))/4; time(i)+(time(i+1)-time(i))*2/4; time(i)+(time(i+1)-time(i))*3/4; time(i+1)];

end

figure

%vq2 = interp1(time,elbowTorque,xq,'spline');

%plot(time,elbowTorque,xq,vq2,'-');

p = pchip(time,elbowTorque,xq);

s = spline(time,elbowTorque,xq);

plot(time,elbowTorque,'o',xq,p,'o',xq,s,'-');

[Jan]

Please learn how to format the code properly. A good readability improves the chance, that readers spend the time to read them.

If you mention, that you get an error, post the error message, such that we do not have to guess it. Thanks.

An interpoltaion does smooth the data, but the purpose is to interpolate the data. For smoothing the command filter is more likely what you need, from a physical point of view.

The clearvars command are not useful. Cleaning up is nice, but if the corresponding variables do not exhaust you RAM completely, this is not efficient in Matlab.

Answer 1

[Star Strider]

You have to do some interpolation, but that is only to provide uniform sampling times so the subsequent filter will perform correctly. I have only processed your ‘m.xlsx’ data, but you can adapt it to your other data.

This routine initially interpolates (resamples) the data in columns 2 and 3, does an FFT analysis, then designs a Cheybshev Type 2 filter (no passband ripple). It plots the original data, the fft, and the filtered data. You will probably need to experiment with the passband (the numerator in the ‘Fpb’ calculation) to get the result you want, since I don’t know what your signal should look like. (Notice that it ‘rails’ at +45° and -60°.) I leave it to you to deal with those details.

The code:

Dm = xlsread('Shaun Koh m.xlsx');
Dms = sortrows(Dm, 1);                              % Sort Data By Time
dDms = diff([0; Dms(:,1)]);
dDms0 = find(dDms == 0);
Dms(dDms0,1) = Dms(dDms0-1,1)+1E-8;                 % Eliminate Duplicate Times
Tsi = linspace(min(Dms(:,1)), max(Dms(:,1)), 1000*(max(Dms(:,1))-min(Dms(:,1))));   % Ts = 0.001 s
Dmi = interp1(Dms(:,1), Dms(:,2:3), Tsi, 'linear'); % Interpolate (Resample) Data

figure(1)
plot(Tsi, Dmi)
grid
title('Time Domain Plot')

Ts = mean(diff(Tsi));                               % Fourier Analysis: Find Passaband Frequencies
Fs = 1/Ts;
Fn = Fs/2;
FTm = fft(Dmi)/size(Dmi,1);
FTm(1,:) = 0;                                       % Set d-c Component To Zero
Fv = linspace(0, 1, fix(size(FTm,1)/2)+1)*Fn;
Iv = 1:length(Fv);

figure(2)
plot(Fv, abs(FTm(Iv,:)))
grid
axis([0  25    ylim])
title('Frequency Domain Plot')

Fpb = 1.8/Fn;                                       % Passband
Fsb = Fpb .* 1/0.5;                                 % Stopband
Rp  =  5;                                           % Passband Ripple
Rs  = 50;                                           % Stopband Ripple
[n,Wn] = cheb1ord(Fpb, Fsb, Rp, Rs);                % Design: Chebyshev Type 2 Filter
[b,a]  = cheby2(n, Rs, Wn, 'low');
[sos,g] = tf2sos(b,a);

filtDmi = filtfilt(sos, g, Dmi);                    % Filter Interpolated Signal

figure(3)
plot(Tsi, filtDmi)
grid
title('Time Domain Plot — Filtered Data')

Answer 2

[John D'Errico]

I think Jan understands interpolation, but he has not explained it well.

An interpolation tool does NOT smooth the data.

An interpolation tool leaves every bit of noise in the data that was there. It will exactly reproduce the data (to within floating point trash.) In interpolation tool, when applied to noisy data can in fact product something that appears even more noisy, exaggerating the swings while still passing exactly through the data points.

An interpolator is capable of producing a "smooth" function BETWEEN the data points (depending on which method is chosen.) So if your data is already smooth but spaded too far apart, then an interpolation tool is nice, in that it can produce a nice smooth looking curve that passes through the points.

IF you need to smooth out data that is not smooth, killing the noise while retaining the essential signal present, then you need to use a tool that can do smoothing, NOT an interpolation tool. There are many such ways to do so, and without writing a book on the subject, I'll stop soon.

You can use tools to do spline smoothing. You can find them in the form of spaps, which now resides in the curve fitting toolbox. Or you can use my SLM tools, which do require the optimization toolbox. Or you can use an exponential moving average of some ilk (if your data is a time series, so equally spaced in time). Filter can help you in the last case.