Remove evenly distributed spikes from curve

Hi, I got some data from the lab giving the height data of holes in a sample I put under a microscope, so I could get a profile of the sample. Since I expect regularly occurring holes in the sample i tried to find the places with the steepest slope, by maxima and minima of the gradient of the data. I found that the gradient curve has many evenly distributed spikes on top of a function that is otherwise really good looking. I tried a smoothing spline to get rid of the spikes, but it also replicates them. I'll give you my code and the graph below. Please help me to get rid of those spikes. Thanks in advance.

% reading in the data from the file
calib5 = importdata(['C:\Users\jan\OneDrive - Carl von Ossietzky Univers',...
    'ität Oldenburg\UniShit\Engineering Physics\Semester III - winter 20',...
    '20-2021\Laboratory Project I\Data from the Lab\5_100x_zStapelBericht.xlsx']);
% calculating the gradient 
c5_1_dDx = gradient(calib5.data.Profil1(:,2))./gradient(calib5.data.Profil1(:,1));
hold on
% creation of the plot above
plot(calib5.data.Profil1(:,1),sum([[diff(calib5.data.Profil1(:,2));0],...
    [0;diff(calib5.data.Profil1(:,2))]],2))     % plot of the gradient replicated using diff()
plot(calib5.data.Profil1(:,1),c5_1_dDx)         % plot of the gradient
hold off

PS: I really need to find the prominent local maxima and minima of this and not just the spikes.

[Jonas]

maybe you want to use the envelope() function. Which value do you want to see near the jittering peaks? the minimum between the peaks or do you want to include the peaks for the interpeak value?

did you try a lowpass filter?

[Jan Heinsoth]

So I basically want to have just the curve, from which the peaks are going of from. As you can see, there is basically a curve with these spikes added/subtracted.

No I did not try a lowpass filter, but that sound like something worth trying

[Jonas]

maybe a moving median is also a good alternative, looks like the peaks you want to remove are only one sample long, so you can try a moving median with a window length of 3

[Rik]

Comment posted as flag by @Jan Heinsoth:

The proposed lowpass worked as well.

Answer 1

[Star Strider]

See if the filloutliers function will do what you want.

[Jan Heinsoth]

Thank you, It did not perfectly smooth the signal, but now it is smooth enough for my purposes.

But the lowpass filter proposed by @Jonas did also work for me.

[Star Strider]

As always, my pleasure!