필터 지우기
필터 지우기

Efficient Moving Quantile Function

조회 수: 30 (최근 30일)
Matthew Casiano
Matthew Casiano 2020년 1월 11일
댓글: Benjamin Stucky 2023년 11월 6일
Hi,
I am looking for thoughts on a recipe to perform a moving quantile in Matlab. Essentially I am trying to obtain the 98% value, for example, of a window that is instantaneously tracked through a large data set. I have tried looping in the quantile function, but this is extremely slow. I have had reasonable success using the sort function each time the window is moved, and then selecting the value based on the index that is 98% of the window size - see script below. But it is still several orders of magnitude slower than the built-in movmedian function. I don't see how a moving quantile should be much different then a moving median, just that the sorted value selected is not at 50%.
My thoughts are that after the initial window is sorted, an insertion sort algorithm is most appropriate since only a single value comes into the window each time the window moves 1 point. But really I am guessing at what is inside the movmedian function. I am not a programmer guy, so any advice here or examples on how to code an efficient moving quantile function is appreciated.
The example below is relatively quick with the given sampling rate, however in my example the sampling rate is 100,000 sps and takes much longer.
% Moving Quantile
fs=1000; % sampling rate (sps)
timeData=0:1/fs:40; % time (sec)
Data=rand(1,length(timeData)); % data (EU)
window=6300; % number of points used in 1 pt sliding window (pts)
dt_wnd=(window-1)/fs; % window time span (sec)
time_wnd=timeData(1)+dt_wnd/2:1/fs:timeData(end)-dt_wnd/2; % time vector, sliding window center points (sec)
Ind=floor(0.98*window); % index at 98% quantile
for j=1:length(time_wnd) % loop through all windows moving by 1 pt
SortedData=sort(Data(j:j+window-1)); % sorts data in the jth window
Val(j)=SortedData(Ind); % value at index
end
% Plotting
plot(timeData, Data)
axis([0 40 0.9 1.0])
hold on
plot(time_wnd,Val,'LineWidth',4)
hold off
title('Moving Quantile')
xlabel('time, s')
ylabel('Amplitude')
  댓글 수: 1
Benjamin Stucky
Benjamin Stucky 2023년 11월 6일
I am also interested in moving quantiles. The implementation of @Eike Petersen works great for smaller data sets. But I am dealing with very large time series. I found this C# reference implementation: https://aakinshin.net/posts/partitioning-heaps-quantile-estimator/ As I have no knowledge about C# and C++ wrappers, I am not able to create a mex file from this. If somebody could help to implement it, this would be nice.

댓글을 달려면 로그인하십시오.

채택된 답변

Eike Petersen
Eike Petersen 2020년 12월 11일
My guess is that your guess is pretty good. :-)
I just hand-implemented a version of this with a naive insertion sort algorithm, and it's roughly 50-80x slower than medfilt1. I'm not sure you can get much quicker without switching to C / a mex-file.
You can find it here; it works for N-D arrays, with NaNs, and with different padding strategies: https://de.mathworks.com/matlabcentral/fileexchange/84200-movquant

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 Shifting and Sorting Matrices에 대해 자세히 알아보기

제품


릴리스

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by