Fit curve to nonstandard data

I'm trying to fit a curve to 2 data sets, image attached. I want to end up with a vector (or function) which defines, for any given x value, what the most likely y value is. It should have the property y(x.1)<= y(x.2) i.e. for an increase in x you get an increase in y.

The data won't fit any traditional function - it's stepped and weirdly curved.

I started by defining a linspace for x, and then trying the median of values in a range around that x value:

XSpace = (0:3500)';
YSpace = 0*XSpace;
 for index = 1:size(X)
     YSpace(index,1) = median(Y(and(X>(XSpace(index)-100),X<(XSpace(index)+100))));
 end

However, this is skewed as there aren't the same number of data points for any given X value (e.g. if there are more data points below the X value of interest than above, the median will be skewed low).

I've tried sorting the scatter data, and then applying a range of filters, but I struggle to maintain the shape. I thought a median filter would work, but it doesn't handle when the outliers become too frequent. I'm also struggling as there's lots of data for low x, and much less data for high x.

Has anyone got any bright ideas about how to fit a curve/space to this data? It seems very obvious looking at the data, how the curve should look, but I'm struggling to work out how to fit it with an algorithm.

Answer 1

[Stephan]

Hi,

if you have the curve fitting toolbox, you could consider this for your purpose:

https://de.mathworks.com/help/curvefit/interpolation-methods.html

You also find an example which could meet your requirements here at about the half of the page:

Best regards

Stephan

Answer 2

[Star Strider]

I am not certain what you want to do. The problem is that you have several sets of data in each vector, so taking the unique ‘x’ values and taking the mean of the corresponding ‘y’ values may be the only possible approach. You can then use interp1 to interpolate to get the ‘y’ corresponding to any ‘x’ within the limits of ‘x’ and ‘y’ for each data set.

The Code —

D = load('DataForForumQuestion.mat');

XData1 = D.XData1;
YData1 = D.YData1;
XData2 = D.XData2;
YData2 = D.YData2;

[UX1,~,ic] = unique(XData1);                            % Unique ‘x’ Values & Indices For Set #1
UY1 = accumarray(ic,YData1, [], @mean);                 % Use ‘bsxfun’ To Calculate ‘mean’ Of ‘y’ Values Corresponding To Unique ‘x’ Values
[UX2,~,ic] = unique(XData2);                            % Unique ‘x’ Values & Indices For Set #2
UY2 = accumarray(ic,YData2, [], @mean);                 % Use ‘bsxfun’ To Calculate ‘mean’ Of ‘y’ Values Corresponding To Unique ‘x’ Values

figure(1)
plot(XData1, YData1, '-b')                              % Display Raw Data
hold on
plot(XData2, YData2, '-r')
hold off
axis([xlim    0 350])

figure(2)
plot(UX1, UY1, '-b')                                    % Display Processed Data
hold on
plot(UX2, UY2, '-r')
hold off
axis([xlim    0 350])

XD1i = 1502;                                            % Choose ‘x’ To Interpolate
YD1i = interp1(UX1, UY1, XD1i);                         % Calculate Corresponding ‘y’

XD2i = 1502;                                            % Choose ‘x’ To Interpolate
YD2i = interp1(UX2, UY2, XD2i);                         % Calculate Corresponding ‘y’

The Plot —

XD1i =
        1502
YD1i =
  157.2170

XD2i =
        1502
YD2i =
   68.2821