how to calculate mean of interrupted data

how can one program matlab to calculate the zero mean of a time series but only for values before a NaN value and then values after a NaN value. i am not talking about the omitnan function.

[Dyuman Joshi]

%Indices of NaN values
idx = find(isnan(data))
%indices of values before and after NaN, and removing indices overlapping
%that of NaN values, in case there are consecutive NaN values
k=setdiff([idx-1 idx+1],idx)
out = data(k);
%zero mean
out = out-mean(out)

[osasunmwen efosa]

Thank you for your suggestion. The zero mean part is still an issue. my data has a length of 2880 with a lot of nans in between. with your help i have been able to get the mean of the data in between the nans (i got 12 individual mean values which is what i needed) . the issue now is substracting the each individual mean from each of the values used in computing it and then at the end of the day, ending up with a new dataset of length 2880 where each element is the zero mean and the nan values from the previous data set remains in the same position because i need to make a plot with another dataset, so the length is important. the last step in your suggestion, out = out-mean(out) only gives me the zero mean of the 12 mean values (dataset of length 12)

[Dyuman Joshi]

I am not sure if I understand what you want to achieve.

Let's assume this to be your data -

A=1:20;
A([2 4 8 16]) = NaN

A = 1×20

1 NaN 3 NaN 5 6 7 NaN 9 10 11 12 13 14 15 NaN 17 18 19 20

What should be the output for this?

[osasunmwen efosa]

Thanks for your response.

lets assume data A is a tracking data and the NaNs are points in time when there were interruptions. what i am trying to achieve is, the zero mean of the period whe tracking was not interrupted.

The idea is, the appearance of a NaN value marks the end of a tracking period when it comes after a real value, and if a real value comes after the NaN, that signifies the start of another tracking period.

I need to get the zero mean of these indivtracking periods i.e the mean of the periods minus the individual real values that make up that period. And my end result should be a 1x20 data like A where there will be NaNs in positions corresponding to that of A

Is this clear now ?

Answer 1

[Dyuman Joshi]

A=1:20;
A([2 4 8 16]) = NaN;
disp(A)

     1   NaN     3   NaN     5     6     7   NaN     9    10    11    12    13    14    15   NaN    17    18    19    20

idx = [0 find(isnan(A)) numel(A)+1]

idx = 1×6

0 2 4 8 16 21

for k = 1:numel(idx)-1
    %Range of indices between starting point, NaNs and ending point
    arr=idx(k)+1:idx(k+1)-1;
    A(arr) = A(arr) - mean(A(arr));
end
disp(A)

  Columns 1 through 19

         0       NaN         0       NaN   -1.0000         0    1.0000       NaN   -3.0000   -2.0000   -1.0000         0    1.0000    2.0000    3.0000       NaN   -1.5000   -0.5000    0.5000

  Column 20

    1.5000

[osasunmwen efosa]

thank you

[osasunmwen efosa]

hello, it turns out that the standard deviation of the result is different from the standard deviation of the original dataset. Is there a way to preserve the standard deviation of the orignal data ?

[Dyuman Joshi]

"it turns out that the standard deviation of the result is different from the standard deviation of the original dataset."

Yes, that is expected as we are manipulating the data.

"Is there a way to preserve the standard deviation of the orignal data ?"

You can preserve the standard deviation of the original data, but that will result in a different output.

[osasunmwen efosa]

ok. please how do i preserve the standard deviation then.

Answer 2

[Antoni Garcia-Herreros]

Hello,

You could try something like this:

A=[1:10]; % For the example
A(5)=NaN; A(8)=NaN

A = 1×10

1 2 3 4 NaN 6 7 NaN 9 10

R=[1 find(isnan(A)) length(A)]; % Indices where the nans are + the beggining and end of vector
MeanVec=zeros(size(R,2)-1,1); % Initialize vector where the means will be stored

for i=1:length(R)-1 % Loop through the different sections between nans
F(i)=nanmean(A(R(i):R(i+1)));
end
F

F = 1×3

2.5000 6.5000 9.5000

[osasunmwen efosa]

thank you for the suggestion. My dataset has many consecutive NaNs and this process only gives the mean and not the zero mean. The mean it gives still has NaNs values