Statistical Significance using ttest2

I'm curious how to determine whether two datasets are statistically similar from a two-sample t-test (ttest2). I understand that the result gives the decision to reject (1) or accept (0) the null hypothesis, but how does the null hypothesis decision give statistical significance?

Here's an example: If I have two thermometers (A and B) that record temperature at the same location (but at different sampling frequencies) for one year, I would get two time series that should look similar but have a different number of data points. Using h = ttest2(A,B,'Vartype','unequal') h would be either 0 or 1.

Given the null hypothesis of: "the data in vectors A and B comes from independent random samples from normal distributions with equal means and equal but unknown variances"(from https://www.mathworks.com/help/stats/ttest2.html ), wouldn't rejection of this hypothesis mean that the datasets are NOT correlated with statistical significance? If so, does 0 represent high correlation in this case?

Thanks for your help.

Answer 1

[Star Strider]

The paired t-test says nothing about correlation. (The 'unequal' flag is about the variances.)

Consider:

t = linspace(0, 10*pi);
v1 = sin(t);
v2 = 2*rand(size(t))-1;
[h,p] = ttest2(v1, v2)

h =
     0

p =
     0.6819

It does indicate if the means are different (rejecting the null hypothesis) at a specific significance.

EDIT — Added example.

[Steve F]

Thanks so much - this makes sense. From my example, it seemed strange that h=0 would indicate significance and equal means instead of h=1. But I supposed everyone uses ttest2 for different purposes.

[Star Strider]

As always, my pleasure.

[Jonathan Mitchell]

Star Strider: With your example would the result suggest that the means are equal, but not at a level of statistical significance? Since the p value is so high? Or is a high p-value acceptable because the null hypothesis is not rejected?

[Star Strider]

The p-value is the probability that the means are not different. (It does not say that they are the same or that they are equal, although in my example, they are close.)

The lower the p-value, the the less likely it is that the means are not different. The generally accepted probability that rejects the null hypothesis (that the means are not different) and accepts the alternate hypothesis (that they are different) is 0.05. There is sctually nothing magic about that value.

[Jonathan Mitchell]

Isn't rejecting that the means are not different the same as (or similar to) accepting that they are different? How is h=0 different than h=1?

Also, thank you =)

[Star Strider]

Well, yes. I’m using the usual terminology.

From the documentation:

• The result h is 1 if the test rejects the null hypothesis at the 5% significance level, and 0 otherwise.

As always, my pleasure! (Voting for my Answer would be nice!)