How to test if values are close togather using p-value (ttest)

조회 수: 2 (최근 30일)
Danial Waleed
Danial Waleed 2021년 9월 15일
답변: Jeff Miller 2021년 9월 17일
I am testing the output of a low-power cell battery and then I measured the output after a while. I did multiple runs and the output was slightly different every time. I would like to calculate the P-value to show that my output measurements are reliable and consistent. The standard is to show a P-value of P < 0.05.
My approach (MATLAB):
mu = 2.366; % Population mean
sample=[2.180213,2.178298 ,2.310851 ,2.114255 ,3.012553 ,2.69234 ,2.079787];
n = numel(sample);
[h,ptest] = ttest(sample,mu,0.05,'right')
My p-vlue is always high. I think I am doing it wrong. I want to show that the numbers are "close together" and not "far apart". How do I do this?

답변 (2개)

Adam Danz
Adam Danz 2021년 9월 15일
편집: Adam Danz 2021년 9월 15일
The ttest assumes that data come from a normal distribution and that a reasonable large sample size is used. Your sample data appear to violate both assumptions. Instead of using a parametric test, I highly urge you to use bootstrap confidence intervals that are free from these assumptions (but still requires a few samples, of course).
This demo uses bootci() from the Stats and Machine Learning toolbox to compute the 95% confidence interval of the sample based on the percentile method. It increases your sample size to 1000 by sampling with replacement to compute the 95% CI. If the expected mean (mu) is outside that interval, you can assume that the data come from a different distribution than the distrubtuion used to calculate the expected mean.
mu = 2.366; % Population mean
sample=[2.180213,2.178298 ,2.310851 ,2.114255 ,3.012553 ,2.69234 ,2.079787];
bci = bootci(1000, {@mean, sample}, 'Alpha', 0.05, 'Type', 'per')
bci = 2×1
2.1512 2.6599
isdiff = ~all(sign(mu-bci)==[1;-1]) % 0/false means mu is within the 95% CI of the sample
isdiff = logical
0
If you do not have the Stats and ML toolbox or if you'd like to see what's going on in bootci, see this answer that computes 90% CIs using bootci and by computing the bootstrapped means and CI directly without any toolboxes.
  댓글 수: 4
Adam Danz
Adam Danz 2021년 9월 15일
편집: Adam Danz 2021년 9월 15일
> Does bootci represent the result in p-value?
No, it's better than a p-value. It produces a 95% confidence interval which is similar to a p-value but without the baggage. Some well-respected scientific journals have even started banning the use of p-values without additional statistics or descriptions of error that do not rely on significance testing.
> Also, I was thinking of using ....
You're p-hacking. You're looking for a test that gives you the results you want rather than choosing a test based on the question you're asking and the test's null-hypothesis and assumptions. P-hacking is really bad (see this explanation which also explains why bootstrap CIs are much better).
Adam Danz
Adam Danz 2021년 9월 16일
@Danial Waleed, if any of the answers to your questions helped solve the problems, please consider accepting the answers to indicate that they were successfull. This will help future visitors find viable solutions quickly.
Your question:

댓글을 달려면 로그인하십시오.


Jeff Miller
Jeff Miller 2021년 9월 17일
A small p value means that the data are incompatible with some underlying assumption/model. For example, you could potentially show that the variability among your measurements is small enough to rule out the assumption that the population standard deviation of those measurements is more than (say) 0.5 units.
A better approach might be to compute a confidence interval for the population variance of your measurements, based on the observed sample that you have. With your data, this calculator gives a 95% confidence interval for the variance among your measurements as ranging from 0.1331 to 0.2316. Taking the square roots of those values, the confidence interval for the population standard deviation of measurements is 0.265 to 0.481. So, you can say with p<.05 that the standard deviation among your measurements is less than .48 or so.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by