How to correct p-value in statistical analysis?

Assume that I want to compare brain states A (e.g., wake) and B (e.g. sleep). To do that, I first allocated 10,000 brain sites (locations on the scalp) of my interest. Then at each state, at each brain site, I measured 5 data. The 10,000 brain sites are the same locations for both experiments A and B. So in a summary:

A has

A1 = [a1_1 a1_2 a1_3 a1_4 a1_5]; % 5 data recorded at brain site #1
A2 = [a2_1 a2_2 a2_3 a2_4 a2_5]; % 5 data recorded at brain site #2
A3 = [a3_1 a3_2 a3_3 a3_4 a3_5]; 
...
A10000 = [a10000_1 a10000_2 a10000_3 a10000_4 a10000_5];

B has

B1 = [b1_1 b1_2 b1_3 b1_4 b1_5]; % 5 data recorded at brain site #1
B2 = [b2_1 b2_2 b2_3 b2_4 b2_5]; % 5 data recorded at brain site #2
B3 = [b3_1 b3_2 b3_3 b3_4 b3_5]; 
...
B10000 = [b10000_1 b10000_2 b10000_3 b10000_4 b10000_5];

in which ai_j and bi_j are numbers ranging from 0 to 1.

Then I compared statistical difference (p-value) between A and B over all brain sites:

p1 = A1 vs B1;
p2 = A2 vs B2;
p3 = A3 vs B3;
...
p1000 = A10000 vs B10000;

I'm wondering

• if I should correct these p values via multiple comparison test? If so, how should I do it in Matlab? Any method better than bonferroni correction? because bonferroni correction is too conservative.
• how I can tell if brain state A is significantly different with B?

[Ben11]

what test do you use to compute the statistical difference?

[Kyle]

I used Wilcoxon rank sum test to calculate the p-value:

p = ranksum(x,y)

since this method doesn't need a specific data distribution (I just have 5 data points for each x and y).

Answer 1

[Ben11]

With the Statistics Toolbox you want to use mutcompare for your statistical tests. There are quite a few options for corrections. Other than Bonferroni, you might want to use the "dunn-sidak" test, which is less conservative than Bonferroni.

Are you comparing an experimental group with a control group for example or 2 experimental groups?

[Kyle]

Hi Ben11,

Thank you for the comments. Could you please specify how to get the corrected p value from mutcompare() function? Seems like it doesn't compute p-value? I'm comparing two experimental groups. The experiment background has been added in my post.

Cheers Kyle

[Ben11]

Hi Kyle,

I've been digging a bit and it seems that we can't get the corrected p-value using multcompare in Matlab, at least in a somewhat easy/straightforward way.

Can you use another software like Prism by Graphpad? It's quite user friendly and versatile. There's a 30 day trial you can get here .

Sorry if I could not be of more help.