How to perform Bonferroni correction on ttest2 comparison of two independent samples?

Hi all!

My question may be simple but I don't get to perform Bonferroni correction on ttest2 comparison of two random independent samples with equal column number but different row number.

I want to perform ttest2 comparison between each column of my two samples, therefore getting 10 p-values (see example below). But i get the following error "Index in position 1 is invalid. Array indices must be positive integers or logical values".

Once I overcome this error, I would like to perform Bonferroni correction on ttest2 comparison.

So many thanks!!! I'm using Matlab R2022b

% Note that r1 and r2 differ in row number, not column number
r1 = randi(125,200,10); % Random sample 1 
r2 = randi(155,100,10); % Random sample 2

% Perform ttest2 comparison, getting 10-pvalues by comparing the 10 columns of r1 and r2
[h,p,ci,stats] = ttest2(r1(i,:),r2(i,:)); % ttest2 comparison of two random, independent samples
results = multcompare(stats,"CriticalValueType","bonferroni") % Bonferroni correction on ttest2

Answer 1

[Shashi Kiran]

Hi Sara,

I understand you are encountering errors while applying the Bonferroni correction to a ttest2 comparison. Here are the issues I have addressed:

• The code attempted to use r1(i,:) and r2(i,:) for ttest2 without defining i. This issue can be resolved by initializing variables to store the results and iterating over each column. Here is how you can set it up:

% Initialize variables to store results
h = zeros(1, 10);
p = zeros(1, 10);
ci = zeros(10, 2);
stats = struct('tstat', {}, 'df', {}, 'sd', {});

% Perform ttest2 comparison for each column
for i = 1:10
    [h(i), p(i), ci(i,:), stats(i)] = ttest2(r1(:,i), r2(:,i)); % ttest2 comparison of two random, independent samples
end

• multcompare is designed for ANOVA results (See the documentation provided) and does not support outputs from ttest2. Instead, the Bonferroni correction can be applied manually by adjusting the significance level and comparing each p-value to the adjusted threshold.

Here is the corrected version of the code.

% Note that r1 and r2 differ in row number, not column number
r1 = randi(125, 200, 10); % Random sample 1 
r2 = randi(155, 100, 10); % Random sample 2

% Initialize variables to store results
h = zeros(1, 10);
p = zeros(1, 10);
ci = zeros(10, 2);
stats = struct('tstat', {}, 'df', {}, 'sd', {});

% Perform ttest2 comparison for each column
for i = 1:10
    [h(i), p(i), ci(i,:), stats(i)] = ttest2(r1(:,i), r2(:,i)); % ttest2 comparison of two random, independent samples
end

% Number of tests
numTests = length(p);

% Apply Bonferroni correction
alpha = 0.05; % Original significance level (Change as per requirement)
bonferroni_alpha = alpha / numTests;

% Adjust hypothesis test results based on Bonferroni correction
h_bonferroni = p < bonferroni_alpha;

% Display results
disp('Original hypothesis test results (h):');disp(h);

Original hypothesis test results (h):
     1     1     1     1     1     1     1     1     1     0

disp('P-values:');disp(p);

P-values:
    0.0003    0.0047    0.0002    0.0025    0.0024    0.0011    0.0000    0.0000    0.0000    0.2180

disp('Adjusted hypothesis test results with Bonferroni correction:');disp(h_bonferroni);

Adjusted hypothesis test results with Bonferroni correction:
   1   1   1   1   1   1   1   1   1   0

Refer to the following MathWorks documentation for multcompare and ttest2 respectively for additional clarification.

1. https://www.mathworks.com/help/stats/multcompare.html?s_tid=doc_ta#bujc800-2
2. https://www.mathworks.com/help/stats/ttest2.html

Hope this helps.

[Sara Woods]

Thank you so much! I did not know about manually performing Bonferroni correction, this will be absolutely useful for further statistical comparisons!