"double" results of multcompare on between x within interaction in repeated measures ANOVA

I am setting up a repeated measures ANOVA with one between and one within factor

The data table (data_table below) contains three variables (I am skipping variables that are not used in the analysis):

conn, Sdata, Tdata

conns is categorical between factor, there are 210 values ('conn1', 'conn2', etc), and each value is repeated 15 times for a total 3150 rows.

Sdata and Tdata are numeric and contain results of repeated measurements under two values of "state" (S and T)

I am mainly interested for which values of conn the difference between the S and T state is significant

The analysis is set up as follows:

within_design = table({'Sstate'; 'Tstate'}, 'VariableNames', {'state'});
rm = fitrm(data_table, 'Sdata,Tdata ~ conn', 'WithinDesign', within_design);
ranovatbl = ranova(rm, 'WithinModel', 'state');
multcompSTXConn = multcompare(rm, 'state', 'By', 'conn');   

This seems to work but the multcompare table compares S to T in both ways, like this:

conn    state_1     state_2      Difference          StdErr              pValue              Lower               Upper
'conn1'	'Sstate'	'Tstate'	0.0705799999999997	0.0353917140457907	0.0461245112376159	0.00121418034688942	0.139945819653110
'conn1'	'Tstate'	'Sstate'	-0.0705799999999997	0.0353917140457907	0.0461245112376159	-0.139945819653110	-0.00121418034688942
'conn2'	'Sstate'	'Tstate'	-0.0227266666666667	0.0353917140457907	0.520778041247466	-0.0920924863197770	0.0466391529864436
'conn2'	'Tstate'	'Sstate'	0.0227266666666667	0.0353917140457907	0.520778041247466	-0.0466391529864436	0.0920924863197770

Why is that? Is it an indication that I set up the analysis wrong, or is that a Matlab idiosyncrasy? Is the number of comparisons applied for the purpose of multiple-comparison-correction doubled and makes the test more conservative?

Answer 1

[Shivansh]

Hi Pawelq,

The results of the provided table doesn't indicate any errors in your analysis of the data. The "multcompare" function provide pairwise comparisons between levels of a factor, and when you have two levels, it will compare each level against the other, resulting in two rows: one for the comparison from "Sstate" to "Tstate" and another for the comparison from "Tstate" to "Sstate". The comparison in both directions does not double the number of comparisons for the purpose of the correction.

You can refer to the following documentation for more information about "multcompare": https://www.mathworks.com/help/stats/multcompare.html.