Kruskal-Wallis multcompare comparison intervals

Hello,

currently I am analysing results of my study and I found that a good multcomparison for it would be using the kruskalwallis with dunn-sidak.

My question is now, how to interpret the comparison intervals, which are not confidence intervals. I know, that if they do not overlap, the data comes from different distributions. But how are these comparison intervals generated?

Through researching and playing with the data and multcompare graphs I found that the width of the comparison intervals equals to 4 x the standard error (obtained through [m] = multcompare(stats)). Why is that the case?

However, I cannot find a reference of how these standard errors are calculated. And I cannot find anything on the MATLAB documentation.

Can you help me?

Another question is how do I label the x-axis of the multcompare graph for a publication? In principal, 'ranks' should be correct, right?

Thank you and kind regards

Michael

[Scott MacKenzie]

It might help if you post your data and any MATLAB code you've written so far. Using MATLAB's kruskalwallis function is fairly striaghtforward. The documentation includes examples and also demonstrates doing a post-hoc multiple comparisons test using multcompare.

[Michael Werthmann]

Hello Scott,

thank you for your reply. I have no problems performing the kruskalwallis and multcompare on my data.

My question is theoretical and aims to get answers on how the comparison intervals are calculated, which I see after executing multcompare in this case (kruskalwallis -> multcompare ('c-type', 'dunn-sidak')) because I could not find documentation for that.

I guess, that the comparison intervals are calculated on the basis of the standard error, which is an output of multcompare. (https://www.mathworks.com/help/stats/multcompare.html#bujc800-m) But I could not find information regarding that either

Regards

Michael

[Scott MacKenzie]

Well, I can't recite the calculcations, but if you want to know how to do the caclulations, just look them up in a stats book. Many moons back, I wrote Java apps to do the Kruskal-Wallis and numerous other tests. The book I used and recommend is David Sheskin's Handbook of Parametric and Non-Parametrics Statistical Procedures from CRC Press.

[Michael Werthmann]

Hi thanks for your suggestion.

How would you label the x-axis in my case? 'ranks'?

Kind regards

Michael

[Scott MacKenzie]

The conversion to ranks is something that occurs behind the scenes in the Kruskal-Wallis and other non-parametric tests. The responses tested are usually just answers to questionnaire items, qualitative assessments, or something like that. I'm not sure what kind of chart you are contemplating or what the data represent, so I can't suggest an x-axis label.

[Michael Werthmann]

Okay, then I'll explain it.

I applied 4 different image registration to 30 image pairs and want to test if the algorithms could improve certain metrics.

For example, I tested if they significantly increased the Jaccard coefficient (segmentation overlap metric) compared to the initial value

In the top row, it is the initial comparison interval, followed by the intervals of the algorithms.

The boxplots for the data are also displayed. The initial Jaccard has the median around ~0.75. As you can see, the medians of the results from the algorithms are higher.

I used kruskal wallis and dunn sidak to create the multcompare.

All mean ranks of the algorithms are different compared to the initial, thus they are significantly different.

Does that help you in suggesting a x-label?

[Scott MacKenzie]

I'm not sure about the 2nd plot, but for the box plot, let's start first with the y-axis.

I doubt "Jaccard" is the correct label. The label should be whatever the values represent. For the first box, the value is about 0.77. What is that? Perhaps "Coefficient Increase" or "Segmentation Overlap", or whatever the measure represents.

For the x-axis label, an appropriate label is probably something like "Image Registration Technique". For the x-axis tick labels, the first one should probably be "Jaccard". The other four tick labels should be names associated with the algorithms. Probably, "Algorithm 1", "Algorithm 2", etc., is a bit bland. I'm sure you have better names for these. Hope this helps. Good luck.