*! version 1.0 2024-2027

Title

icc2 -- Intraclass correlation coefficients based on crossed mixed regression

Syntax

icc2
varlist(min=2
max=3)
[if]
[in]
[,
options]

Optional
reps(#) perform # bootstrap replications, see(help bootstrap:bootstrap)

seed(#) set random-number seed to #, see(help bootstrap:bootstrap)

level(#) set confidence level; default is level(95)

Description

Textbooks often calculate the ICC using sums of squares on a
subject-by-measurement matrix with non-missing cells.

The idea of the ICC is to compare the wanted variation explained by a
factor variable on an outcome with the total variation, the total variation
being the wanted variation by the factor variable plus the unwanted variation.

Bias occurs sometimes from the measurement repetitions.

The ANOVA-like calculations ignore all measurements by a subject if just
one measurement for that subject is missing and may also return ICC estimates
below zero.

The latter is theoretically impossible.

To better utilize subjects with missing measurements and avoid obtaining
negative ICCs, it is better to use estimates from a mixed, crossed regression.

The command -icc2- returns a matrix with the absolute and consistency ICCs
with a 95% confidence interval and a P-value for the ICCs equal to zero.

The user can obtain more precise confidence intervals using the bootstrap.

Examples

Setup

`". webuse judges, clear"'

Calculate ICCs for one-way random-effects model

`". icc2 rating target"'

| ICC [95% CI] P(ICC=0)
-------------+---------------------------------------
absolute | 0.166 -0.272 0.603 0.458

A mixed crossed regression and nlcom is the
basis for the confidence interval whereas the icc uses the
F-distribution.

`". icc rating target"'

rating | ICC [95% conf. interval]
-----------------------+--------------------------------------
Individual | .1657418 -.1329323 .7225601

We can use bootstrap options to get a more precise confidence interval.

Setting the option seed alone, the default number of repetitions is 50.

`". icc2 rating target, seed(1)"'

| ICC [95% CI] P(ICC=0)
-------------+---------------------------------------
absolute | 0.166 -0.199 0.531 0.374

Calculate ICCs for two-way random-effects model.

`". icc2 rating target judge"'

| ICC [95% CI] P(ICC=0)
-------------+---------------------------------------
absolute | 0.290 -0.111 0.691 0.157
consistency | 0.715 0.394 1.036 0.000

We can use the bootstrap with 100 repetitions for the confidence interval.

`". icc2 rating target judge, seed(1) reps(100)"'

| ICC [95% CI] P(ICC=0)
-------------+---------------------------------------
absolute | 0.290 0.119 0.461 0.001
consistency | 0.715 0.560 0.870 0.000

The difference between the absolute and consistency ICCs indicates bias from the judges.

Calculate ICCs for two-way mixed-effects model

As argued in 2019 Liljequist, the two-way random-effects model is the same as the two-way mixed-effects model.

`". icc2 rating target judge"'

Stored results

Matrices

r(icc2) The absolute and consistency ICCs with a 95% confidence
interval and a P-value for the ICCs equal to zero.

References

1979 Shrout - Intraclass correlations-uses in assessing rater reliability

1996 McGraw - Forming Inferences About Some Intraclass Correlation Coefficients

2006 Marchenko - Estimating variance components in Stata

2021 Bruun - Regression modeling for reliability/ICC in Stata

2019 Liljequist - Intraclass correlation – A discussion and demonstration of basic features

Authors and support

Author:

Niels Henrik Bruun,

Aalborg University Hospital

Support:

niels.henrik.bruun@gmail.com