*! version 1.0.0 14July2023

Title

iota -- Iota coefficient of interrater agreement for interval or nomimal multivariate observations

Syntax

iota
depvar1
[depvar2 ...]
[if] [in]
,
rater(varname)
target(varname)
[ standardize
nominal
]

rater(varname) rater identifier; rater() is required

target(varname) subject identifier; target() is required

standardize z-transforms the depvar(s) when the data type is interval (quantitative)

nominal indicates that the depvar(s) are nominal level data

by is allowed with iota; see [D] by.

Description

iota computes the iota coefficient of interrater agreement proposed by Janson and Olsson (2001). This measure of agreement
is an extension of Cohen's (1960) kappa for univariate (single depvar) nominal data and Berry and Mielke's (1988) generalization
of Cohen's kappa agreement measure to univariate (single depvar) interval level data to the multivariate case.

As described by Janson and Olsson (2001), their proposed extension of Cohen's agreement measure is applicable to interobserver
agreement in the case where a set of two or more judges (sampled from a larger population of possible judges) have rated a set of
targets (sampled from a larger population of targets) on several dimensions or variables. The basic assumptions are that targets are
independent and that judges have operated independently. When there are two judges and one variable, iota is exactly equivalent to
Cohen's (1960) kappa. And when there is one variable and several judges, iota exactly equals Berry and Mielke's (1988) R and
also Conger's (1980) "Fleiss exact" coefficient — a modification of a measure originally proposed by Fleiss (1971).

Options

rater(varname) specifies the raters' identifier; rater() is required.

target(varname) specifies the subjects' identifier. target() is required.

standardize z-transforms the depvar(s) when the data type is interval (quantitative). This is particularly appropriate
when the depvars are on different scales.

nominal indicates that the depvar(s) are nominal level data.

Examples

Setup for interval (quantitative) level data

. use iota_interval.dta

These data are from Example 1 in Janson and Olsson (2001) where three judges have
rated height and weight of five men on the basis of photographs.

. iota weight height, rater(judge) target( id)

Same as above, but we standardize the variables because they are on different scales.

. iota weight height, rater(judge) target( id) standardize

We now apply the bootstrap to compute 95% confidence intervals

. bootstrap iota = r(iota), reps(1000): iota weight height, rater(judge) target(id) standardize

. estat bootstrap, all

Setup for nominal level data

. iota_nominal.dta

These data are from Example 2 in Janson and Olsson (2001) where two judges classify six targets in
terms of their color and shape.

. iota shape color, rater(judge) target(id) nominal

We now apply the bootstrap to compute 95% confidence intervals.

. bootstrap iota = r(iota), reps(1000): iota shape color, rater(judge) target(id) nominal

. estat bootstrap, all

Stored results

iota stores the following in r():

Scalars

r(iota) the iota coefficient of interrater agreement

r(ntar) the number of unique targets (subjects)

r(nrat) the number of unique raters

r(nvar) the number of variables

r(obs) the total number of observations

References

Berry, K. J., & P. W. Mielke Jr. 1988. A generalization of Cohen's kappa agreement measure to interval measurement and multiple raters.
Educational and Psychological Measurement 48: 921-933.

Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37-46.

Conger, A.J. 1980. Integration and generalisation of Kappas for multiple raters. Psychological Bulletin 88: 322-328.

Fleiss, J. L. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76: 378-382.

Janson, H., & U Olsson. 2001. A measure of agreement for interval or nominal multivariate observations. Educational and Psychological Measurement
61: 277-289.

Citation of iota

iota is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden A. (2023). IOTA: Stata module to compute the Iota coefficient of interrater agreement for interval or nominal multivariate observations.

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Online: icc, kappaetc (if installed)