*! version 1.0.0 24Jun2023

Title

finn -- Finn's coefficient of reliability

Syntax

finn
varname1
varname2
[varname3 ...]
[if] [in]
[,
Id(varname)
CATegories(#)]

id(varname) subject identifier; when id() is not specified, the Finn coefficient is computed assuming that observations are independent

categories(#) the number of categories in a categorical level rating system; when categories() is not specified the data are assumed to be continuous

by is allowed with finn; see [D] by.

Description

finn computes Finn's reliability coefficient (Finn 1970). Although initially designed for categorical (sequential integer)
ratings (e.g. 1,2,3,4,5), Gwet (2021) has generalized Finn's coefficient to accommodate continuous (quantitative) ratings.
The coefficient, which is also known in the literature as a Within-Group Inter-Rater Reliability statistic, may be considered
an alternative to the intraclass correlation coefficient (ICC) for estimating inter-rater reliability. The statistic may be helpful
when the sample is either small or not sufficiently diverse -- which will result in a very low ICC, even though there may be
a good-to-high correlation between ratings.

To compute the Finn's reliability coefficient, first, all subject-level variances are averaged to obtain the mean subject variance (MSV). This
number tells us how far on average the rating from any given judge will stray away from the average rating. The smaller the mean subject variance,
the higher the rater agreement. Second, we compute the expected value that this mean subject variance (eMSV) would take if the ratings were assigned to subjects
in a purely random manner. Finally Finn's coefficient is computed as 1-(MSV/eMSV). See Gwet (2021) for a comprehensive discussion.

Options

id(varname) specifies the subjects' identifier. When id() is not specified,
the Finn coefficient is computed assuming that observations are independent.

categories(#) the number of categories possible in the rating scale. For example, if the data are from a Likert scale with possible responses ranging from 0 to 7, the user
should specify categories(7). When categories() is not specified, the Finn coefficient is computed using Gwet's (2021) generalized method for continuous data.

Examples

Setup for repeated observations and continuous scores

. use example7_1.dta

These data have 4 raters, subjects have repeated measurements, and the data are continuous

. finn judge1- judge4, i(subject)

We now apply the bootstrap to compute 95% confidence intervals

. bootstrap finn = r(finn), cluster(subject) reps(1000): finn judge1- judge4, i( subject)

. estat bootstrap, all

Setup for independent observations and categorical scores

. example7_3.dta

These data have 5 raters and 4 independent observations. The ratings are on a 1-5 Likert scale even though the actual scores only range from 1-3

. finn judge1- judge5, cat(5)

We now apply the bootstrap to compute 95% confidence intervals

. bootstrap finn = r(finn), reps(1000): finn judge1- judge5, cat(5)

. estat bootstrap, all

Stored results

finn stores the following in r():

Scalars

r(MSV) mean subject variance

r(eMSV) expected mean subject variance

r(finn) finn's coefficient of reliability

References

Finn, R. H. 1970. A note on estimating the reliability of categorical data. Educational and Psychological Measurement 30: 71-76.

Gwet, K. L. 2021. Handbook of Inter-Rater Reliability, 5th Edition. Volume 2: Analysis of Quantitative Ratings. Gaithersburg, MD: Advanced Analytics.

Citation of finn

finn is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden A. (2023). FINN: Stata module to compute Finn's coefficient of reliability.

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Online: icc, kappaetc (if installed)