*! version 1.3.3 29dec2020

Title

krippalpha -- Krippendorff's alpha intercoder reliability coefficient

Syntax

krippalpha
varlist [if] [in] [, method(datalevel) format(fmt)]

Note: the data used must be numeric or factor. String variables do not work.

Main
method set datalevel (nominal, ordinal, interval, ratio)

Options
format set display format for alpha

Description

krippalpha calculates Krippendorff's alpha intercoder reliability coefficient for given observations on all data levels.

Please note that the bootstrap-option added in version 1.2.0 has been removed in the current version due to conceptional issues.

Options

Main

method defines whether Krippendorff's alpha shall be calculated for nominal, ordinal, interval or ratio level data.

Options

format(fmt) sets the display format for alpha. Default is %-8.3f. For more details, see format.

Remarks

Please note that the dataset needs to have a certain structure in order for Krippendorff's alpha to be calculated properly.

Each column in the dataset stands for one single coder and his or her ratings of one given category. The rows define units which are rated concerning the given category. If coders rate different
categories, the coders' ratings of the different categories have to be placed in separate columns.

Let c1, c2, c3 be different coders and stub1, stub2 different categories.
The corresponding dataset would be structured as follows:

+
+
|c1_stub1 c2_stub1 c3_stub1 c1_stub2 c2_stub2 c3_stub2 |
|
|
| 1 1 1 1 1 1 |
| 2 2 2 3 1 3 |
| 1 1 1 1 2 1 |
| 3 2 3 2 3 3 |
| 2 2 1 2 2 2 |
+
+

Examples

. sysuse krippendorff

. krippalpha *stub1, method(nominal)

. krippalpha c1_stub1 c3_stub1, method(nominal)

. krippalpha *stub2, method(ordinal)

The examples above compute coefficients for single variables. If one wants to compute an average Krippendorff's alpha (i.e. over different variables), consider following code:

sysuse krippendorff

foreach x in stub1 stub2 {

krippalpha *`x', method(nominal)

gen alpha_`x' =`r(k_alpha)'

}

qui egen alphamean=rowmean(alpha*) in 1

qui sum alphamean

scalar overallalpha=`=r(mean)'

drop alpha_* alphamean

display `=overallalpha'

Stored Results

krippalpha stores the following in r():

Scalars

r(k_alpha) Krippendorff's alpha intercoder reliability coefficient

r(rater) number of coders

r(units) number of rated cases

Macros

r(method) contains level of the data for which Krippendorff's alpha was computed

Matrices

r(table) matrix contains the scalars described above

Authors

Alexander Staudt, staudtlex@live.de

Mona Krewel, Victoria University of Wellington, mona.krewel@vuw.ac.nz

References

Krippendorff, K. 1980. Content Analysis: An Introduction to Its Methodology.
Beverly Hills, CA: Sage.

Krippendorff, K. 2011. Computing Krippendorff's Alpha-Reliability. Departmental Papers (ASC) 43. http://repository.upenn.edu/asc_papers/43/

Lemon, J. 2013. kripp.alpha. In: Gamer, M./Lemon, J./Fellows, I./Singh, P. Package 'irr': Various Coefficients of Interrater Reliability and Agreement
(version 0.84). http://ftp5.gwdg.de/pub/misc/cran/web/packages/irr/index.html.

Acknowledgements

The code computing Krippendorff's alpha coefficient draws on Jim Lemon's implementation "kripp.alpha" for R; the implementation is part of the "irr"-package, maintained by Matthias Gamer.