*! version 1.0 31Mar2016

Title

kanom -- Krippendorff's alpha for nominal variables

Syntax

kanom varname1 varname2
[, options]

Main
wvar weight variable

cvar cluster variable

svar stratum variable

noinf skips inference

Description

kanom computes points estimates and 95 percent confidence intervals for Krippendorff's reliability coefficient alpha, for nominal
variables and two measurements. It also tests the null hypotheses that alpha is not larger than 0.67, 0.75 and 0.80, and reports the
corresponding p-values. Calculations are based on the "delta method" (an approximation appropriate in large samples), and optionally take
into account the main features of a complex sampling design.

Options

Main

wvar(varname) specifices a sampling weight variable.

cvar(varname) specifies that the observations are independent across groups (clusters)
but not necessarily within groups; varname specifies to which group each
observation belongs.

svar(varname) specifies the name of a variable that contains sampling stratum identifiers.

noinf specifies that only point estimates be computed; this may be useful when the variables have many values, as in that case computing confidence intervals and conducing hypothesis tests may become very slow.

Comparison with other commands/software

Unlike the previously existing Stata command krippalpha, SAS and SPSS macros "kalpha" (Hayes and Krippendorff 2007), and R package "irr," kanom
allows taking into account the main features of complex survey designs (sampling weights, clusters, strata). Unlike krippalpha and "irr," kanom
produces confidence intervals and tests the null hypotheses that alpha is not larger than 0.67, 0.75, and 0.8. Unlike the SAS and SPSS macros, which
base statistical inference on the nonparametric bootstrap, in kanom inference is based on the delta method. The previously existing programs, however,
allow computation of Krippendorff's alpha for variables with various metrics, not just the nominal metric.

Examples

. sysuse kanom_exfile.dta", clear

. kanom measure1 measure2

. kanom measure1 measure2, wvar(sampwgt)

. kanom measure1 measure2, wvar(sampwgt) noinf

. kanom measure1 measure2, wvar(sampwgt) cvar(cluster) svar(strat)

Stored results

kanom stores the following in e():

Scalars

e(N) number of observations

e(pe) point estimate

e(lb) lower bound

e(ub) upper bound

e(se) standard error

e(p_67) p-value of H0 that alpha < 0.67

e(p_75) p-value of H0 that alpha < 0.75

e(p_80) p-value of H0 that alpha < 0.80

Macros

e(cmd) kanom

e(var1) name of variable 1

e(var2) name of variable 2

e(wvar) name of weight variable

e(cvar) name of cluster variable

e(svar) name of stratum variable

Author

Pablo A. Mitnik, Center on Poverty and Inequality, Stanford University

Suggested citations if using kanom in published work

kanom is not an official Stata command. It is a free contribution to the research community, produced as a companion to Mitnik and Cumberworth (2016). Citation of this paper when using kanom would be appreciated.

References

Hayes, Andrew, and Klaus Krippendorff. 2007. "Answering the Call for a Standard Reliability Measure for Coding Data." Communication Methods and Measures 1(1):77-89.

Krippendorff, Klaus. 2013. Content Analysis: An Introduction to its Methodology (3rd edition). Los Angeles: Sage.

Mitnik, Pablo and Erin Cumberworth. 2016. "Measuring Social Class Trends with Changing Occupational Classifications: Reliability, Competing Measurement Strategies, and the 1970-1980 U.S. Classification Divide."
Stanford Center on Poverty and Inequality Working Paper.

Disclaimer

kanom is provided as is, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement.