{smcl} {cmd:help kalpha} {hline} {p 4 4 2} {cmd:kalpha} has been superseded by {helpb kappaetc} that is available from the {help ssc:SSC} archives. {cmd:kappaetc} estimates Krippendorff's alpha and other inter-rater agreement coefficients along with their standard errors and confidence intervals. {cmd:kalpha} continues to work but there will not be any further updates for the software. {title:Title} {p 5} {cmd:kalpha} {hline 2} Krippendorff's Alpha-Reliability {title:Syntax} {p 5} Calculate Krippendorff's alpha coefficient {p 8} {cmd:kalpha} {varlist} {ifin} [ {cmd:,} {it:options} ] {p 5} Create a variable containing Krippendorff's alpha {p 8} {cmd:egen} {dtype} {newvar} {cmd:= kalpha(}{varlist}{cmd:)} {ifin} [{cmd:,} {it:options}] {synoptset 28 tabbed}{...} {marker opts}{...} {synopthdr} {synoptline} {synopt:{cmd:{ul:s}cale(}{it:metric}{cmd:)}}specify data metric (level of measurement). Default is {opt i:nterval} for numeric variables and {opt n:ominal} for string variables{p_end} {synopt:{opt t:ranspose}}specify that variables are observers (raters, coders, judges, ...) and observations are units of analysis (subjects, objects, ...){p_end} {synopt:{cmd:{ul:boot}strap}[{cmd:(}{it:boot_options}{cmd:)}]}bootstrap confidence interval and probability to fail to reach a minimum alpha{p_end} {synoptline} {p2colreset}{...} {p 4 6 2} {cmd:by} is allowed; see {manlink D by}. {p_end} {title:Description} {pstd} {cmd:kalpha} calculates Krippendorff's alpha reliability coefficient (Hayes and Krippendorff 2007; Krippendorff 2004, 2011, 2013a). {pstd} {cmd:kalpha} assumes that each observation is an observer (rater, coder, judge, ...) and that variables are units of analysis (subjects, objects, ...). Thus, the first observation identifies the first observer, the second observation identifies the second rater, and so on. The first variable is the first unit of analysis, the second variable is the second unit of analysis, and so on. {pstd} In most datasets, observations are units of analysis (subjects, objects, ...) and variables are observers (raters, coders, judges, ...). In this case, the {helpb kalpha##transpos:transpose} option must be specified. {pstd} {cmd:kalpha} assumes that numeric variables are measured on interval scale and string variables are measured on nominal scale. See option {helpb kalpha##scale:scale()} below. {pstd} The {opt bootstrap} option implements the algorithm described in Krippendorff (2013b). This algorithm has recently been criticized by Gwet (2015) and Zapf et al. (2016). {title:Options} {marker scale}{...} {phang} {opt scale(metric)} specifies the data's metric or level of measurement. Default {it:metric} is {opt interval} for numeric variables and {opt nominal} for string variables. {cmd:kalpha} also supports {opt o:rdinal}, {opt r:atio}, {opt c:ircular} and {opt p:olar} data. {p 8 8 2} For circular data {it:metric} may be specified as {opt c:ircular}[{cmd:(}{it:U}{cmd:)}], where {it:U} specifies the fixed circumference (i.e. number of equal intervals) of the circle. If {it:U} is not specified it defaults to {cmd:{it:max} - {it:min} + 1}, where {it:max} and {it:min} are the observed maximum and minimum values in the data. Differences are calculated in radiance. To calculate differences in degrees, specify {opt circulard:eg}[{cmd:(}{it:U}{cmd:)}]. {p 8 8 2} For polar data {it:metric} may be specified as {opt p:olar}[{cmd:(}{it:min max}{cmd:)}], where {it:min} and {it:max} specify the minimum and maximum value of the scale and default to their observed counterparts if not specified. {marker transpose}{...} {phang} {opt transpose} specifies that variables are observers (coders, judges, raters, ...) and observations are units of anlysis (subjects, objects, ...). This option does not affect the dataset itself. The {cmd:by} prefix, {cmd:if}, and {cmd:in} qualifiers apply to the original structure of the dataset. See {helpb xpose} to interchange observations and variables in the dataset. {phang} {cmd:bootstrap}[{cmd:(}{it:boot_options}{cmd:)}] bootstraps the distribution of Krippendorff's alpha to obtain confidence intervals and probabilities to fail to reach a required minimum alpha. {it:boot_options} are {p 8 8 2} {opt r:eps(#)} specifies the number of bootstrap replications and defaults to 20,000. {p 8 8 2} {opt l:evel(#)} sets the confidence level. Default is {cmd:level({ccl level})}. {p 8 8 2} {opt mina:lpha(numlist)} bootstraps the probabilities to fail to reach a minimum alpha of {it:{help numlist}}. {p 8 8 2} {opt seed(#)} sets the random-number seed. This option may not be combined with {cmd:by}. If {cmd:by} is specified use {helpb set seed} instead. {p 8 8 2} [{cmd:no}]{cmd:dots}[{cmd:(}{it:#}{cmd:)}] prints a dot each {it:#} replication, where {it:#} defaults to {cmd:max(1, floor(}{it:reps}{cmd:/50))}. {title:Example} {phang2}{cmd:. clear}{p_end} {phang2}{cmd:. inp u1 u2 u3 u4 u5 u6 u7 u8 u9 u10 u11 u12}{p_end} {p 10}{cmd:1 2 3 3 2 1 4 1 2 . . .}{p_end} {p 10}{cmd:1 2 3 3 2 2 4 1 2 5 . 3}{p_end} {p 10}{cmd:. 3 3 3 2 3 4 2 2 5 1 .}{p_end} {p 10}{cmd:1 2 3 3 2 4 4 1 2 5 1 .}{p_end} {phang2}{cmd:end}{p_end} {phang2} {cmd:. kalpha u1-u12} {p_end} {phang2} {cmd:. kalpha u1-u12 , scale(ordinal)} {p_end} {phang2} {cmd:. kalpha u1-u12 , scale(nominal) bootstrap(minalpha(0.667 0.8))} {p_end} {title:Saved results} {pstd} {cmd:kalpha} saves the following in {cmd:r()}: {pstd} Scalars{p_end} {synoptset 15 tabbed}{...} {synopt:{cmd:r(kalpha)}}Krippendorff's alpha coefficient{p_end} {synopt:{cmd:r(observers)}}number of observers (coders, judges, raters, ...){p_end} {synopt:{cmd:r(units)}}number of units (with pairable values){p_end} {synopt:{cmd:r(n)}}number of pairable values (n_..){p_end} {synopt:{cmd:r(Do)}}observed disagreement{p_end} {synopt:{cmd:r(De)}}expected disagreement{p_end} {pstd} Macros{p_end} {synoptset 15 tabbed}{...} {synopt:{cmd:r(metric)}}{it:metric} (level of measurement){p_end} {pstd} Matrices{p_end} {synoptset 15 tabbed}{...} {synopt:{cmd:r(coin)}}coincidence matrix{p_end} {synopt:{cmd:r(delta2)}}delta matrix{p_end} {synopt:{cmd:r(uniqv)}}distinct values (numeric variables only) {p_end} {pstd} With the {opt bootstrap} option {cmd:kalpha} additionally returns {pstd} Scalars{p_end} {synoptset 15 tabbed}{...} {synopt:{cmd:r(level)}}confidence level{p_end} {synopt:{cmd:r(reps)}}number of replications{p_end} {synopt:{cmd:r(ci_lb)}}lower bound of confidence interval{p_end} {synopt:{cmd:r(ci_ub)}}upper bound of confidence interval{p_end} {pstd} Matrices{p_end} {synoptset 15 tabbed}{...} {synopt:{cmd:r(q)}}probabilities to fail to reach minimum alpha{p_end} {pstd} If {help version} is set to 15.1 (or lower), {cmd:kalpha} additionally returns {pstd} Matrices{p_end} {synopt:{cmd:r(rel)}}reliability data matrix (numeric variables only){p_end} {synopt:{cmd:r(vbu)}}values-by-units matrix{p_end} {synopt:{cmd:r(rsum)}}row sums (n_.c){p_end} {synopt:{cmd:r(csum)}}column sums (n_u.){p_end} {title:References} {pstd} Gwet, Kilem L. (2015). Standard Error of Krippendorff's Alpha Coeffcient. {it:K. Gwet's Inter-Rater Reliability Blog}. {browse "http://inter-rater-reliability.blogspot.de/2015/08/standard-error-of-krippendorffs-alpha.html"} {pstd} Hayes, Andrew F., Krippendorff, Klaus (2007). Answering the call for a standard reliability measure for coding data. {it:Communication Methods and Measures}, 1, 77-89. {pstd} Krippendorff, Klaus (2013a). Computing Krippendorff's Alpha-Reliability. {browse "http://www.asc.upenn.edu/usr/krippendorff/mwebreliability5.pdf"} {pstd} Krippendorff, Klaus (2013b). Algorithm for bootstrapping a distribution of c_a. {browse "http://www.asc.upenn.edu/usr/krippendorff/Bootstrapping%20Revised(5).pdf"} {pstd} Krippendorff, Klaus (2011). Agreement and Information in the Reliability of Coding. {it:Communication Methods and Measures}, 5(2), 93-112. {browse "http://dx.doi.org/10.1080/19312458.2011.568376"} {pstd} Krippendorff, Klaus (2004). Reliability in Content Analysis: Some common Misconceptions and Recommendations. {it:Human Communication Research 30}, 3, 411-433. {pstd} Zapf, Antonia, Castell, Stefanie, Morawietz, Lars, and Karch, André (2016). Measuring inter-rater reliability for nominal data which coeffcients and confidence intervals are appropriate? {it:BMC Medical Research Methodology}, 16:93. {title:Acknowledgments} {pstd} We are grateful to Klaus Krippendorff and Andrew Hayes for clarifying questions about the computation of reliability for only one unit of analysis. {title:Authors} {pstd} Daniel Klein, INCHER-Kassel, University of Kassel, klein.daniel.81@gmail.com {pstd} Rafael Reckmann, University of Kassel, rafael.reckmann@hotmail.com {title:Also see} {psee} Online: {helpb kappa}, {helpb icc}, {helpb spearman}, {helpb egen} {p_end} {psee} if installed: {helpb krippalpha}, {helpb kappaetc} {p_end}