*! version 1.1.3 29dec2020

Title

holsti -- Calculates the Holsti intercoder reliability coefficient.

Syntax

holsti
varname [if] [in] [, options]

varname defines either the variable, for which the Holsti coefficient is computed,
or two or more coders, for which intercoder reliability shall be calculated.

Main
generate(var) define a variable containing the number of disagreements per case

by is not allowed; see [D] by.

fweights are not allowed; see weight.

Description

holsti calculates the intercoder reliability coefficient for different
coders' ratings of one given variable or the intracoder reliability coefficient
for one or more coders for one variable at different points in time. Variables
can be either integer or string; however, mixing both types will not work.

Options

Main

generate defines a variable in which the number of disagreements for each
analyzed case (given a certain variable or different coders) is stored (useful for
diagnostic purposes).

Remarks

(1) Disagreements for diagnostic purposes are computed as follows: holsti
retrieves the different codings that have been assigned to each case and computes
their respective frequency. The prevailing coding for a given case is defined as
"correct". Every occurence of a code different from the "correct" code is counted
as a disagreement. In case there are equal frequencies for more than one code,
holsti defines the first most occuring code as being "correct", the others as disagreeing.

(2) Please note that the dataset needs to have a certain structure in order for the
Holsti-coefficient to be calculated properly.

Each column in the dataset stands for one single coder and his or her ratings of
one given category. The rows define units which are rated concerning the given category.
If coders rate different categories, the coders' ratings of the different categories have
to be placed in separate columns.

Let c1, c2, c3 be different coders and stub1, stub2 different categories. The
corresponding dataset would be structured as follows:

+
+
|c1_stub1 c2_stub1 c3_stub1 c1_stub2 c2_stub2 c3_stub2 |
|
|
| 1 1 1 a a a |
| 2 2 2 c a c |
| 1 1 1 a b a |
| 3 2 3 b c c |
| 2 2 1 b b b |
+
+

Examples

. sysuse holstidata

. holsti stub1

. holsti stub1, gen(dev)

. holsti c1_stub1 c3_stub1

Stored Results

holsti stores the following in r():

Scalars

r(holsti) computed overall Holsti-coefficient

r(coder) number of coders

r(units) number of units belonging to one variable (i.e. rownumber in the dataset)

Macros

r(varcoder) name of the variable and/or the coders for which Holsti-coefficients were computed

r(varnames) list of all variables/coders (i.e. columns in the dataset) that holsti selects using r(varcoder)

Matrices

r(pw_holsti) matrix containing pairwise Holsti-coefficients

r(denominator) matrix containing the number of total codings for each pair of coders

r(numerator) matrix containing the number of disagreeing codings for each pair of coders

Authors

Mona Krewel, Victoria University of Wellington, mona.krewel@vuw.ac.nz

Julia Partheymueller, Vienna Center for Electoral Research, julia.partheymueller@univie.ac.at

Alexander Staudt, staudtlex@live.de

References

Holsti, O. R. 1969. Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley.

Lombard, M./Snyder-Duch, J./Bracken, C. C. 2002. Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability. Human Communication Research 28(4): 587-604.