*! version 10.1.1 20mar2013

Title

crater -- Calculates computer scorer vs. human rater agreement statistics

Syntax

crater
[]

Description

crater calculates the number of computer scores produced for each human score,
% missing, agreement raters, correlation, Kappa (unweighted, weighted, and quadratic weighted),
mean difference, and standardized mean difference. Detailed describtions of the kappa statistic
can be found in Fleiss, 1981. SMD is calculated as the mean difference between the computer and
human rater divided by the average variance.

Options

There are no options available for this command.

Remarks

For rules of thumb regarding reccomended agreement rates between computers and raters see
Williamson et. al. 2012.

Example syntax

. use reading.dta, clear

. crater human computer

References

Fleiss, J. L. (1981) Statistical methods for rates and proportions. 2nd ed.
(New York: John Wiley) pp. 38–46.

Williamson, D. M., Xi, X. and Breyer, F. J. (2012), A Framework for Evaluation and Use
of Automated Scoring. Educational Measurement: Issues and Practice, 31: 2–13.