*! version 0.31

Title

confmat -- Confusion matrix calculations

Syntax

confmat
gold_std_var test_var
[if]
[in]
[,
options]

Optional
by(varname numeric) Compare two confusion matrices grouped by varname.

scale(#) Scale estimates and confidence intervals. Default value is 100 (ie %).

level(cilevel) level for confidence intervals. Default is c(level).

citype(string) Type of confidence intervals. See options at cii proportions.

labels Use labels when possible.

noquietly Show underlying code and log output.

coleq(string) Add coleq label to single confusion matrix.

cmptype(string) Type of comparison for two confusion matrices.
Is always a test. Option rr adds relative risks with confidence intervals.
Option rd adds risk differences. See cs.

exact Whether the test is chisquare test (default) or Fisher's
exact test. See cs.

Description

Given a binary variable for the gold standard and a binary variable for
the test confmat returns sensitivity, specificity, prevalence, accuracy,
ppv and npv (ie derived values from the confusion matrix) with confidence intervals.

When a binary grouping variable is specified in option by the
confusion matrix for each grouping values is reported as well as comparison tests.

To learn more about confusion matrices see
Wikipedia on sensitivity and specificity
and Wikipedia on Confusion_matrix

Examples

Example data from "https://en.wikipedia.org/wiki/Sensitivity_and_specificity"
cls
clear
input actual test n
0 0 1820
0 1 180
1 0 10
1 1 20
end
expand n

To get sensitivity, specificity, prevalence, accuracy, ppv and npv
(ie derived values from the confusion matrix) with confidence intervals.

confmat actual test

Example data for two confusion matrices

clear
input actual test n grp
0 0 1820 1
0 1 180 1
1 0 10 1
1 1 20 1
0 0 1620 2
0 1 150 2
1 0 18 2
1 1 40 2
end
expand n

One can condition to get a single confusion matrix.

confmat actual test if grp == 1

-------------------------------
test
0 1 Total
-------------------------------
actual 0 1820 180 2000
1 10 20 30
Total 1830 200 2030
-------------------------------

-----------------------------------------------------------
N p(%) [95% Conf. interval]
-----------------------------------------------------------
Sensitivity P(TP|C+) 30 66.67 47.19 82.71
Specificity P(TN|C-) 2000 91.00 89.66 92.22
Prevalence P(C+) 2030 1.48 1.00 2.10
Accuracy P(TP + TN) 2030 90.64 89.29 91.87
PPV P(TP|P+) 200 10.00 6.22 15.02
NPV P(TN|P-) 1830 99.45 99.00 99.74
-----------------------------------------------------------

Or get a comparison report including relative risks of the two confusion matrices.

confmat actual test, by(grp) cmp("rr")

-----------------------------------------------------------------------------------------------------------------------------------------------------
grp(1) grp(2) grp(1 vs 2)
N p(%) [95% Conf. interval] N p(%) [95% Conf. interval] RR [95% Conf. interval] P(Chisquare)
-----------------------------------------------------------------------------------------------------------------------------------------------------
Sensitivity P(TP|C+) 30 66.67 47.19 82.71 58 68.97 55.46 80.46 0.97 0.71 1.31 0.83
Specificity P(TN|C-) 2000 91.00 89.66 92.22 1770 91.53 90.13 92.78 0.99 0.97 1.01 0.57
Prevalence P(C+) 2030 1.48 1.00 2.10 1828 3.17 2.42 4.08 0.47 0.30 0.72 0.00
Accuracy P(TP + TN) 2030 90.64 89.29 91.87 1828 90.81 89.39 92.10 1.00 0.98 1.02 0.86
PPV P(TP|P+) 200 10.00 6.22 15.02 190 21.05 15.49 27.54 0.47 0.29 0.78 0.00
NPV P(TN|P-) 1830 99.45 99.00 99.74 1638 98.90 98.27 99.35 1.01 1.00 1.01 0.07
-----------------------------------------------------------------------------------------------------------------------------------------------------

Labels can be included in the returned matrix.

label variable grp "Group"
label define grp 1 "Poor" 2 "Rich"
label values grp grp
confmat actual test, by(grp) label

----------------------------------------------------------------------------------------------------------------------------------
Group(Poor) Group(Rich) Group(Poor vs Rich)
N p(%) [95% Conf. interval] N p(%) [95% Conf. interval] P(Chisquare)
----------------------------------------------------------------------------------------------------------------------------------
Sensitivity P(TP|C+) 30 66.67 47.19 82.71 58 68.97 55.46 80.46 0.83
Specificity P(TN|C-) 2000 91.00 89.66 92.22 1770 91.53 90.13 92.78 0.57
Prevalence P(C+) 2030 1.48 1.00 2.10 1828 3.17 2.42 4.08 0.00
Accuracy P(TP + TN) 2030 90.64 89.29 91.87 1828 90.81 89.39 92.10 0.86
PPV P(TP|P+) 200 10.00 6.22 15.02 190 21.05 15.49 27.54 0.00
NPV P(TN|P-) 1830 99.45 99.00 99.74 1638 98.90 98.27 99.35 0.07
----------------------------------------------------------------------------------------------------------------------------------

Stored results

confmat stores the following in r():

Matrices

r(confmat) The estimated values from one or two confusion matrices
and if two matrices comparison calculations in a table/matrix.

Authors and support

Author:

Niels Henrik Bruun,

Aalborg University Hospital

Support:

niels.henrik.bruun@gmail.com