*! version 1.0.0 07nov2022

Title

power roc -- Power and sample-size analysis for receiver operating characteristic (ROC) analysis

Syntax

Compute sample size

power method
...
[, power(numlist)
power_options ...]

Compute power

power method
...
[, n(numlist)
[power_options ...]

method

One sample
oneroc One-sample ROC analysis

Two samples (independent and paired)
tworoc Two-sample ROC analysis

power_options

Main
* alpha(numlist) significance level; default is alpha(0.05)

* power(numlist) power; default is power(0.80)

* n(numlist) total sample size. For and independent two-sample analysis, n() indicates the total number of subjects taking one
of the diagnostic tests (not the sum of both tests)

* n1(numlist) sample size of the diseased group. For an independent two-sample analysis n1() indicates the number of diseased
patients taking one of the diagnostic tests (not the sum of both tests)

* n0(numlist) sample size of the non-diseased group. For an independent two-sample analysis n0() indicates the number of
non-diseased patients taking one of the diagnostic tests (not the sum of both tests)

* ratio(numlist) ratio of sample sizes, N0/N1; default is ratio(1), meaning equal group sizes

* corr(numlist) correlation between auc1 and auc2 when the same patients are being tested on both tests (power tworoc only); default is corr(0),
meaning independent samples.

onesided one-sided test; default is two sided

hanley variance functions computed using the Hanley and McNeil (1982) method; default is to use the method by Obuchowski et al. (2004)

graph[(graphopts)] graph results; see [PSS-2] power, graph

* Specifying a list of values in at least two starred options, or
two command arguments, or at least one starred option and one argument
results in computations for all possible combinations of the values; see
numlist.

Description

The power roc package is helpful for planning studies that use receiver operating characteristic (ROC) curves to evaluate the accuracy of diagnostic tests.
An advantage of ROC curves for evaluating the accuracy of diagnostic tests is that it incorporates both sensitivity (the probability of a test detecting disease
when the patient actually has the disease) and specificity (the probability that the test is negative when the patient is actually disease-free), into a single
measure of accuracy. power roc is loosely based upon the code used in the SAS® macro ROCPOWER (Zepp 1995).

power oneroc computes sample size or power for a one-sample receiver operating characteristic (ROC) analysis. Variance functions
are computed using either the method described in Obuchowski, Lieber and Wians (2004) for continuous and ordinal data assuming a binormal distibution (the default),
or the method described by Hanley and McNeil (1982) for continuous data which is based on the Mann-Whitney version of the rank-sum test.

power tworoc computes sample size or power for a two-sample receiver operating characteristic (ROC) analysis.
When there is no correlation corr(0) between the two alternative AUCs (auc1 and auc2), power tworoc
computes sample size (power) for an independent two-sample test. When the correlation does not equal 0, power tworoc
computes sample size (power) for a paired two-sample test. As with power oneroc, variance functions are computed using either the method described
in Obuchowski, Lieber and Wians (2004) for continuous and ordinal data assuming a binormal distribution (the default), or the method described by Hanley and McNeil (1982)
for only continuous data which is based on the Mann-Whitney version of the rank-sum test.

Sample size can be computed given power, and power can be computed given sample size. Results can be displayed in a table (default) and on a graph
([PSS-2] power, graph).

Options

alpha(numlist) sets the significance level of the test. The
default is alpha(0.05).

power(numlist) specifies the desired power at which sample size is to be computed.
If power() is specified in conjunction with n(), n1(), or n0(),
then the actual power of the test is presented.

n(numlist) specifies the total number of subjects in the study to be used for determining power. For an independent two-sample
analysis n() indicates the total number of subjects taking one of the diagnostic tests (not the sum of both tests).

n1(numlist) specifies the number of subjects in the diseased group to be used for determining power. For an independent two-sample
analysis n1() indicates the number of diseased patients taking one of the diagnostic tests (not the sum of both tests).

n0(numlist) specifies the number of subjects in the non-diseased group to be used for determining power. For an independent two-sample
analysis n0() indicates the number of non-diseased patients taking one of the diagnostic tests (not the sum of both tests).

ratio(numlist) specifies the sample-size ratio of the non-diseased group relative to the diseased group,
N0/N1. The default is ratio(1), meaning equal allocation between the two groups.

corr(numlist) specifies the hypothesized correlation between auc1 and auc2 (power tworoc only). When the same patients
take both diagnostic tests, we expect a correlation between the AUCs. When different groups of patients take
the two diagnostic tests (i.e. independent samples), we expect the correlation to be 0. The default is
corr(0), meaning independent samples.

onesided indicates a one-sided test. The default is two sided.

hanley uses the Hanley and McNeil (1982) method for computing the variance functions for continuous data.
The default is the method described by Obuchowski, Lieber, and Wians (2004) which is designed for use with
continuous and ordinal data assuming a binormal distribution.

graph, graph(); see [PSS-2] power, graph.

Examples

One-sample ROC analysis

Compute the sample size required to detect an AUC of 0.70 given an
AUC of 0.50 under the null hypothesis using a two-sided test and computing
variances for continuous data; assume a 5% significance level and 80%
power (the defaults)

. power oneroc 0.50 0.70

Same as above, using a power of 90% and a one-sided test, computing variances
using the Hanley and McNeil (1982) method

. power oneroc 0.50 0.70, power(0.90) onesided hanley

Applying a range of AUC values under the alternative hypothesis
and setting alpha levels to 0.05 and 0.01; and graphing the results

. power oneroc 0.50 (0.60(0.05).95), power(0.90) onesided ratio(4) alpha(0.01 0.05) graph

For a total sample of 50 subjects, compute the power of a two-sided test to
detect an AUC of 0.70 given a null AUC of 0.50 and computing variances using the Obuchowski et al. (2004) method;
assume a 5% significance level (the default)

. power oneroc 0.50 0.70, n(50)

For a diseased group of 20 subjects and a ratio of 2 non-diseased patients for each diseased subject,
compute the power of a one-sided test to detect an AUC of 0.70 given a null AUC of 0.50
at the 5% significance level

. power oneroc 0.50 0.70, n1(20) ratio(2) onesided

Compute powers for a range of alternative AUCs and total sample sizes,
graphing the results

. power oneroc 0.50 (0.70(.10)0.90), n(5(5)30) graph

Independent two-sample ROC analysis

Compute the sample size required to detect a difference between two diagnostic tests
where the hypothesized AUCs are 0.70 and 0.90 and the null AUC is 0.70 for two independent
samples, using a two-sided test and computing variances using the Obuchowski et al. (2004) method;
assume a 5% significance level and 80% power (the defaults)

. power tworoc 0.70 0.70 0.90

Same as above, using a power of 90% and a one-sided test, computing variances
using the Hanley and McNeil (1982) method

. power tworoc 0.70 0.70 0.90, power(0.90) onesided hanley

Compute the power to detect a difference between two diagnostic tests where the hypothesized
AUCs are 0.70 and 0.90 and the null AUC is 0.70, where the total sample size of one of the tests
is 100 subjects (we assume that the other test will also have 100 subjects) using a two-sided test
and computing variances using the Obuchowski et al. (2004) method;
assume a 5% significance level and 80% power (the defaults)

. power tworoc 0.70 0.70 0.90, n(100)

Same as above, but presume a ratio of 2 non-diseased patients for each diseased subject,
compute the power to detect a difference between two AUCs of 0.70 and 0.90, given a null AUC of 0.50
at the 5% significance level, computing variances using the Obuchowski et al. (2004) method

. power tworoc 0.70 0.70 0.90, n(100) ratio(2)

Paired two-sample ROC analysis

Compute the sample size required to detect a difference between two diagnostic tests
where the hypothesized AUCs are 0.70 and 0.90 and the null AUC is 0.70 and
the same patients are tested on both diagnostic tests, assuming a correlation of 0.45 between tests.
We specify the test to be one-sided and use the Obuchowski et al. (2004) method for computing variances.

. power tworoc 0.70 0.70 0.90, power(0.80) onesided corr(0.45)

Same as above, but applying a range of AUC values under the auc2
and setting alpha levels to 0.05 and 0.01, graphing the results

. power tworoc 0.70 0.70 (0.75(0.05)0.95), power(0.80) onesided alpha(0.01 0.05) corr(0.45) graph

Compute the power to detect a difference between two diagnostic tests
where the same 60 patients take both tests and the correlation between
AUCs is 0.50 and the hypothesized AUCs are 0.70 and 0.90 and the null
AUC is 0.70, using a two-sided test and computing variances using the Obuchowski et al. (2004) method;
assume a 5% significance level and 80% power (the defaults)

. power tworoc 0.70 0.70 0.90, n(60) corr(0.50)

For a diseased group of 50 subjects and a non-diseased group of 80 subjects who took both diagnostic tests,
compute the power to detect a difference between two AUCs of 0.70 and 0.90,
given a null AUC of 0.70 at the 1% significance level, computing variances
using the Obuchowski et al. (2004) method and assume a correlation between AUCs of 0.30

. power tworoc 0.70 0.70 0.90, n1(50) n0(80) alpha(0.01) corr(0.30)

Compute powers for a range of auc2 and total sample sizes when auc1 is 0.70 and the null (auc0)
is 0.50, with the correlation between AUCs of 0.30, graphing the results

. power tworoc 0.70 0.70 (0.75(.05)0.95), n(10(5)120) corr(0.30) graph

Stored results

power roc stores the following in r():

Scalars

r(alpha) significance level

r(auc0) null AUC

r(auc1) alternative AUC1

r(auc2) alternative AUC2 (power tworoc only)

r(beta) probability of a type II error

r(delta) effect size

r(divider) 1 if divider is requested in the table, 0 otherwise

r(ratio) ratio of sample sizes, N0/N1

r(corr) correlation between AUC1 and AUC2 (power tworoc only)

r(N) total sample size

r(N0) sample size of the non-diseased group

r(N1) sample size of the diseased group

r(onesided) 1 for a one-sided test, 0 otherwise

r(power) power

r(V0) variance function of the null AUC

r(V1) variance function of the alternative AUC1

r(V2) variance function of the alternative AUC2 (power tworoc only)

Macros

r(type) test

r(method) oneroc or tworoc

r(columns) displayed table columns

r(labels) table column labels

r(widths) table column widths

r(formats) table column formats

Matrices

r(pss_table) table of results

References

Hanley, J.A., and B. J. McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29-36

Obuchowski, N.A., Lieber, M.L. and F.H. Wians Jr. 2004. ROC curves in clinical chemistry: uses, misuses, and possible solutions. Clinical chemistry 50:1118-1125.

Zepp, R.C. 1995. A SAS® macro for estimating power for ROC curves one-Sample and two-sample cases. Proceedings of the 20th SAS Users Group International Conference (Vol. 223).

Citation of power roc

power roc is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden, A. 2022. POWER ROC: Stata package to compute power and sample size for ROC analyses

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Online: roc, power, power diagnostic (if installed)