*! version 1.0.0 11nov2022

Title

power tworoc -- Power analysis for a two-sample (independent or paired) ROC analysis

Syntax

Compute sample size

power tworoc auc0 auc1 auc2
[, power(numlist) alpha(numlist) ratio(numlist) corr(numlist) onesided hanley graph[(graphopts)] ]

Compute power

power tworoc auc0 auc1 auc2
[, n(numlist) n1(numlist) n0(numlist) alpha(numlist) ratio(numlist) corr(numlist) onesided ordinal
graph[(graphopts)] ]

where auc0 is the null (hypothesized) area under the ROC curve (AUC) and
auc1 and auc2 are the alternative (target) AUCs. auc0, auc1 and auc2 may each be
specified either as one number or as a list of values in parentheses
(see numlist).

* alpha(numlist) significance level; default is alpha(0.05)

* power(numlist) power; default is power(0.80)

* n(numlist) total sample size; required to compute power or effect size

* n1(numlist) sample size of the diseased group

* n0(numlist) sample size of the non-diseased group

* ratio(numlist) ratio of sample sizes, N0/N1; default is ratio(1), meaning equal group sizes

* corr(numlist) correlation between auc1 and auc2 when the same patients are being tested on both tests; default is corr(0),
meaning independent samples

onesided one-sided test; default is two sided

hanley variance functions computed using the Hanley and McNeil (1982) method; default is to use the method by Obuchowski et al. (2004)

graph[(graphopts)] graph results; see [PSS-2] power, graph

* Specifying a list of values in at least two starred options, or
two command arguments, or at least one starred option and one argument
results in computations for all possible combinations of the values; see
numlist.

Description

power tworoc computes sample size or power for a two-sample receiver operating characteristic (ROC) analysis.
When there is no correlation corr(0) between the two alternative AUCs (auc1 and auc2), power tworoc
computes sample size (power) for an independent two-sample test. When the correlation does not equal 0, power tworoc
computes sample size (power) for a paired two-sample test. Variance functions are computed using either the method described
in Obuchowski, Lieber and Wians (2004) for continuous and ordinal data assuming a binormal distribution (the default), or the method described by Hanley and McNeil (1982)
for only continuous data which is based on the Mann-Whitney version of the rank-sum test. power tworoc is loosely based upon
the code used in the SAS® macro ROCPOWER (Zepp 1995).

Options

alpha(numlist) sets the significance level of the test. The
default is a:lpha(0.05).

power(numlist) specifies the desired power at which sample size is to be computed.
If power() is specified in conjunction with n(), n1(), or n0(),
then the actual power of the test is presented.

n(numlist) specifies the number of subjects in the study to be used for determining power. For an independent two-sample
analysis n() indicates the total number of subjects taking one of the diagnostic tests (not the sum of both tests).

n1(numlist) specifies the number of subjects in the diseased group to be used for determining power. For an independent two-sample
analysis n1() indicates the number of diseased patients taking one of the diagnostic tests (not the sum of both tests).

n0(numlist) specifies the number of subjects in the non-diseased group to be used for determining power. For an independent two-sample
analysis n0() indicates the number of non-diseased patients taking one of the diagnostic tests (not the sum of both tests).

ratio(numlist) specifies the sample-size ratio of the non-diseased group relative to the diseased group,
N0/N1. The default is ratio(1), meaning equal allocation between the two groups.

corr(numlist) specifies the hypothesized correlation between auc1 and auc2. When the same patients
take both diagnostic tests, we expect a correlation between the AUCs. When different groups of patients take
the two diagnostic tests (i.e. independent samples), we expect the correlation to be 0. The default is
corr(0), meaning independent samples.

onesided indicates a one-sided test. The default is two sided.

hanley uses the Hanley and McNeil (1982) method for computing the variance functions for continuous data.
The default is the method described by Obuchowski, Lieber, and Wians (2004) which is designed for use with
continuous and ordinal data assuming a binormal distribution.

graph, graph(); see [PSS-2] power, graph.

Remarks: Using power tworoc

power tworoc computes sample size or power for
a two-sample ROC analysis. All computations are performed for a two-sided
hypothesis test where, by default, the significance level is set to 0.05. You
may change the significance level by specifying the alpha() option. You
can specify the onesided option to request a one-sided test.

To compute sample size, you must specify the AUCs under
the null hypothesis (auc0) and the alternative hypotheses (auc1 and auc2),
and the power of the test in the power() option. The default power
is set to 0.80.

To compute power, you must specify the sample size(s) in any of the n(),
n1() or n0() options, along with the AUCs under the null and
alternative hypotheses, auc0, auc1 and auc2, respectively.
Note that for an independent two-sample test, the user specifies the n, n1 or n0 of only one of the two tests, not the total of the two tests combined.

By default, the computed sample size is rounded up to the next integer.

Examples

Computing sample size - independent two-sample analysis

Compute the sample size required to detect a difference between two diagnostic tests
where the hypothesized AUCs are 0.70 and 0.90 and the null AUC is 0.70 for two independent
samples, using a two-sided test and computing variances using the Obuchowski et al. (2004) method; assume a 5% significance
level and 80% power (the defaults)

. power tworoc 0.50 0.70 0.90

Same as above, using a power of 90% and a one-sided test, computing variances
using the Hanley and McNeil (1982) method

. power tworoc 0.70 0.70 0.90, power(0.90) onesided hanley

Computing power - independent two-sample analysis

Compute the power to detect a difference between two diagnostic tests where the hypothesized
AUCs are 0.70 and 0.90 and the null AUC is 0.70, where the total sample size of one of the tests
is 100 subjects (we assume that the other test will also have 100 subjects) using a two-sided test
and computing variances using the Obuchowski et al. (2004) method; assume a 5% significance
level and 80% power (the defaults)

. power tworoc 0.70 0.70 0.90, n(100)

Same as above, but presume a ratio of 2 non-diseased patients for each diseased subject,
compute the power to detect a difference between two AUCs of 0.70 and 0.90, given a null AUC of 0.50
at the 5% significance level

. power tworoc 0.70 0.70 0.90, n(100) ratio(2)

Computing sample size - paired two-sample analysis

Compute the sample size required to detect a difference between two diagnostic tests
where the hypothesized AUCs are 0.70 and 0.90 and the null AUC is 0.70 and
the same patients are tested on both diagnostic tests, assuming a correlation of 0.45 between tests.
We specify the test to be one-sided and use the Obuchowski et al. (2004) method for computing variances.

. power tworoc 0.70 0.70 0.90, power(0.80) onesided corr(0.45)

Same as above, but applying a range of AUC values under the auc2
and setting alpha levels to 0.05 and 0.01, graphing the results

. power tworoc 0.70 0.70 (0.75(0.05)0.95), power(0.80) onesided alpha(0.01 0.05) corr(0.45) graph

Computing power - paired two-sample analysis

Compute the power to detect a difference between two diagnostic tests
where the same 60 patients take both tests and the correlation between
AUCs is 0.50 and the hypothesized AUCs are 0.70 and 0.90 and the null
AUC is 0.70, using a two-sided test and computing variances using the
Obuchowski et al. (2004) method; assume a 5% significance level and
80% power (the defaults)

. power tworoc 0.70 0.70 0.90, n(60) corr(0.50)

Same as above but specify a ratio of 2 non-diseased patients for each diseased patient, and
compute variances for ordinal data

. power tworoc 0.70 0.70 0.90, n(60) ratio(2) corr(0.50)

For a diseased group of 50 subjects and a non-diseased group of 80 subjects who took both diagnostic tests,
compute the power to detect a difference between two AUCs of 0.70 and 0.90,
given a null AUC of 0.70 at the 1% significance level, computing variances
using the Obuchowski et al. (2004) method and assume a correlation between AUCs of 0.30

. power tworoc 0.70 0.70 0.90, n1(50) n0(80) alpha(0.01) corr(0.30)

Compute powers for a range of auc2 and total sample sizes when auc1 is 0.70 and the null (auc0)
is 0.70, with the correlation between AUCs of 0.30, graphing the results

. power tworoc 0.70 0.70 (0.75(.05)0.95), n(10(5)120) corr(0.30) graph

Stored results

power tworoc stores the following in r():

Scalars

r(alpha) significance level

r(auc0) null AUC

r(auc1) alternative AUC1

r(auc2) alternative AUC2

r(beta) probability of a type II error

r(delta) effect size

r(divider) 1 if divider is requested in the table, 0 otherwise

r(ratio) ratio of sample sizes, N0/N1

r(corr) correlation between AUC1 and AUC2

r(N) total sample size

r(N0) sample size of the non-diseased group

r(N1) sample size of the diseased group

r(onesided) 1 for a one-sided test, 0 otherwise

r(power) power

r(V0) variance function of the null AUC

r(V1) variance function of the alternative AUC1

r(V2) variance function of the alternative AUC2

Macros

r(type) test

r(method) tworoc

r(columns) displayed table columns

r(labels) table column labels

r(widths) table column widths

r(formats) table column formats

Matrices

r(pss_table) table of results

References

Hanley, J.A., and B. J. McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29-36

Obuchowski, N.A., Lieber, M.L. and F.H. Wians Jr. 2004. ROC curves in clinical chemistry: uses, misuses, and possible solutions. Clinical chemistry 50:1118-1125.

Zepp, R.C. 1995. A SAS® macro for estimating power for ROC curves one-Sample and two-sample cases. Proceedings of the 20th SAS Users Group International Conference (Vol. 223).

Citation of power tworoc

power tworoc is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden A. (2022). POWER TWOROC: Stata module to compute power and sample size for a two-sample (independent or paired) ROC analysis

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Online: roc, power, power oneroc (if installed)