*! version 1 2019-11-29

ciwidth proportions_mc -- Precision and power analysis for a CI-based comparison of two independent proportions (RD, RR, or OR)

Syntax

Precision analysis:

Compute CI width

ciwidth proportions_mc,
probwidth(numlist)
p1(numlist)
nspec
[options]

Compute probability of CI width

ciwidth proportions_mc,
width(numlist)
p1(numlist)
nspec
[options]

Power analysis:

Compute power focusing on lower bound of CI

ciwidth proportions_mc,
lbgt(numlist)
table(lbgt power, add)
p1(numlist)
nspec
[options]

Compute power focusing on upper bound of CI

ciwidth proportions_mc,
ublt(numlist)
table(ublt power, add)
p1(numlist)
nspec
[options]

nspec specifies N1 and N2 using a combination of n(numlist), n1(numlist), n2(numlist), nratio(numlist)

options

* effecttype(numlist) Effect Size type; default is effecttype(1) = risk difference (RD). Alternatively, effecttype(2) = relative risk (RR), effecttype(3) = odds ratio (OR)

Main
* level(numlist) two-sided confidence level; default is level(95)

* probwidth(numlist) probability of CI width; required to compute CI width

* width(numlist) CI width, i.e. CI_ub - CI_lb if effecttype(1); CI_ub / CI_lb if effecttype(2) or effecttype(3); required to compute probability of CI width

* halfwidth(numlist) CI half-width, i.e. CI_ub - observed RD if effecttype(1); CI_ub / observed RR if effecttype(2); CI_ub / observed OR if effecttype(3); can be specified in place of width

* n(numlist) total sample size

* n1(numlist) sample size of the control group

* n2(numlist) sample size of the experimental group

* nratio(numlist) ratio of sample sizes, N2/N1; default is nratio(1), meaning equal group sizes

* p1(numlist) is the proportion in the control (reference) group; required

* p2(numlist) is the proportion in the experimental (comparison) group

* trueeffectsize(numlist) difference between p2 and p1, i.e. p2-p1 if effecttype(1); p2/p1 if effecttype(2); p2(1-p1)/p1(1-p2) if effecttype(3)

* lbgt(numlist) compute power that CI_lb > specified value

* ublt(numlist) compute power that CI_ub < specified value

Table
table(tablespec) see [PSS-3] ciwidth, table

saving(filename [, replace]) save the table data to filename; use replace to overwrite existing filename

parallel treat number lists in starred options or in command arguments as parallel when multiple values per option or argument are specified (do not enumerate all possible combinations of values)

notitle suppress the title

clear leave in memory all possible study results

* Specifying a list of values in at least two starred options
results in computations for all possible combinations of the values; see
numlist. Also see the parallel option.

where tablespec is

column[:label]
[column[:label] [...]]
[, tableopts]

column is one of the columns defined
below, and label is a column label (may
contain quotes and compound quotes).

column

ES_type effecttype; 1=RD, 2=RR, 3=OR

level confidence level

N total number of subjects

N1 number of subjects in the control group

N2 number of subjects in the experimental group

nratio ratio of sample sizes, experimental to control

p1 the proportion in the control group

p2 the proportion in the experimental group

true_ES numerical RD/RR/OR difference between p2 and p1

Pr_width probability of CI width

width CI width

halfwidth CI halfwidth

Pr_noCI probability of no CI calculated

lbgt CI_lb > specified value

ublt CI_ub < specified value

power power

_all display all supported columns

Columns nratio Pr_noCI lbgt ublt power are NOT shown in the default table

Description

ciwidth proportions_mc computes confidence interval (CI) width, and
probability of CI width for a CI for a difference between two proportions
from independent samples. See [PSS-3] ciwidth for precision
analysis for other CI methods.

ciwidth proportions_mc can also be used to do CI-based power calculations
(see [PSS-2] power twoproportions),
including non-inferiority power calculations.

N.B. The difference between two proportions can be quantified as the risk
difference (RD), the relative risk (RR) or the odds ratio (OR). For the risk difference,
I have expressed the width of the CI in the straightforward manner:
width = CI_ub - CI_lb. For the relative risk or the odds ratio,
I have expressed the width of the CI as a ratio: width = CI_ub / CI_lb,
this naturally follows from the CIs first being calculated on the log scale before
they are exponentiated.

Options

Main

level(), probwidth(), width(), n(), n1(), n2(), nratio(), parallel, notitle; see [PSS-3] ciwidth.

probwidth(), width(), halfwidth(), lbgt(), ublt() ... must specify exactly one of these

p2(), trueeffectsize() ... must specify exactly one of these

Remarks

ciwidth proportions_mc begins by creating a dataset where each row represents a possible study result,
e.g. row 1: 0 successes in the control group and 0 successes in the experimental group.
N.B. This creates a dataset of size 1001 x 1001 = 1,002,001 if N1 = N2 = 1,000. Results could take some time for large studies.

The probability of each study result occurring is then calculated (assuming p1, N1, p2, N2).

The methods described in cs are used to construct CIs. For the odds ratio, the simple Woolf method (rather than the more involved default Cornfield method) is used.
For rows where there are 0 successes in one group, no CI for RR can be constructed.
For rows where there are 0 successes or 0 failures in one group, no CI for OR can be constructed.
Only when a CI is calculated, does ciwidth proportions_mc compute CI width, probability of CI width, or power.
The probability that no CI is calculated is very often negligible, but it can be returned.

The dataset of all possible study results can be left in memory using clear.

Examples

Background. One possible study result - to show how definition of halfwidth and width vary according to RD/RR/OR

. csi 20 40 80 60, or woolf

The observed proportions were 0.4 (40/100) in the control group and 0.2 (20/100) in the experimental group. Therefore,

RD: Risk Difference = -0.2 (-0.32 to -0.08) -> width = CI_ub - CI_lb = 0.24. Note: 95% CI could be expressed as -0.2 +- 0.12

RR: Relative Risk = 0.5 (0.32 to 0.79) -> width = CI_ub / CI_lb = 2.51. Note: 95% CI could be expressed as 0.5 ×/ 1.58

OR: Odds ratio = 0.375 (0.20 to 0.71) -> width = CI_ub / CI_lb = 3.54. Note: 95% CI could be expressed as 0.375 ×/ 1.88

Examples A: Computing CI width

Describe the width of a two-sided 95% CI that will arise from a
study comparing proportions between two equally-sized groups (total
sample size 200), assuming the underlying true proportions
are 0.4 in the control group and 0.2 in the experimental group.

For the risk difference,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) probwidth(0.025 0.25 0.5 0.75 0.975)

For the relative risk,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) probwidth(0.025 0.25 0.5 0.75 0.975) effecttype(2)

For the odds ratio,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) probwidth(0.025 0.25 0.5 0.75 0.975) effecttype(3)

All possible study results can be left in memory using the option clear

. ciwidth proportions_mc, n1(134) n2(134) p1(0.8) p2(0.6) probwidth(0.25 0.5 0.75) clear

Here is the study result in Background:

. list if c1 == 40 & c2 == 20, abbreviate(20)

To reproduce mean halfwidth of 0.11 in example on p92 https://www.power-analysis.com/pdfs/power_precision_manual.pdf

. summarize rd_halfwidth [aweight = probc2c1], detail

Examples B: Computing probability of CI width

Similar to Examples A. halfwidth() is specified below, but it is fine to instead specify width()

For the risk difference,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) halfwidth(0.125)

For the relative risk,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) halfwidth(2) effecttype(2)

For the odds ratio,

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) halfwidth(2) effecttype(3)

Example C: How CI width varies with N (graph)

The (median) CI width may not always decrease as sample size increases,
as we see in this example, because of the discrete nature of the
binomial distribution

. ciwidth proportions_mc, n(26 (2) 32) p1(0.2) p2(0.5) probwidth(0.5) graph(xsize(10))

Examples D: How CI width varies with underlying true proportions

Holding constant p1, and varying the true effect size:

. ciwidth proportions_mc, n(200) p1(0.4) effecttype(1) trueeffectsize(-0.2 (0.1) 0) probwidth(0.5)

. ciwidth proportions_mc, n(200) p1(0.4) effecttype(2) trueeffectsize(0.5 (0.25) 1) probwidth(0.5)

. ciwidth proportions_mc, n(200) p1(0.4) effecttype(3) trueeffectsize(0.375 0.65 1) probwidth(0.5)

Varying p1, and holding constant the true effect size:

. ciwidth proportions_mc, n(200) p1(0.25 0.4 0.6) effecttype(1) trueeffectsize(-0.2) probwidth(0.5)

. ciwidth proportions_mc, n(200) p1(0.25 0.4 0.6) effecttype(2) trueeffectsize(0.5) probwidth(0.5)

. ciwidth proportions_mc, n(200) p1(0.25 0.4 0.6) effecttype(3) trueeffectsize(0.375) probwidth(0.5)

Example E: Probability no CI can be calculated

By default, this information is not displayed. It has to be asked for.
Calculations of width etc. are made based only on study results where a CI can be calculated

. ciwidth proportions_mc, n(20) p1(0.5) p2(0.5) probwidth(0.5) effecttype(1 2 3) table(Pr_noCI, add)

Examples F: How observed ciwidth relates to observed effect size (not true for means)

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) probwidth(0.5) effecttype(1) clear

. collapse (sum) probc2c1, by(rd_width rd)

. scatter rd_width rd [aweight = probc2c1]

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) probwidth(0.5) effecttype(2) clear

. collapse (sum) probc2c1, by(logrr_width logrr)

. scatter logrr_width logrr [aweight = probc2c1], msize(vsmall)

The next examples are CI-based power analysis. By default, power is not displayed.
It has to be asked for using the option table(). true_ES, halfwidth & width are no longer relevant as the program is
focussing on the upper or lower bound of the CI now in relation to a cutpoint,
and not the difference between the two bounds.

Examples G [Power]: Non-inferiority CI-based power calculation

Answers should be similar to those given by the user-written program ssi,
which uses a normal approximation. ciwidth proportions_mc is flexible
enough for power to be calculated in situations other than p1=p2.

. ciwidth proportions_mc, n(400) p1(0.15) p2(0.15) lbgt(-0.1) table(level N N1 N2 p1 p2 true_ES ES_type lbgt power)

. ciwidth proportions_mc, n(400) p1(0.15) p2(0.15) ublt(0.1) table(level N N1 N2 p1 p2 true_ES ES_type ublt power)

. ssc install ssi

. ssi 0.15 0.1, alpha(0.025) n(200) noninferiority

Examples H [Power]: Usual (superiority) CI-based power calculation

Answers should be similar to those given by power twoproportions,
which uses a complicated equation

. ciwidth proportions_mc, n(200) p1(0.2) p2(0.4) effecttype(1) lbgt(0.000001) table(lbgt power, add)

. ciwidth proportions_mc, n(200) p1(0.4) p2(0.2) effecttype(1) ublt(0.000001) table(ublt power, add)

. power twoproportions 0.4 0.2, n(200) test(chi2)

Stored results

ciwidth proportions_mc stores the following in r():

Scalars

r(level) confidence level

r(alpha) significance level

r(onesided) 0 two-sided CI

r(N) total sample size

r(N1) sample size of the control group

r(N2) sample size of the experimental group

r(nratio) ratio of sample sizes, N2/N1

r(p1) proportion in the control group

r(p2) proportion in the experimental group

r(true_ES) numerical RD/RR/OR difference between p2 and p1

r(ES_type) effecttype 1=RD, 2=RR, 3=OR

r(Pr_width) probability of CI width

r(width) CI width

r(halfwidth) CI halfwidth

r(Pr_noCI) probability of no CI calculated

r(lbgt) CI_lb > specified value, if lbgt() specified)

r(ublt) CI_ub < specified value, if ublt() specified

r(power) power, if lbgt() or ublt() specified

r(separator) number of lines between separator lines in the
table

r(divider) 1 if divider is requested in the table,
0 otherwise

Macros

r(type) ci

r(method) proportions_mc

r(columns) displayed table columns

r(labels) table column labels

r(widths) table column widths

r(formats) table column formats

Matrices

r(pss_table) table of results

Author

Mark Chatfield, The University of Queensland, Australia.

m.chatfield@uq.edu.au