-------------------------------------------------------------------------------help metapowalso see:metasim,metapowplot-------------------------------------------------------------------------------

Title

metapow-- Tool to estimate power through simulation of a 2-arm clinical trial(s) or diagnostic test accuracy study(ies) to change inferences when updating an existing meta-analysis including the new study(ies)

Syntax

metapowvarlist,n(#)nit(#)type(string)pow(numlist)[options]

optionsDescription ------------------------------------------------------------------------- Mainn(#)number of patients in group 1 of the new study (see below)nit(#)number of simulations the power calculation is based ontype(string)type of study being simulatedpow(numlist)cut-off value for determining powerOptional

measure(string)outcome measure used in meta-analysisinference(string)inference statistic on which power is basedp(#)event rate or probability of being diseased dependent ontyper(#)ratio of patients in two groups; treatment and control or diseased and healthystudies(#)number of new studies to be simulatedmodel(string)meta-analysis model used on pre-existing datanpow(numlist)additional inference statistic on which power is basedci(#)width of confidence interval for power estimate (default=95%)dist(string)distribution of effect sizes used to simulate the new study fromindcalculates power for new study on its ownnip(#)number of integration points used for quadrature in bivariate modelsos(string)inference option for sensitivity and specificity to be used withciwidthorlcilevel(cilevel)specifies the confidence level for the study confidence intervals -------------------------------------------------------------------------

Description

metapowimplements an approach to estimating the power of a newly simulated study(ies), generated using the program metasim. It can also estimate the power of the updated meta-analysis including the new study. The power estimates are then stored in a file called temppow2 within the working directory.

Options+------+ ----+ Main +-------------------------------------------------------------

n(integer)relates to the number of patients in the new study. If simulating a new clinical trial, then this specifies the number of patients in the control group. If simulating a new diagnostic accuracy study thennis the number of diseased patients when using sensitivity and specificity as the outcome measure of accuracy, or the number of positive test results if using the diagnostic odds ratio (DOR).

nit(integer)is the number of simulations that are run on which the estimated poweris based. The larger the number specified the more accurate the estimate will be, but the longer the analysis will take.type(clinical/diagnostic)specifies the type of new study that the user would like to simulate; either a 2-arm clinical trial or a diagnostic test accuracy study.

pow(numlist)specifies the value used as a cut-off in determining the power. One or two values can be inputted. The value/s represents different things depending on the option chosen forinference.+----------+ ----+ Optional +---------------------------------------------------------

measure(or/rr/rd/nostandard/dor/ss)specifies the outcome measure used in the meta-analysis to pool the results. The odds ratio (or), relative risk (rr), risk difference (rd) and unstandardised mean difference (nostandard) can only be used when simulating a new clinical study. The diagnostic odds ratio (dor) and sensitivity and specificity (ss) can only be used when simulating a new diagnostic accuracy study. The default for a clinicaltypestudy with 4 variables entered into thevarlistis relative risk (rr, the default for a clinicaltypestudy with 6 variables entered into thevarlistis unstandardised mean difference (nostandardand the default for a diagnostictypestudy is sensitivity and specificity (ss).

inference(ciwidth/pvalue/lci/uci)defines the approach to inference used to calculate power. The default is the confidence interval width (ciwidth). This counts the number of times that the confidence interval width of the estimate/s from the updated meta-analysis (i.e. with the simulated study(ies) included) is less than the specified value/s. This option can be used regardless of the measure of accuracy. Two other approaches to inference available arelcianduci. These will count the number of times that the lower or upper confidence interval is higher or lower than a given value respectively. Thelcioption can be used regardless of the measure of accuracy. However, theucioption is currently only available when working with clinical trial data and not diagnostic data. A final option only available when using clinical trial data is thepvalue. This counts the number of times that a p-value is significant to a specified level. When using sensitivity and specificity two values may be inputted intopowforciwidthandlci. This will instruct the program to count the number of times that the confidence interval widths for both sensitivity and specificity are less than their respective specified values. These must be given in the order sensitivity and specificity to be calculated correctly. In order to use theciwidthorlcioptions for just sensitivity or just specificity thesosoption should be used in addition to this option.

p(real)if simulating a new clinical study then this is the estimated event rate in the control group in the new study. When simulating a new diagnostic study this is the estimated probability of being diseased given a positive result in the new study. When this option is not specified by the user, the program will calculate this value by averaging the probabilities across the studies included in the dataset memory. Note thatpis only relevant in the diagnostic framework when using the diagnostic odds ratio (dor) as the option inmeasure.

r(real)is the ratio of patients in the control group to the treatment group when simulating a new clinical study. When simulating a new diagnostic accuracy study this is the ratio of diseased to healthy people if using sensitivity and specificity and the ratio of positive to negative results if using the DOR (default 1).

studies(integer)specifies the number of new studies to be simulated (default 1). When more than one are specified they are all assumed to have the same sample size.

model(fixed/fixedi/random/randomi/bivariate)defines the type of model used to meta-analyse the pre-existing data. The default is the fixed effect Mantel-Haenszel method (fixed) unless the outcome measure is the nonstandardised mean difference in which case the default is the inverse variance method (fixedi). The (fixedi) option specifies a fixed effect model using the inverse variance method. The (random) option uses the random effect DerSimonian & Laird method, taking the estimate for heterogeneity from the Mantel-Haenszel method. The (randomi) option specifies a random effects model using the method of DerSimonian and Laird, with the estimate of heterogeneity being taken from the inverse-variance fixed-effect model. All of the above options call on themetancommand within the program. The final option is the bivariate random effects model (bivariate). This method calls on a combination of themetandiandmidascommands. It may only be specified when simulating a new diagnostic accuracy study.npow(numlist)recalculates the power using a newly specified value for the sameinferencewithout having to re-run the whole program. Instead, it uses the data that is stored in temppow2 and allows alternative approaches to inference to be explored. This is particularly valuable when the required simulation time is lengthy.

ci(real)specifies the width of the confidence interval for the corresponding power estimate (default 95%).

dist(normal/t)specifies the distribution of effect sizes used to sample a value from in order to simulate a new study(ies). The default for the (random) and (randomi) is a predictive distribution based on the t-distribution (t) allowing for heterogeneity between studies (and the uncertainty in the heterogeneity). The default for all other models is the (normal) distribution based on the mean and variance entered inesandvar.

indinstructs the program to calculate the power for the newly simulated study on its own in addition to the newly updated meta-analysis.

nip(integer)specifies the number of integration points used for quadrature when the bivariate model is selected. Higher values should result in greater accuracy but typically at the expense of longer execution times (see Rabe-Hesketh, Skrondal, and Pickles 2005, app. B).

sos(sens/spec)used in addition to theinferenceoption this specifies whether inferences are focused on sensitivity or specificity when usingciwidthorlcias inference options. The default option is to use sensitivity. Ifsosis not specified then the inferences are based on both the sensitivity and specificity and two values should be entered forpow.

levelspecifies the confidence level, as a percentage, for the individual study and pooled confidence intervals. This is the level that is given in themetan,metandiandmidascommands when called on to meta-analyses the current data set. The default is level(95).

ExamplesFixed effect meta-analysis of clinical trials using the odds ratio, assuming 100 patients in both the control group and the treatment group of the new study and using a confidence interval width cut-off of 2 on the odds ratio scale to calculate power through 100 iterations (i.e. power is based on the proportion of the simulations in which the confidence interval for the pooled OR is less than 2).

. metapow e_trt ne_trt e_ctrl ne_ctrl, model(fixed) measure(or) n(100)nit(100) type(clinical) pow(2) inference(ciwidth)

Random effects meta-analysis of clinical trials using the risk difference, assuming 50 patients in both the control group and the treatment group of the new study and using a p-value of 0.05 for the hypothesis test that the pooled treatment effect is different from 0 to calculate power through 1000 iterations (i.e. power is based on the proportion of simulations in which the p-value for the treatment effect being different from 0 is less then 0.05).

. metapow e_trt ne_trt e_ctrl ne_ctrl, model(random) measure(rd) n(50)nit(1000) type(clinical) pow(0.05) inference(pvalue)

Unstandardised mean difference comparing exercise to no treatment in patients with chronic back pain (Ferreira et al. 2012). We investigate the power of the updated meta-analysis when comparing the lower confidence interval to a reduction in pain of 20 points. We choose a sample size of 250 patients in each arm. We then use

extfunnelto intuitively explain the simulated individual studies on the extended funnel plot (Crowther et al. 2012).

. use http://fmwww.bc.edu/repec/bocode/m/metapow_eg1. metapow nrxpain rxmeanpain rxsdpain ncomppain compmeanpain compsdpain,n(250) nit(1000) type(clinical) measure(nostandard) model(randomi)pow(-20) inference(lci). merge 1:1 _n using "`c(pwd)'\temppow2.dta", nogen noreport. extfunnel paindiff painse, randomi cpoints(500) null(-20)name(painfunnel20, replace) ///> nometan title("PAIN: smallest worthwhile effect = -20", size(medium))xsc(titlegap(medium)) ///> xrange(-100 75) xlabel(-100(25)75)yrange(0 15) scale(1.2)legend(order(1 "Unclear if worthwhile" 2 ///> "Clearly not worthwhile" 3 "Clearly worthwhile" 4 "Smallest worthwhileeffect" 5 "Current pooled estimate" ///> 6 "Current point estimates" 7 "Simulated studies") size(small))scheme(color) ///> addplot(scatter indse_es indes, msize(tiny) msym(T) mcol(black))(click to run)

Random effect meta-analysis of diagnostic test accuracy studies using sensitivity and specificity to base inferences on, assuming 1000 patients in the diseased group and 250 patients in the healthy group of the new study and using 95% confidence interval widths of 0.2 and 0.1 for sensitivity and specificity respectively to calculate power through 1000 iterations (i.e. power is estimated as the proportion of simulations in which the updated meta-analysis estimates sensitivity and specificity with 95% confidence interval widths less than 0.2 and 0.1 respectively).

. metapow TP FP FN TN, model(random) measure(ss) n(1000) nit(1000)type(diagnostic) pow(0.2 0.1) inference(ciwidth) r(0.25)

Bivariate random effect meta-analysis of diagnostic studies using sensitivity and specificity, assuming 80 patients in both the diseased group and the healthy group of the new study and using a lower confidence interval value of 0.25 for sensitivity to calculate power through 50 iterations (i.e. power is estimated as the proportion of simulations in which the updated meta-analysis estimates the lower confidence interval for sensitivity to be less than 0.25).

. metapow TP FP FN TN, model(bivariate) measure(ss) n(80) nit(50)type(bivariate) pow(0.25) inference(lci) sos(sens)

AuthorsMichael J. Crowther, University of Leicester, United Kingdom. Email: michael.crowther@le.ac.uk.

Sally R. Hinchliffe, University of Leicester, United Kingdom. Email: srh20@le.ac.uk.

Alison Donald, University of Leicester, United Kingdom.

Alex J. Sutton, University of Leicester, United Kingdom. Email: ajs22@le.ac.uk.

ReferencesCrowther MJ, Langan D, Sutton AJ. Graphical augmentations to the funnel plot to assess the impact of a new study on an existing meta-analysis.

The Stata Journal2012; (In Press).Hinchliffe S, Crowther MJ, Phillips RS, Sutton AJ. Using meta-analysis to inform the design of subsequent studies of diagnostic test accuracy (Submitted)

Ferreira ML, Herbert RD, Crowther MJ, Verhagen A, Sutton AJ. When is another clinical trial justified? (Submitted)

Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, Abrams KR. Evidence-based sample size calculations based upon meta-analysis.

Statistics in Medicine2007; 26:2479-2500.

Also seeOnline:

metasim,metapowplot