help rdesigni

Title

rdesigni -- Design analysis

Syntax

rdesigni
D
se
[ , options ]

where D is the true effect size and se is the standard error of
the estimate, and both, D and se, may be specified as # or
(numlist).

alpha(numlist) significance level; default is
1-/100

df(numlist) degrees of freedom for t distribution;
default is df(.), resulting in the standard normal
distribution

reps(#) number of random draws; default is
reps(10000)

parallel process numlists parallel; default
is to process all possible combinations

Description

rdesigni implements the design analysis approach discussed in Gelman
and Carlin (2014). The authors suggest simulating replicated results given
a true effect size and the parameters of a specific study. The command
estimates power, Type S (sign) error rate and Type M (magnitude) error
(exaggeration ratio).

By default, the type M error is found by a series of random draws; to be
able to reproduce results, set the random-number seed
(see set seed).

The r in rdesigni stands for replication, research,
retro(spective), or R, the software used by the original authors.

Also see immed for a general description of
immediate commands.

Options

alpha(numlist) specifies the significance level used in the study. The
default is set to 1-/100, according to c(level).

df(numlist) specifies the degrees of freedom used in the study. The
default is df(.) and means that the standard normal distribution is
used.

reps(#) specifies the number of random draws from the distribution to
simulate replicated results. Default is reps(10000). When reps(0)
is specified, a closed-form expression for the Type M error, suggested by
Lu, Qiu, and Deng (2019), is used.

parallel processes the specified numlists in parallel. Default is
to process all possible combinations of the numbers in numlists. The last
values of shorter numlists are used repeatedly.

Examples

. rdesigni 0.1 3.28

. rdesigni 2 8.2

. rdesigni (0.1 2) (3.28 8.2) , parallel

Saved results

rdesigni saves the following in r():

Scalars

r(reps) number of replications

r(alpha) significance level

r(df) degrees of freedom

r(se) standard error of estimate

r(D) true effect size

r(crit) critical value

r(pr_0) probability wrong sign

r(pr_1) probability correct sign

r(power) power

r(typeS) type S error

r(typeM) type M error

Macros

r(cmd) rdesigni

r(seed) random-number seed

Matrices

r(table) information from the coefficient table

Acknowledgments

Ariel Linden's retrodesign stimulated the implementation of
the closed-form expression for the Type M error and helped identify
a bug that produced wrong results for negative effect sizes.

References

Gelman, Andrew, Carlin, John (2014). Beyond Power Calculations: Assessing
Type S (Sign) and Type M (Magnitude)
Errors. Perspectives on Psychological Science, 9, 641-651.

Lu, Jiannan, Qiu, Yixuan, and Deng, Alex (2019). A note on
Type S/M errors in hypothesis
testing. British Journal of Mathematical and Statistical Psychology,
72, 1-17.

Author

Daniel Klein, University of Kassel, klein.daniel.81@gmail.com

Also see

Online: power

if installed: retrodesign