*! version 1.1 24 Dec 2020

Title

fdrisk -- False Discovery or Confirmation Risk for Second-Generation P-Values (SGPV)

Syntax

fdrisk, nulllo(lower_bound) nullhi(upper_bound) stderr(#) nullspace(# #) altspace(# #)
[fcr level(#) likelihood(#) nulltruncnormal alttruncnormal pi0(#)]

Main
fcr calculate the false confirmation risk instead of the default false discovery risk.

nulllo(lower_bound) the lower bound of the indifference zone (null interval) upon which the second-generation p-value was based.

nullhi(upper_bound) the upper bound of the indifference zone (null interval) upon which the second-generation p-value was based.

stderr(#) standard error of the point estimate.

level(#) confidence interval with the specified level was used to calculated the SGPV; default is level(95).

likelihood(#) likelihood support interval with level 1/k was used to calculate the SGPV.

nullspace(# [#]) support of the null probability distribution.

nulltruncnormal use Truncated-Normal-distribution as the probability distribution for the null parameter space.

altspace(# [#]) support for the alternative probability distribution.

alttruncnormal use Truncated-Normal-distribution as the probability distribution for the alternative parameter space.

pi0(#) prior probability of the null hypothesis.

Description

This command computes the false discovery risk (sometimes called the "empirical bayes FDR") for a second-generation p-value of 0, or the false confirmation risk (FCR) for a second-generation p-value of 1.
The default is to calculate the FDR. The calculation of the FCR needs to be requested by setting the option fcr.
This command should be used mostly for single calculations.
For calculations after estimation commands use the sgpv command.
A dialog box for easier usage of this command is available.

The false discovery risk is defined as: P(H_0|p_δ=0) = (1 + P(p_δ = 0| H_1)/P(p_δ=0|H_0) * r)^(-1)

The false confirmation risk is defined as: P(H_1|p_δ=1) = (1 + P(p_δ = 1| H_0)/P(p_δ=1|H_1) * 1/r )^(-1)

with r = P(H_1)/P(H_0) being the ratio of the prior probabilities for the alternative and null hypothesis
and p_δ being the calculated SGPV.

See equation(4) in Blume et.al.(2018)

When possible, one should compute the second-generation p-value and FDR/FCR on a scale that is symmetric about the null hypothesis.
For example, if the parameter of interest is an odds ratio, inputs "stderr", "nulllo", "nullhi", "nullspace", and "altspace" are typically on the log scale.

If option "nulltruncnormal" is used, then the distribution used is a truncated Normal distribution with mean equal to the midpoint of "nullspace",
and standard deviation equal to "stderr", truncated to the support of "nullspace".
If option "alttruncnormal" is used, then the distribution used is a truncated Normal distribution with mean equal to the midpoint of "altspace",
and standard deviation equal to "stderr", truncated to the support of "altspace".
Further customization of these parameters for the truncated Normal distribution are not possible,
although they may be implemented in future versions.

Options

Main

fcr calculate the false confirmation risk, when the observed second-generation p-value is 1.
The default is to calculate the false discovery risk, when the observed second-generation p-value is 0.

nulllo(lower_bound) the lower bound of the indifference zone (null interval) upon which the second-generation p-value was based.

nullhi(upper_bound) the upper bound of the indifference zone (null interval) upon which the second-generation p-value was based.

stderr(#) standard error of the point estimate.

level(#) confidence interval with level (1-α)100% was used to calculate the SGPV. The default is level(95) if option likelihood or no other confidence level is not set.

likelihood(#) likelihood support interval with level 1/k was used to calculate the SGPV. The level is 1/k (not k).
For technical reasons, a fraction like 1/8 as the level must be converted to a real number like 0.125 first, when used for this option.

nullspace(# [#]) support of the null probability distribution.
The "nullspace" can contain either one or two numbers. These numbers can be also formulas which must enclosed in " ".
If "nullspace" is one number, then no distribution for the null parameter space is used.
If "nullspace" contains two numbers separated by a space, then the distribution for the null parameter space are either the Uniform-distribution or the Truncated-Normal-distribution (option nulltruncnormal).
The Uniform-distribution is used as the default.

nulltruncnormal use the Truncated-Normal-distribution as the probability distribution for the null parameter space with the mean being the middlepoint of the nullspace and standard deviation given by option stderr.

altspace(# [#]) support for the alternative probability distribution.
The "altspace" can contain either one or two numbers. These numbers can be also formulas which must enclosed in " ".
If "altspace" is one number, then no distribution for the alternative parameter space is used.
If "altspace" contains two numbers separated by a space, then the distribution for the alternative parameter space are either the Uniform-distribution or the Truncated-Normal-distribution (option alttruncnormal).
The Uniform-distribution is used as the default.

alttruncnormal use Truncated-Normal-distribution as the probability distribution for the alternative parameter space with the mean being the middlepoint of the altspace and the standard deviation given by option stderr.

pi0(#) prior probability of the null hypothesis. Default is 0.5.
This value can be only between 0 and 1 (exclusive).
A prior probability outside of this interval is not sensible.
The default value assumes that both hypotheses are equally likely.

Examples

To run the examples copy the lines into a Stata or use the file which is part of the ancillary files of the sgpv-package.
To download the file together with the rest of the examples click here.

False discovery risk with 95% confidence level: (Click to run the example.)

. fdrisk, nulllo(log(1/1.1)) nullhi(log(1.1)) stderr(0.8) nullspace(log(1/1.1) log(1.1))
altspace("2-1*invnorm(1-0.05/2)*0.8" "2+1*invnorm(1-0.05/2)*0.8") level(95)

False discovery risk with 1/8 likelihood support level:(Click to run the example.)

. fdrisk, nulllo(log(1/1.1)) nullhi(log(1.1)) stderr(0.8) nullspace(0)
altspace("2-1*invnorm(1-0.041/2)*0.8" "2+1*invnorm(1-0.041/2)*0.8") likelihood(0.125)

with truncated normal weighting distribution:(Click to run the example.)

. fdrisk, nulllo(log(1/1.1)) nullhi(log(1.1)) stderr(0.8) nullspace(0) alttruncnormal
altspace("2-1*invnorm(1-0.041/2)*0.8" "2+1*invnorm(1-0.041/2)*0.8") likelihood(0.125)

False discovery risk with LSI and wider null hypothesis:(Click to run the example.)

. fdrisk, nulllo(log(1/1.5)) nullhi(log(1.5)) stderr(0.8) nullspace(0) altspace("2.5-1*invnorm(1-0.041/2)*0.8" "2.5+1*invnorm(1-0.041/2)*0.8") likelihood(0.125)

False confirmation risk example:(Click to run the example.)

. fdrisk, fcr nulllo(log(1/1.5)) nullhi(log(1.5)) stderr(0.15) nullspace("0.01-1*invnorm(1-0.041/2)*0.15" "0.01+1*invnorm(1-0.041/2)*0.15") altspace(log(1.5) 1.25*log(1.5)) likelihood(0.125)

Stored results

Scalars

r(fdr) false discovery risk

r(fcr) false confirmation risk

References

Blume JD, D’Agostino McGowan L, Dupont WD, Greevy RA Jr. (2018). Second-generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses. PLoS ONE 13(3): e0188299.
https://doi.org/10.1371/journal.pone.0188299

Blume JD, Greevy RA Jr., Welty VF, Smith JR, Dupont WD (2019). An Introduction to Second-generation p-values. The American Statistician. In press. https://doi.org/10.1080/00031305.2018.1537893

Author

Sven-Kristjan Bormann, School of Economics and Business Administration, University of Tartu.

Bug Reporting

Please submit bugs, comments and suggestions via email to: sven-kristjan@gmx.de

Further Stata programs and development versions can be found under https://github.com/skbormann/stata-tools

See Also

Related commands:

plotsgpv, sgpvalue, sgpower, sgpv