*! version 1.07 13 Jul 2022

Title

sgpvalue -- Second-Generation P-Values

Syntax

sgpvalue,
estlo(boundlist) esthi(boundlist) nulllo(boundlist) nullhi(boundlist)
[options]

Main
estlo(boundlist) lower bound of interval estimate.

esthi(boundlist) upper bound of interval estimate.

nulllo(boundlist) lower bound of null interval.

nullhi(boundlist) upper bound of null interval.

Display
nowarnings disable showing the warnings about potentially problematic intervals.

nodeltagap disable the display and calculations of the delta-gap.

noshow do not show the outcome of the calculations.

Larger input
nomata do not use Mata for calculating the SGPVs if esthi() and estlo() are variables as inputs or if c(matsize) is smaller than the size of these options.

replace replace existing variables in case the nomata-option was used.

Further options
infcorrection(#) a small number to denote a positive but infinitesimally small SGPV.

Description

Compute the second-generation p-value (SGPV) and its associated delta-gaps, as introduced in Blume et al. (2018).
See the formulas at the end for a more detailed description.
This command and its companions commands (sgpower, fdrisk, plotsgpv) are based on the R-code for the sgpv-package from https://github.com/weltybiostat/sgpv.
The command sgpv also exists which makes the computation after common estimation commands easier.
The sgpvalue command should be used mostly for individual SGPV calculations.
A dialog box exists to make using this command easier.

Options

Main

estlo(boundlist) lower bound of interval estimate. Values may be finite or infinite.
To specify that the lower limit is -infinity, just specify the missing value . in this option.
The number of lower bounds must match the number of upper bounds specified in option esthi.
Typically, the lower bound of a confidence interval can be used.
A variable/matrix containing the lower bounds can be also used, but then a variable/matrix containing the upper bounds must be also used for option esthi.

esthi(boundlist) upper bound of interval estimate. Values may be finite or infinite.
To specify that the upper limit is +infinity, just specify the missing value . in this option.
The number of upper bounds must match the number of lower bounds specified in option estlo.
Typically, the upper bound of a confidence interval is used.
A variable/matrix containing the upper bounds can be also used, but then a variable/matrix containing the lower bounds must be also used for option estlo.

nulllo(boundlist) lower bound of null interval. Values may be finite or infinite.
To specify that the lower limit is -infinity, just specify the missing value . in this option.
If multiple bounds are used then the number of lower bounds must match the number of lower bounds specified in option estlo.
The number of lower bounds must always match the number of upper bounds in option nullhi.

nullhi(boundlist) upper bound of null interval. Values may be finite or infinite.
To specify that the upper limit is +infinity, just specify the missing value . in this option.
If multiple bounds are used, the number of upper bounds must match the number of upper bounds specified in option esthi.
The number of lower bounds must always match the number of upper bounds in option nulllo.

A boundlist is:
# [# ...]
exp [exp ...]

Display

nowarnings disable showing the warnings about potentially problematic intervals.

nodeltagap disable the display of the delta-gap. Mainly used inside of sgpv, since delta-gaps are less useful to most users of p-values.

noshow do not show the outcome of the calculations. Useful for larger calculations.

Larger input

nomata deactivate the usage of Mata for calculating the SGPVs with large matrices or variables.
If this option is set, an approach based on variables is used.
Using variables instead of Mata will be faster, but new variables containing the results are created.
The results will saved in one variable named "pdelta" for the SGPVs and one variable named "dg" for the delta gaps.
If you don't want to create new variables and time is not an issue then don't set this option.
Stata might become unresponsive when using Mata because it takes time to return a large matrix.

replace replace existing variables in case the nomata-option was used.

Further options

infcorrection(#) a small number to denote a positive but infinitesimally small SGPV. Default is 1e-5. SGPVs that are infinitesimally close to 1 are assigned 1-infcorrection.
This option has an effect only if one of the intervals has infinite length.

Examples

The following examples are based on the original documentation for the R-code, but are modified to resemble more closely the usual Stata conventions.

Simple example for three estimated log odds ratios but the same null interval (To run this example copy the following lines into Stata and hit return.)

. sgpvalue, estlo(log(1.05) log(1.3) log(0.97)) esthi(log(1.8) log(1.8) log(1.02)) nulllo(log(1/1.1)) nullhi(log(1.1))

One sided intervals/infinite interval bounds:

. sgpvalue, estlo(log(1.3)) esthi(.) nulllo(.) nullhi(log(1.1))

. sgpvalue, estlo(log(1.05)) esthi(.) nulllo(.) nullhi(log(1.1))

Example t-test with simulated data: (To run this example copy the following lines into Stata and hit return.)

. set seed 1776

. qui set obs 15

. qui gen x1 = rnormal(0,2)

. qui gen x2 = rnormal(3,2)

. qui ttest x1==x2

. local ci1 = (`r(mu_1)'-`r(mu_2)')- `r(se)'*invt(`=_N-2',0.975)

. local ci2 = (`r(mu_1)'-`r(mu_2)')+ `r(se)'*invt(`=_N-2',0.975)

. sgpvalue, estlo(`ci1') esthi(`ci2') nulllo(-1) nullhi(1)

. set seed 2019

. qui set obs 15

. qui gen x1 = rnormal(0,2)

. qui gen x2 = rnormal(3,2)

. qui ttest x1==x2

. local ci1 = (`r(mu_1)'-`r(mu_2)')- `r(se)'*invt(`=_N-2',0.975)

. local ci2 = (`r(mu_1)'-`r(mu_2)')+ `r(se)'*invt(`=_N-2',0.975)

. sgpvalue, estlo(`ci1') esthi(`ci2') nulllo(-1) nullhi(1)

Simulated two-group dichotomous data for different parameters:

. set seed 1492

. local n1 30

. local n2 30

. local x1 = rbinomial(`n1',0.15)

. local x2 = rbinomial(`n2',0.5)

. local p1 = `x1'/`n1'

. local p2 = `x2'/`n2'

On the difference in proportions for a 95% confidence interval

. local mean = `p1'-`p2'

. local se = 1.96*sqrt((`p1'*(1-`p1')/`n1' + `p2'*(1-`p2')/`n2'))

. sgpvalue, estlo(`mean'-`se') esthi(`mean'+`se') nulllo(-0.2) nullhi(0.2)

On the log odds ratio scale for a 95% confidence interval

. local mean2 = log(`x1'*(`n2'-`x2')/(`x2'*(`n1'-`x1')))

. local se2 = 1.96*sqrt(1/`x1'+1/`x2'+1/(`n1'-`x1')+1/(`n2'-`x2'))

. sgpvalue, estlo(`mean2'-`se2') esthi(`mean2'+`se2') nulllo(log(1/1.5)) nullhi(log(1.5))

A simple more Stata-like example with a point null hypothesis

. sysuse auto, clear

. regress price mpg foreign weight

. mat table = r(table) //Copies the regression results into a new matrix for the next calculations

. mat ub = table[6,1..3] //Copy the upper bounds of the confidence interval into a separate matrix

. mat lb = table[5,1..3] //Copy the lower bounds of the confidence interval into a separate matrix

The numbers for the options could be also copied by hand, we use here directly the matrices.

. sgpvalue, esthi(ub) estlo(lb) nullhi(0) nulllo(0)

Stored results

Matrices

r(results) matrix with the resulting the SGPVs in the first column and delta-gaps in the second column.

Remarks & Formulas

When the options "nullhi" and "nulllo" have only one value, the same null interval is used for every interval estimate of ["estlo", "esthi"].
If the options "nullhi" and "nulllo" have more than one value, then options "esthi" and "estlo" must have the same number of values.

The SGPV is defined as : p_δ = |I ∩ H_0|/|I|*max{|I|/(2|H_0|), 1}

= |I ∩ H_0|/|I| when |I|<=2|H_0|
= 1/2*|I ∩ H_0|/|H_0| when |I|> 2|H_0|
with I = {θ_l,θ_u} and |I|= θ_u - θ_l.

θ_u and θ_l are typically the upper and lower bound of a (1-α)100% confidence interval but any other interval estimate is also possible.

H_0 is the null hypothesis and |H_0| its length.

The null hypothesis should be an interval which contains all effects that are not scientifically relevant.

The p-values reported by most of Stata's estimation commands are based on the null hypothesis of exactly 0.

Point null-hypothesis are supported by SGPVs, but they are discouraged.
See answer 11 in the Frequently asked questions to Blume et al. (2018).
You could set a small null-hypothesis interval which includes effects of less than 1% or 0.1%. The exact numbers depend on what you deem a priori as not scientifically relevant.

p_δ lies between 0 and 1.

A p_δ of 0 indicates that 0% of the null hypotheses are compatible with the data.

A p_δ of 1 indicates that 100% of the null hypotheses are compatible with the data.

A p_δ between 0 and 1 indicates inconclusive evidence.

A p_δ of 1/2 indicates strictly inconclusive evidence.

For more information about how to interpret the SGPVs and other common questions,
see Frequently Asked Questions by Blume et al. (2018).

The delta-gap is have a way of ranking two studies that both have second-generation p-values of zero (p_δ = 0).
It is defined as the distance between the intervals in δ units with δ being the half-width of the interval null hypothesis.

The delta-gap is calculated as: gap = max(θ_l, H_0l) - min(H_0u, θ_u)
delta = |H_0|/2
delta.gap = gap/delta

For the standard case of a point 0 null hypothesis and a 95% confidence interval, delta is set to be equal to 1.

Then the delta-gap is just the distance between either the upper or the lower bound of the confidence interval and 0.

If both θ_u and θ_l are negative then, the delta-gap is just θ_u, the upper bound of the confidence interval.

If both bounds of the confidence interval are positive, then the delta-gap is equal to the lower bound of the confidence interval.

When possible, one should compute the second-generation p-value on a scale that is symmetric about the null hypothesis.
For example, if the parameter of interest is an odds ratio, computations are typically done on the log scale.
This keeps the magnitude of positive and negative delta-gaps comparable.
Also, recall that the delta-gaps magnitude is not comparable across different null intervals.

References

Blume JD, D’Agostino McGowan L, Dupont WD, Greevy RA Jr. (2018). Second-generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses. PLoS ONE 13(3): e0188299.
https://doi.org/10.1371/journal.pone.0188299

Blume JD, Greevy RA Jr., Welty VF, Smith JR, Dupont WD (2019). An Introduction to Second-generation p-values. The American Statistician. In press.
https://doi.org/10.1080/00031305.2018.1537893

Author

Sven-Kristjan Bormann, School of Economics and Business Administration, University of Tartu.

Bug Reporting

Please submit bugs, comments and suggestions via email to:	sven-kristjan@gmx.de

Further Stata programs and development versions can be found under https://github.com/skbormann/stata-tools

See Also

Related commands:
plotsgpv, sgpower, fdrisk, sgpv