*! version 1.1.0 2026-04-27

Title

mht_test -- Apply optimal multiple-hypothesis testing adjustment to a list of p-values

Syntax

mht_test varname [if] [in], alphabar(#) [options]

where varname is a variable holding a list of p-values (one p-value per
observation, one observation per hypothesis). With option zstat, varname
is interpreted as a list of one-sided z-statistics instead.

Quick start

Apply the optimal MHT correction to 6 hypotheses (Linear/FDA calibration):

. mht_test pval, alphabar(0.05)

Same input given as z-statistics:

. mht_test zstat, alphabar(0.05) zstat

Use the Cobb-Douglas (J-PAL) calibration with custom elasticities:

. mht_test pval, alphabar(0.05) model(cobbdouglas) beta(0.13) iota(0.075)

Save rejection indicators with a custom prefix:

. mht_test pval, alphabar(0.05) generate(myrej)

Description

mht_test applies the optimal multiple-hypothesis-testing adjustment from
Viviano, Wuthrich, and Niehaus (2026) to a collection of p-values stored
in a single variable. Each observation in the active sample contributes one
hypothesis, so the number of hypotheses |J| equals the number of in-sample
observations.

Note on |J|: |J| is set to the number of in-sample observations,
i.e., the number of p-values being tested. If you pass if or in
to subset the data, |J| changes accordingly. Make sure the active sample
contains exactly the hypotheses you want to correct over.

The command compares five rejection rules side by side:

1. Optimal model-based rule (Proposition 4.1, paper):
critical value derived from the cost function.

2. Bonferroni: alphabar/J.

3. Holm step-down procedure.

4. Benjamini-Hochberg step-up FDR control.

5. Unadjusted (alphabar).

Output includes a summary line for each procedure (test size and number of
rejections) and a per-hypothesis table showing which rule rejects which
hypothesis. Five new variables are also generated for downstream use.

Options

Required

alphabar(#) is the benchmark single-hypothesis size alphabar that
would apply if there were only one hypothesis. Standard choices are 0.05 and 0.025.

Input format

zstat indicates that varname contains one-sided z-statistics (not p-values).
The command internally converts them via 1 - normal(z) before applying the
adjustment.

Cost model

model(string) selects the cost function. Either linear (default,
calibrated to FDA Phase III data) or cobbdouglas (calibrated to J-PAL data).

cfshare(#) (Linear only): fixed-cost share c_f / E[C]. Default 0.46
(Sertkaya et al. 2016).

jbar(#) (Linear only): average number of subgroups in the benchmark study.
Default 3.

nmratio(#): ratio n_bar / m_bar of per-arm to benchmark sample size.
Default 1.0.

beta(#) (Cobb-Douglas only): elasticity of cost with respect to |J|.
Default 0.13 (J-PAL).

iota(#) (Cobb-Douglas only): elasticity of cost with respect to per-arm
sample size. Default 0.075 (J-PAL).

Output

generate(string) sets the prefix for the five rejection-indicator variables
created by the command. Default mht, yielding mht_reject_opt,
mht_reject_bonf, etc.

replace replaces existing variables with the chosen prefix.

Examples

Setup: a small set of one-sided p-values

. clear

. set obs 6

. gen pval = .

. replace pval = 0.003 in 1

. replace pval = 0.015 in 2

. replace pval = 0.030 in 3

. replace pval = 0.048 in 4

. replace pval = 0.120 in 5

. replace pval = 0.500 in 6

Apply the Linear (FDA) MHT correction

. mht_test pval, alphabar(0.05)

Cobb-Douglas (J-PAL) calibration

. mht_test pval, alphabar(0.05) model(cobbdouglas) replace

Z-statistic input

. gen z = invnormal(1 - pval)

. mht_test z, alphabar(0.05) zstat generate(zr) replace

Stored results

mht_test stores the following in r():

Scalars

r(alpha_opt) optimal test size alpha*

r(alpha_bonf) Bonferroni test size alphabar/J

r(alpha_bar) benchmark alpha (input)

r(J) number of hypotheses tested

r(n_reject_opt) rejections under the optimal rule

r(n_reject_bonf) rejections under Bonferroni

r(n_reject_holm) rejections under Holm

r(n_reject_bh) rejections under BH/FDR

r(n_reject_unadj) rejections unadjusted

Macros

r(model) cost model used (linear / cobbdouglas)

mht_test also creates the following new variables (with prefix generate, default mht):

prefix_reject_opt - 1 if rejected by the optimal rule

prefix_reject_bonf - 1 if rejected by Bonferroni

prefix_reject_holm - 1 if rejected by Holm

prefix_reject_bh - 1 if rejected by BH/FDR

prefix_reject_unadj - 1 if rejected unadjusted

prefix_alpha_opt - the optimal test size (constant across rows)

References

Viviano, D., K. Wuthrich, and P. Niehaus (2026).
A model of multiple hypothesis testing.
arXiv:2104.13367v10.

Sertkaya, A., et al. (2016). Key cost drivers of pharmaceutical clinical trials in
the United States. Clinical Trials, 13(2), 117-126.

Also see

Online: mht_critical, mht_est, mht_table, mht_cost_estimate