*! version 1.1.0 2026-04-27

Title

mht_critical -- Compute the optimal per-test significance level for multiple hypothesis testing

Syntax

mht_critical, jhypotheses(#) alphabar(#) [options]

Quick start

Optimal alpha for 5 hypotheses, Linear/FDA calibration, alphabar=0.05:

. mht_critical, jhypotheses(5) alphabar(0.05)

Same with Cobb-Douglas / J-PAL calibration:

. mht_critical, jhypotheses(5) alphabar(0.05) model(cobbdouglas)

Linear model with a larger-than-benchmark sample (less conservative threshold):

. mht_critical, jhypotheses(3) alphabar(0.025) nmratio(1.5)

Description

mht_critical computes the optimal per-test significance level alpha* for
multiple hypothesis testing based on Proposition 4.1 of Viviano, Wuthrich, and
Niehaus (2026):

alpha*(J, n/m) = C(J, n/m) / (b * omega_bar(J))

where C is the research cost function, b is the per-unit benefit of a true
rejection, and omega_bar is the sum of treatment weights. The command
also reports Bonferroni and Sidak critical values for comparison.

Two cost-function calibrations are supported:

Linear model (Equation 26): C = c_f + c_v * |J| * n, calibrated to
clinical-trial data (Sertkaya et al. 2016).

Cobb-Douglas model (Appendix A): C = k * |J|^beta * n^iota, calibrated
to J-PAL field-experiment data.

Options

Required

jhypotheses(#) number of hypotheses |J| (positive integer).

alphabar(#) benchmark single-hypothesis test size, in (0, 1).

Cost model

model(string) linear (default) or cobbdouglas.

Linear model parameters

cfshare(#) fixed cost share c_f / E[C]. Default 0.46.

jbar(#) average number of subgroups. Default 3.

nmratio(#) per-arm-to-benchmark sample size ratio. Default 1.0.

Cobb-Douglas parameters

beta(#) elasticity wrt |J|. Default 0.13.

iota(#) elasticity wrt sample size. Default 0.075.

Examples

Basic usage with Linear calibration:

. mht_critical, jhypotheses(5) alphabar(0.05)

Cobb-Douglas with J-PAL parameters:

. mht_critical, jhypotheses(5) alphabar(0.05) model(cobbdouglas) beta(0.13) iota(0.075)

Linear with a non-benchmark sample size:

. mht_critical, jhypotheses(3) alphabar(0.025) nmratio(1.5)

Loop to reproduce a portion of Table 1:

. forvalues j = 1/9 {

. mht_critical, jhypotheses(`j') alphabar(0.05)

. display "J=`j': alpha_opt = " r(alpha_opt)

. }

Stored results

mht_critical stores the following in r():

Scalars

r(alpha_opt) optimal test size

r(t_star) optimal z-threshold

r(alpha_bonf) Bonferroni test size

r(t_bonf) Bonferroni z-threshold

r(alpha_sidak) Sidak test size

r(t_sidak) Sidak z-threshold

r(alpha_bar) benchmark alpha (input)

r(J) number of hypotheses

r(nm_ratio) sample size ratio used

Macros

r(model) cost model used

References

Viviano, D., K. Wuthrich, and P. Niehaus (2026).
A model of multiple hypothesis testing. arXiv:2104.13367v10.

Sertkaya, A., H.-H. Wong, A. Jessup, and T. Beleche (2016).
Key cost drivers of pharmaceutical clinical trials in the United States.
Clinical Trials 13(2), 117-126.

Also see

Online: mht_test, mht_est, mht_table, mht_cost_estimate