*! version 1.0.0 2026-03-15

Title

mht_est -- Optimal MHT adjustment after any estimation command

Syntax

mht_est,
vars(varlist)
alphabar(#)
[options]

Required
vars(varlist) names of coefficients to test (must match e(b) column names)

alphabar(#) benchmark single-hypothesis significance level

P-value direction
onesided use one-sided p-values, positive direction (default)

twosided use two-sided p-values instead

Cost model
model(string) linear (default) or cobbdouglas

cfshare(#) fixed cost share (Linear); default 0.46

jbar(#) average subgroups (Linear); default 3

nmratio(#) sample size ratio n_bar/m_bar; default 1.0

mbar(#) benchmark per-arm sample size; if given, nm_ratio is computed as (e(N)/J)/mbar (overrides nmratio())

beta(#) arms elasticity (Cobb-Douglas); default 0.13

iota(#) size elasticity (Cobb-Douglas); default 0.075

mht_est reads coefficient estimates and standard errors directly from
e(b) and e(V) of the most recently fitted estimation results.
This means it works after regress, logit, probit, areg,
xtreg, ivregress, poisson, and any other Stata estimation
command -- no need to re-run the model.

Quick start

Apply the optimal MHT correction to three coefficients of the most recent regression:

. regress price mpg weight foreign, robust

. mht_est, vars(mpg weight foreign) alphabar(0.05)

Same regression, Cobb-Douglas (J-PAL) calibration:

. mht_est, vars(mpg weight foreign) alphabar(0.05) model(cobbdouglas)

After a clustered field experiment with multiple treatment arms:

. areg y treat1 treat2 treat3, absorb(strata) cluster(hh_id)

. mht_est, vars(treat1 treat2 treat3) alphabar(0.05)

Two-sided tests (e.g., when there is no a priori sign of the effect):

. mht_est, vars(treat1 treat2) alphabar(0.05) twosided

Description

mht_est is a postestimation command that applies the optimal multiple
hypothesis testing (MHT) adjustment from Proposition 4.1 of Viviano, Wuthrich,
and Niehaus (2026) to the results of the most recently fitted model. It reads
coefficient estimates and standard errors directly from e(b) and
e(V), and therefore works after regress, logit,
probit, ivregress, areg, xtreg, poisson, and
virtually any other Stata estimation command.

mht_est computes rejection decisions under five procedures:

1. Optimal (model-based) -- Proposition 4.1: alpha* = C(J,n) / (b * omega_bar(J))

2. Bonferroni -- alpha_bonf = alpha_bar / J

3. Holm -- step-down procedure

4. Benjamini-Hochberg (BH) -- FDR control

5. Unadjusted -- compare directly to alpha_bar

The number of hypotheses |J| is set automatically to the number of variables
in vars().

Options

Required

vars(varlist) specifies the coefficient names to test. These must match
column names in e(b) exactly, which are typically the variable names
used in the regression. Use ereturn list to see available names.

alphabar(#) is the benchmark single-hypothesis significance level,
typically 0.05 or 0.10.

P-value direction

onesided (default) converts t-statistics to one-sided p-values in the
positive direction: p = p_two/2 for t >= 0, and p = 1 - p_two/2 for t < 0.
This means a variable with a negative coefficient will not be rejected.

twosided uses two-sided p-values directly from the model. Use this
if your application does not have a directional hypothesis.

Cost model

model(string) selects the research cost function:

linear (default) Linear model (Equation 26 in the paper).
Optimal alpha = alpha_bar * [(1 + r/J)/(1 + r) + (nm_ratio - 1)/(1 + r)],
where r = cfshare*jbar/(1-cfshare). Calibrated to Sertkaya et al. (2016).

cobbdouglas Cobb-Douglas model (Appendix A).
Optimal alpha = alpha_bar * J^(beta-1) * nm_ratio^iota. Calibrated to J-PAL
data (Table 2 of the paper).

cfshare(#) is the fixed cost share for the Linear model. Default 0.46
(Sertkaya et al. 2016). Must be in (0, 1).

jbar(#) is the average number of subgroups/arms per trial used in
the linear calibration. Default 3 (Pocock et al. 2002).

nmratio(#) is the ratio of the study's per-arm sample size to the
benchmark (n_bar/m_bar). Default 1.0. A value greater than 1 indicates
a larger-than-benchmark study, warranting a less conservative threshold.

mbar(#) is the benchmark per-arm sample size m_bar. When provided,
mht_est computes nm_ratio automatically as
(e(N) / J) / mbar, where e(N) is the total sample size from the
preceding regression and J is the number of variables in vars().
This option overrides nmratio().

Caveat for clustered/non-RCT designs. e(N)/J is a rough proxy
for "per-arm sample size" that treats the regression's full sample as if
it were evenly distributed across J arms. In a clustered RCT or any
design where treatment-arm sample sizes are unequal, the proxy may differ
from the true per-arm size. If you know the per-arm size externally
(e.g., from study design), set nmratio() = (true per-arm) / mbar
directly and skip mbar().

beta(#) is the elasticity of cost with respect to the number of
treatment arms (Cobb-Douglas model). Default 0.13 (J-PAL estimate). When
beta = 0, the optimal procedure is Bonferroni; when beta = 1, no adjustment
is needed.

iota(#) is the elasticity of cost with respect to sample size
(Cobb-Douglas model). Default 0.075 (J-PAL estimate).

Remarks

Choosing between Linear and Cobb-Douglas.
The linear calibration (default) is appropriate for settings resembling clinical
trials: experiments with defined regulatory approval processes and mixed
fixed/variable costs. The Cobb-Douglas calibration is appropriate for
development economics field experiments, based on J-PAL cost data.

Interpreting the optimal procedure.
Under linear calibration with default parameters, the optimal procedure is
intermediate between Bonferroni and unadjusted testing. For example, with
J = 5 and alpha_bar = 0.05, the optimal threshold is approximately 0.021,
compared to 0.010 (Bonferroni) and 0.050 (unadjusted).

Connection to standard procedures.
Bonferroni is optimal when research costs are entirely fixed (beta = 0 in
Cobb-Douglas, or equivalently high cf_share). No adjustment is optimal when
costs scale proportionally with the number of hypotheses (beta = 1). The Linear
and J-PAL calibrations both imply intermediate adjustment.

Sample size adjustment.
Setting nmratio() > 1 makes the optimal threshold less conservative.
If your study has twice the benchmark sample size (nmratio = 2), the
researcher has already incurred higher costs to produce better-powered tests,
and the threshold should reflect this.

Multiple outcomes.
When testing multiple treatment effects (one treatment, multiple outcomes),
the paper recommends using an index if there is a single decision-maker. Use
mht_est when either (a) multiple distinct treatments are being compared
or (b) the audience is heterogeneous with different preferences over outcomes.

Examples

Setup: load auto data

. sysuse auto, clear

Example 1: OLS after regress

. regress price mpg weight foreign, robust

. mht_est, vars(mpg weight foreign) alphabar(0.05)

Example 2: Cobb-Douglas with J-PAL calibration

. regress price mpg weight foreign, robust

. mht_est, vars(mpg weight foreign) alphabar(0.05) ///

model(cobbdouglas) beta(0.13) iota(0.075)

Example 3: After logit

. logit foreign price mpg weight, robust

. mht_est, vars(price mpg weight) alphabar(0.05) twosided

Example 4: After areg with cluster SE (field experiment)

. * Simulate a field experiment:

. gen treat1 = runiform() > 0.5

. gen treat2 = runiform() > 0.5

. gen cluster_id = ceil(_n/5)

. areg price treat1 treat2 mpg, absorb(rep78) cluster(cluster_id)

. mht_est, vars(treat1 treat2) alphabar(0.05)

Example 5: Using stored results with estimates store

. quietly regress price mpg weight, robust

. estimates store model_a

. mht_est, vars(mpg weight) alphabar(0.05)

. local alpha_opt_a = r(alpha_opt)

. display "Model A optimal alpha: " `alpha_opt_a'

Example 6: Larger study, less conservative threshold (nmratio)

. regress price mpg weight foreign, robust

. * Compare: benchmark sample vs. double sample

. mht_est, vars(mpg weight foreign) alphabar(0.05) nmratio(1.0)

. mht_est, vars(mpg weight foreign) alphabar(0.05) nmratio(2.0)

Example 7: Using estimated cost parameters from mht_cost_estimate

. * First estimate the cost function:

. mht_cost_estimate cost_var arms_var size_var, alphabar(0.05)

. local est_beta = e(beta)

. local est_iota = e(iota)

. * Then apply to regression results:

. regress y treat1 treat2 treat3, cluster(hh_id)

. mht_est, vars(treat1 treat2 treat3) alphabar(0.05) ///

model(cobbdouglas) beta(`est_beta') iota(`est_iota')

Stored results

mht_est stores the following in r():

Scalars

r(alpha_opt) optimal test size (Proposition 4.1)

r(alpha_bonf) Bonferroni test size (alpha_bar / J)

r(alpha_bar) benchmark alpha

r(J) number of hypotheses tested

r(n_reject_opt) number of rejections under optimal procedure

r(n_reject_bonf) number of rejections under Bonferroni

r(n_reject_holm) number of rejections under Holm

r(n_reject_bh) number of rejections under BH/FDR

r(n_reject_unadj) number of rejections without adjustment

Per-variable results (where varname is each name in vars()):

r(coef_varname) coefficient estimate

r(se_varname) standard error

r(t_varname) t-statistic

r(p_varname) p-value used for testing

r(rej_opt_varname) 1 if rejected under optimal, 0 otherwise

r(rej_bonf_varname) 1 if rejected under Bonferroni, 0 otherwise

r(rej_holm_varname) 1 if rejected under Holm, 0 otherwise

r(rej_bh_varname) 1 if rejected under BH, 0 otherwise

Macros

r(model) cost model used (linear or cobbdouglas)

r(vars) variable list tested

r(cmd) estimation command (from e(cmd))

References

Viviano, D., K. Wuthrich, and P. Niehaus (2026).
A model of multiple hypothesis testing.
arXiv:2104.13367v10.

Sertkaya, A., H. H. Wong, A. Jessup, and T. Beleche (2016).
Key cost drivers of pharmaceutical clinical trials in the United States.
Clinical Trials, 13(2), 117-126.

Benjamini, Y. and Y. Hochberg (1995).
Controlling the false discovery rate: a practical and powerful approach
to multiple testing. Journal of the Royal Statistical Society, Series B,
57(1), 289-300.

Holm, S. (1979). A simple sequentially rejective multiple test procedure.
Scandinavian Journal of Statistics, 6(2), 65-70.

Also see

mht_critical -- compute optimal critical values

mht_test -- test a variable of p-values

mht_cost_estimate -- estimate cost function parameters