Title

mhtexp -- Stata command for the procedure detailed in List, Shaikh, and Xu (2015)

Syntax

mhtexp
varlist
, treatment [options]

Main
treatment(varlist) treatment status variables varlist

subgroup(varname) group identifier variable varname

combo(string) compare "treatmentcontrol" or "pairwise"; default is
combo("treatmentcontrol")

only(name) the numoc*numsub*numpc hypotheses to be tested

exclude(name) the numoc*numsub*numpc hypotheses not to be tested

boostrap(integer) the number of simulated samples to use

Description

mhtexp testing procedure for multiple hypothesis testing that asymptotically controls
familywise error rate and is asymptotically balanced for outcomes specified via varlist

Options

Main

treatment(varlist) user provided variable containing treatment status of the observations; required.

subgroup(varname) user provided variable containing subgroup ids; optional.

combo(string) user provided string to specify the comparison between treatment and control.
combo("pairwise") will compare all pairwise comparisons across treatment and control.
The default is combo("treatmentcontrol"), compares each treatment to the control; optional

only(name) N by 3 matrix specifying which hypothesis to be tested; optional.

exclude(name) N by 3 matrix specifying which hypothesis not to be tested; optional.

The matrix in either case should be defined where in each row, column 1 is the outcome, column 2 is the
subgroup and column 3 is the treatment-control comparison. Where...

1 <= column 1 <= number of outcomes

1 <= column 2 <= number of subgroups

1 <= column 3 <= number of treatment-control comparisons

By default mhtexp will calculate all hypothesis based on the number of outcomes, subgroups and treatments provided by the user
in varlist group(varname) and treatment(varname), respectively. In section 4.4 of List, Shaikh and Xu (2015) simultaniously consider
4 outcome variables, 4 subgroups and 3 treatment conditions, producting a table of 48 hypothesis test. However, there are cases in which you
may only be interested in certain outcome by subgroup by treatment hypothesis. use only or exclude.

bootstrap(integer) the number of simulated samples. the default is 3000, but a larger number is recommended when there are a large number of hypotheses; optional.

Remarks

For detailed information on the procedure, see URL Multiple Hypothesis Testing in Experimental Economics.

If you are running the command for the first time and receive an error message claiming certain functions are not found,
ie nchoosek(), make sure that lmhtexp.mlib exists in your current dir and enter the command

. mata: mata mlib index

Which tells Stata to look in lmhtexp.mlib for mata functions that are required to run the command

Examples

Suppose a data set containing. You can access this dataset at github.com/seidelj/mht "data/data.csv"

outcome variables gave amount amountchange

treatment variables treatment ratio

Setup

. gen amountmat = amount * ratio

. gen groupid = (redcty==1 & red0 == 1) + (redcty==0 & red0 == 1)*2 + (redcty==0 & red0 == 0)*3 + (redcty==1 & red0 == 0)*4

. replace groupid = . if groupid == 0

Example 1: Hypothesis testing with multiple outcomes

. mhtexp gave amount amountmat amountchange, treatment(treatment)

example 2: Hypothesis testing with multiple subgroups

. mhtexp gave, treatment(treatment) subgroup(groupid)

example 3: Hypothesis testing with multiple treatments

. mhtexp amount, treatment(ratio)

Example 4: Hypothesis testing for all pairwise comparisons among the treatment and control groups

. mhtexp amount, treatment(ratio) combo("pairwise")

Example 5: Hypothesis testing with multiple outcomes, subgroups and treatments

. mhtexp gave amount amountmat amountchange, subgroup(groupid) treatment(ratio)

Example 6: Now let's consider example 5, however we are only interested in the first outcome, subgroup and treatment-control comparison
hypothesis

First an N by 3 matrix must be defined. For more on basic mata and matrices see

. mata: onlyHyp = (1,1,1)

Now we have a 1 by 3 matrix named onlyHyp to be passed to only

. mhtexp gave amount amountmat amountchange, subgroup(groupid) treatment(ratio) only(onlyHyp)

Example 7: Lets consider example 5 once more, but this time we are interested in all
but the last outcome, subgroup and treatment hypothesis.

Create another N by 3 matrix. Recall, we have 4 outcomes, 4 subgroups and 3 treatment control comparisons.

. mata: excludeHyp = (4,4,3)

. mhtexp gave amount amountmat amountchange, subgroup(groupid) treatment(ratio) exclude(excludeHyp)