help mseffect

Introduction

mseffect -- Estimate the mean effect size of (binary/multiple group) treatment on multiple outcomes.

This command is a part of the online appendix for Lavy et.al (2016) “Empowering Mothers and Enhancing Early Childhood Investment: Effect on Adults Outcomes and Children Cognitive and Non-Cognitive Skills”. It is designed to calculate the mean effect size on multiple outcome variables (summary index) with the advantage that we account for different weights, reversibility of outcome sign, multiple treatment groups and different types of robust standard errors.
The command can estimate the effect by taking account covariance of treatment effects using a seemingly uncorrelated regression; or direct estimate the effect using a linear regression with missing data imputation (at group means).

Cite this command

This command is not the official Stata command. It is issued as a part of online appendix of Lavy et. al (2016). If you would like to use our command for the research, please cite it as such:

Lavy, Victor, Giulia Lotti, and Zizhong Yan. “Empowering Mothers and Enhancing Early Childhood Investment: Effect on Adults Outcomes and Children Cognitive and Non-Cognitive Skills”. National Bureau of Economic Research, (2016): No. w22963.

Coded and maintained by

Zizhong Yan
Department of Economics, University of Warwick

Email helloyzz@gmail.com if you observe any problems.

Description

When we estimate the average treatment effects on multiple outcomes,
one may want to use a single statistic to present an aggregate measure of treatment effects.
However, simply averaging the estimators for the treatment effect is not likely to produce a meaningful statistic
since different outcomes may have different data scales and outcomes can be related to each other.
To address this concern, we follow the summary-index approach as in Kling, Liebman and Katz (2007).
The summary index is a special case of the z-score and is identical to the mean effect size of treatment if there is no missing value.
This approach yields a single standardized normal estimator which indicates an aggregate impact of treatment on a class of outcomes.

In the regression specification (with or without covariates), the mean size effect can be acquired through a linear regression without considering the covariance of effects.
Alternatively, we can consider the covariance strucure and therefore adapt a seemingly uncorrelated regression (O'Brien, 1984; Kling, Liebman and Katz, 2007)
(For more details on the implementation, please refer to Lavy, Lotti and Yan (2016) and Kling, Liebman and Katz (2007))

Syntax

mseffect
outcome1 outcome2 ...
[if] [in] [weight] ,
treat(treatment)
controls(varlist)
reverse(outcomes)
nosur
vce(vcetype)
cluster(varname)
detail

iweights and pweights are allowed;
see weight.

treat(treatment) is required and asks users to specify the binary treatment variable(s).

controls(varlist) allows to add control variables (do not include the treatment variable here)

reverse(outcomes) allows to reverse signs of specific outcomes when calculating the mean effect size

nosur is optional to fit the linear regression with missing data imputation (at group means). As a default, the command uses a seemingly uncorrelated regression model.

vce(vcetype) vcetype
may be oim, robust, or opg

cluster(varname) adjusts
standard errors for intragroup correlation; implies vce(robust)

detail displays more detailed output (e.g. seemingly uncorrelated regressions and effect size formulas) for the diagnostic purpose

Remarks

The treatment has to be binary. In the presence of multiple treatment groups (e.g. treatment intensity), one could input multiple treatment dummies in the treat option

Please only specify the treatment variable(s) using the treat option. Do not put it in the parentheses for control variables.

If the nosur option is not specified, the command, by default, uses a seemingly uncorrelated regression model to consider the covariance of treatment effects.

The white type robust standard errors should be specified using vce(robust) option.

When sampling weights or robust standard errors are used, mseffect uses ml and mysureg to fit a seemingly uncorrelated regression model. The command installs these packages from Stata-press website automatically the first time of use. Or visit http://www.stata-press.com/data/ml3.html for manual installation.

Examples 1: single treatment group

We use an artificial dataset to illustrate the usage of this command. The data generating process of the artificial data can be found at: http://www2.warwick.ac.uk/fac/soc/economics/staff/zyan/mseffect_DGP.txt

Setup

. webuse set http://www2.warwick.ac.uk/fac/soc/economics/staff/zyan

. webuse Summary_index

Estimate the mean effect of Treatment on Work Fulltime Hours and Ave_income

. mseffect Work Fulltime Hours Ave_income , treat(Treatment) controls( )

And with control variables x1 x2 and x3

. mseffect Work Fulltime Hours Ave_income , treat(Treatment) controls(x1 x2 x3)

And reverse the signs of the outcomes Fulltime and Ave_income

. mseffect Work Fulltime Hours Ave_income , treat(Treatment) controls(x1 x2 x3) reverse(Fulltime Ave_income)

And with the white standard errors

. mseffect Work Fulltime Hours Ave_income , treat(Treatment) controls(x1 x2 x3) reverse(Fulltime Ave_income) vce(robust)

And with the population weight weight_eb

. mseffect Work Fulltime Hours Ave_income [pweight=weight_eb], treat(Treatment) controls(x1 x2 x3) reverse(Fulltime Ave_income) vce(robust)

And show more detailed output

. mseffect Work Fulltime Hours Ave_income [pweight=weight_eb], treat(Treatment) controls(x1 x2 x3) reverse(Fulltime Ave_income) vce(robust) details

Examples 2: three treatment groups

Setup

. webuse set http://www2.warwick.ac.uk/fac/soc/economics/staff/zyan

. webuse Summary_index

Estimate the mean effect of Treat1, Treat2 and Treat3 on Work Fulltime Hours and Ave_income with control variables x1 x2 and x3

. mseffect Work Fulltime Hours Ave_income , treat(Treat1 Treat2 Treat3) controls(x1 x2 x3)

Saved results

mseffect saves the following in r():

r(N) number of observations

r(beta) estimated mean effect size on the summary index

r(variance) estimated variance of the mean effect size

r(stderr) estimated standard error of the mean effect size

r(up95) upper bound of the 95% confidence interval

r(low95) lower bound of the 95% confidence interval

r(p_value) p-value

r(sig_level) asterisks for the level of statistic significance (* p < 0.1, ** p < 0.05, *** p < 0.01.)

Suffixes "1, 2,...,N" on the returns indicate the returns of first, second,..., Nth treatment group respectively.

Also see

Online: help for sureg, ml and mysureg if installed.

Bibliography and Sources

Angrist, Joshua D., and Jorn-Steffen Pischke. Mostly harmless econometrics: An empiricist's companion. Princeton university press, (2008).

Gould, William, Jeffrey Pitblado, and William Sribney. "Maximum likelihood estimation with Stata." Stata Press (2006).

Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz. "Experimental analysis of neighborhood effects." Econometrica 75.1 (2007): 83-119.

Lavy, Victor, Giulia Lotti, and Zizhong Yan. “Empowering Mothers and Enhancing Early Childhood Investment: Effect on Adults Outcomes and Children Cognitive and Non-Cognitive Skills”. National Bureau of Economic Research, (2016): No. w22963.

O'Brien, Peter C. "Procedures for comparing samples with multiple endpoints." Biometrics (1984): 1079-1087.