A simulation-based sensitivity analysis for matching estimators
sensatt outcome treatment [varlist] [weight] [if exp] [in range] [ , alg(att*) p(varname) p11(#) p10(#) p01(#) p00(#) reps(#) ycent(#) se(se_type) pscore(scorevar) logit index comsup bootstrap ]
fweights, iweights, and pweights are allowed; see help weights.
sensatt makes use of the commands for the propensity-score matching estimation of average treatment effects written by Becker and Ichino (2002): attnd, attnw, attk, attr. Before using sensatt, you should install them and be familiar with their utilization. Download: sj 5-3 st0026_2
Description
sensatt implements the sensitivity analysis for matching estimators proposed by Ichino, Mealli and Nannicini (2006). The analysis builds on Rosenbaum and Rubin (1983a) and Rosenbaum (1987), and simulates a potential binary confounder in order to assess the robustness of the estimated treatment effects with respect to specific deviations from the Conditional Independence Assumption (CIA).
As a first step, the Average Treatment effect on the Treated (ATT) is estimated by using one of the following propensity-score matching estimators: Nearest Neighbor (attnd, attnw); Radius (attr); Kernel (attk). The options that are common to these commands specify how the baseline ATT is estimated (see Becker and Ichino, 2002).
As a second step, a potential binary confounder (U) is simulated in the data, on the basis of four parameters: pij (with i,j=0,1). Defining Y as the outcome (or as a binary transformation of the outcome in the case of continuous outcomes) and T as the binary treatment, each simulation parameter pij represents the probability that U=1 if T=i and Y=j. The user can either specify the four parameters (with the options: p11(#) p10(#), p01(#), p00(#)) or let the distribution of U mimic the distribution of some relevant covariate (with the option: p(varname)).
Finally, U is considered as any other covariate and is included in the set of matching variables used to estimate the propensity score and the ATT. The imputation of U and the ATT estimation are replicated many times (according to the option: reps(#)), and a simulated ATT is retrieved as an average of the ATTs over the distribution of U. This estimate is robust to the specific failure of the CIA implied by the parameters pij. A comparison of the simulated ATT and the baseline ATT tells us to what extent the latter is robust, with respect to the specific deviation from the CIA that we are assuming.
In order to further emphasize the characteristics of the failure of the CIA implied by the simulated confounder (i.e., by the chosen pij), the estimated effect of U on the selection into treatment - selection effect - and the estimated effect of U on the outcome of untreated subjects - outcome effect - are also reported as odds ratios.
Remarks
The treatment must be binary.
When the outcome is not binary, the potential confounder U is simulated on the basis of a binary transformation of the outcome: Y=1 if the outcome is above the mean, the 25, the 50 or the 75 centile, according to the choice of the user (see option ycent). The ATT, of course, is still estimated for the continuous outcome.
Options that are specific to sensatt
alg(att*) specifies the name of the command (i.e., of the matching algorithm) that is used in the ATT estimation. One of the following commands can be specified: attnd, attnw, attk, attr. The default is attnd.
p(varname) indicates the binary variable which is used to simulate the confounder. The parameters pij used to simulate U are set equal to the ones observed for varname. Instead of selecting this option, the user can directly specify the parameters pij.
p11(#), p10(#), p01(#) and p00(#) jointly specify the parameters pij used to simulate U in the data. Since they are probabilities, they must be between zero and one. For each parameter, the default is zero.
reps(#) specifies the number of iterations, i.e., how many times the simulation of U and the ATT estimation are replicated. The default is 1,000.
ycent(#) is relevant only with continuous outcomes. It means that U is simulated on the basis of the binary transformation of the outcome: Y=1 if the continuous outcome is above the # centile. Three centiles are allowed: 25, 50, 75. If ycent(#) is not selected but the outcome is continuous, U is simulated on the basis of the transformation: Y=1 if the outcome is above the mean.
se(se_type) allows the user to decide which standard error should be displayed with the simulated ATT. Three se_types are possible: set uses the total variance in a multiple-imputation setting; sew uses the within-imputation variance; seb uses the between-imputation variance. The default is set.
Options that are common to attnd, attnw, attk and attr
pscore(varname) specifies the name of the user-provided variable for the estimated propensity score. If no name is provided, the propensity score is estimated on the basis of the specification in varlist.
logit uses a logit model to estimate the propensity score instead of the default probit model when the option pscore(varname) is not specified by the user.
index requires the use of the linear index as the propensity score when the option pscore(varname) is not specified by the user. Otherwise no effect is produced.
bootstrap bootstraps the standard error of the estimated ATTs.
comsup restricts the computation of the ATT to the region of common support.
Saved results
The program stores the simulated treatment effect in r(att).
Standard errors are stored in: r(se) (multiple-imputation); r(sew) (within-imputation); r(seb) (between-imputation).
The estimated outcome effect and selection effect of the simulated confounder are respectively stored in r(yodds) and r(todds).
Examples
. sensatt emp training male age yschool, p(male) or . sensatt emp training male age yschool, p(male) alg(attk) se(seb) boot . sensatt emp training male age yschool, p11(0.6) p10(0.5) p01(0.5) p00(0.2) . sensatt emp training male age yschool, p11(0.6) p10(0.5) p01(0.5) p00(0.2) lo > git alg(attr) r(100) comsup boot
Author
Tommaso Nannicini (Universidad Carlos III de Madrid) Email tnannici@eco.uc3m.es if you observe any problem.
Also see
Help for pscore, attnd, attnw, attk, attr.
Analytical formulas and further details on the sensitivity analysis implemented by this program can be found in: Ichino, A., Mealli, F. and Nannicini, T. (2006), "From Temporary Help Jobs to Permanent Employment: What Can We Learn from Matching Estimators and their Sensitivity?", IZA DP No. 2149.
Useful references
Becker, S.O. and Ichino, A. (2002), "Estimation of average treatment effects based on Propensity Scores", The Stata Journal, 2, 4, 358-377. Dehejia, R.H and Wahba, S. (1999), "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programmes", Journal of the American Statistical Association, 94, 1053-1062. Dehejia, R.H. and Wahba, S.(2002), "Propensity Score Matching Methods for Non-Experimental Causal Studies", Review of Economics and Statistics, 84(1), 151-161. Heckman, J.J., Ichimura, H. and Todd, P.E. (1997), "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme", Review of Economic Studies, 64, 605-654. Heckman, J.J., Ichimura, H. and Todd, P.E. (1998), "Matching as an Econometric Evaluation Estimator", Review of Economic Studies, 65, 261-294. Ichino, A., Mealli, F. and Nannicini, T. (2006), "From Temporary Help Jobs to Permanent Employment: What Can We Learn from Matching Estimators and their Sensitivity?", IZA DP No. 2149. Imbens, G.W. (2003), "Sensitivity to Exogeneity Assumptions in Program Evaluation", AEA Papers and Proceedings, 93, 2, 126-132. Imbens, G.W. (2004), "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review", The Review of Economics and Statistics, 86, 1, 4-29. Rosenbaum, P.R. (1987), "Sensitivity analysis to certain permutation inferences in matched observational studies", Biometrika, 74, 1, 13-26. Rosenbaum, P.R. and Rubin, D.B. (1983a), "Assessing Sensitivity to an Unobserved Binary Covariate in an Observational Study with Binary Outcome", Journal of the Royal Statistical Society, B, 45, 212-218. Rosenbaum, P.R. and Rubin, D.B. (1983b), "The Central Role of the Propensity Score in Observational Studies for Causal Effects", Biometrika, 70(1), 41-55. Smith, J., and Todd, P.E. (2005), "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?", Journal of Econometrics, 125, 305-353.