Test of robustness of specification
checkrob #corevars #testingvars [options] [filename.txt] [noisily] : estimation command
where
#corevars is the number of core variables (to be retained in all regressions).
#testingvars is the number of testing variables (to be systematically included/excluded).
options are additional output options. All return variables [e(name)] can be specified with a space between them. Default (=none).
filename.txt is an option for specifying the name of the output file. Note, that it must have the ".txt" extension. Default (=result.txt). The table file will get the name table_filename.txt. If filename.txt already exists checkrob will end with an error message.
noisily displays the output for each regression. Default (=quietly).
estimation command can be any single equation estimation command and some 2-equation estimation commands where the second equation is fixed (such as heckman or heckprob with a fixed (unaltered) selection equation in all regressions). Note: the order of the explanatory variables is important for checkrob to work properly. First should the core variables be specified and only thereafter the testing variables (see Examples below). A full set of options supported by the estimation command can be applied to each estimation (e.g. weights, cluster, in range, if etc.)
checkrob supports survey settings via the svy: prefix (see also help svyset).
OBS...OBS checkrob only does limited syntax checking. Thus, the user should check the prerun explanation by checkrob of what it is about to do carefully.
Description
checkrob estimates a set of regressions where the dependent variable is regressed (with whatever method is specified in estimation command) on the core variables - which are included in all regressions - and all possible combinations of the testing variables. In total 2^#testingvars regressions are generated. For each regression the coefficients, their standard errors and optional estimation return variables (ereturn list) are recorded in comma separated format in the file named filename.txt. This file can be accessed after checkrob has run with a standard text editor. Further, checkrob produces a second text file table_filename.txt including the following statistics for each core and testing variable: maximum (Max), minimum (Min) and average value (Mean) of the coefficient over all regressions, average standard deviation (AvgSTD), share of regressions where the coefficient is significant (PercSigni), share of regressions where the sign is positive (Perc+), respective negative (Perc-), the average t-value (AvgT) and the number of times the variable was included in the regressions (Obs) (this might differ between core variables (and between testing variables) if a variable is sometimes dropped. For each variable the average over optional estimation return variables in these regression where the variable featured is also reported.
The table will look like this (recorded in comma separated form):
table_filename.txt
Core var Max Min Mean AvgSTD PercSigni Perc+ Perc- AvgT Obs > e(r2) ------------------------------------------------------------------------------- ------- Education 4 > Experience 4 > age 4 >
Testing Max Min Mean AvgSTD PercSigni Perc+ Perc- AvgT Obs > e(r2) ------------------------------------------------------------------------------- ------- Educ.spouse 2 > Children 2 >
checkrob is a useful tool if you want to check the robustness of including/excluding one or more controls in your preferred specification. checkrob was used to produce the robustness results in Barslund et al. (2007). See the command rcheck (if installed) by Kenneth L. Simons for a similar procedure, however rcheck cannot handle multiple equation such as the heckman estimation command.
Examples
. checkrob #corevars #testingvars [options] [filename.txt] [noisily] : estimation command
. checkrob 2 3: reg wage education experience age educspouse children Here education and experience are core variables; age, educspouse and children are testing variables.
. checkrob 1 4: reg wage education experience age educspouse children Here education is the only core variables; experience, age, educspouse and children are testing variables.
. checkrob 1 4: reg wage education experience age educspouse children, cluster(ea)
. checkrob 2 2 output.txt noisily: reg wage education experience age children if children>4, cluster(ea)
Author
Please address comments and suggestions to:
Mikkel Barslund, Department of Economics, University of Copenhagen, Denmark <mikkelbarslund@gmail.com>
References
Barslund, M., Chiconela, J., Rand, J. & Tarp, F., 2007, 'Understanding Victimization: The Case of Mozambique'. World Development Vol. 35, No. 7 (July)
Also see
Online: help for rcheck if installed