help for qqcompare

Evaluating balance after matching using quantile-quantile plots.

qqcompare varlist [,
precision(integer)
treatment(varname)
weight(varname) drawgraph
matchopts(string)
unmatchopts(string)
lineopts(string)
indivoptions(string)
overalloptions(string)]

Description

qqcompare helps evaluate balance after propensity score or
Mahalanobis matching by comparing covariate balance using
quantile-quantile plots. This allows for comparing the entire
distribution of covariates, and not just their means, and thereby
choosing the best matching algorithm among different alternatives
according to which algorithm is most effective in reducing imbalance.
qqcompare reports the mean and maximum deviation in
quantile-quantile plots from the 45 degree line of identical
distribution post-matching, and the relative (percentage) reduction
that results from matching, compared to the unmatched data. Higher relative reductions indicate
better balance. If the drawgraph option is specified,
qqcompare draws quantile-quantile plots for each covariate, before
and after matching.

qqcompare also reports the difference in means as a
proportion of the pooled standard deviation, which, according to
Cochran and Rubin's (1973) rule of thumb, should be less than 0.25.

Imai et al (2008) and Stuart (2010) suggest using these, amongst
other measures to evaluate balance (and explain why t-tests are not
appropriate). While not mandatory, qqcompare is designed to be
used after psmatch2, as detailed below.

cquantile must be installed for qqcompare to work.
Type ssc install cquantile if required.

Basic Syntax

(if used after psmatch2)

. qqcompare varlist

varlist specifies the covariates for which balance is to be
evaluated. qqcompare compares the distribution of each variable
specified in varlist post-matching, between treatment and control
groups.

To also obtain quantile-quantile plots in addition to their
summary measures, specify the drawgraph option.

. qqcompare varlist, drawgraph

Options

precision(integer) specifies the accuracy with which
qqcompare functions. Matching results in a set of weights for
treatment and control units. To construct quantile-quantile plots, we
need to create a new set of data consisting of matched treated and
control units where the frequency of each unit is proportional to its
weight. Since weights are seldom integers, qqcompare works by
multiplying weights by an integer specified by precision and
rounding off to the nearest whole number. The default value for
precision is 10.

So, for example, if the matching results in weights of 1, 0.33
and 0.5, then the default for precision results in an expanded
data set that duplicates respective units 10, 3 and 5 times. Clearly,
the higher the value of precision, the more precise the
quantile-quantile plots and their summary measures. However, since
qqcompare creates a temporary, expanded dataset, higher values of
precision may lead to memory problems on versions of Stata older
than 12, in which case more memory should be allocated beforehand using
set memory.

treatment(varname) need not be specified
if matching was undertaken using psmatch2. Otherwise, this can be
used to specify the treatment variable.

weight(varname) need not be specified if
matching was undertaken using psmatch2. Otherwise, varname
can be used to specify the weights that the matching process resulted
in.

drawgraph is used when the actual quantile-quantile plots
are also required to be drawn (the default is to omit these plots).

The following options can be used to change the default rendering
of quantile-quantile plots. qqcompare constructs a twoway plot for
each variable, consisting of quantiles of treated and control
observations for unmatched and matched data, and in addition, a line to
show identical distributions. These plots are then combined.

matchopts(string) can be used to pass
marker_options and jitter_options to change the rendering
of quantiles for matched units.

unmatchopts(string) can be used to pass
marker_options and jitter_options options to change the
rendering of quantiles for unmatched units.

lineopts(string) can be used to pass line_options
to change the rendering of the line showing identical
distribution.

indivoptions(string) can be used to pass
twoway_options to change the rendering of all quantile-quantile
plots (together).

overalloptions(string) can be used to pass
options for graph combine to change how the individual
quantile-quantile plots are combined.

The results of qqcompare are returned in r(qqcompare).

Example

In the following, collgrad is the binary treatment and the outcome is wage.

. sysuse nlsw88.dta, clear
. psmatch2 collgrad age tenure race married, outcome(wage) n(2)
. qqcompare age tenure, drawgraph
. qqcompare age tenure, drawgraph matchopts(jitter(3) msize(0.3)) overalloptions(scheme(s2manual))

References

Cochran, W. G. and Rubin, D. B. (1973) Controlling bias in observational studies: a review. Sankhya A, 35,
417Ð446.

Imai, K., G. King, and E.A. Stuart. 2008.â Misunderstandings
Between Experimentalists and Observationalists About Causal Inference.
Journal of the Royal Statistical Society: Series A (Statistics in
Society) 171 (2): 481-502.

Stuart, E.A. 2010. Matching Methods for Causal Inference: A
Review and a Look Forward. Statistical Science 25 (1): 1-21.

Author

Sunil Mitra Kumar, King's College London.
stuff.sunil@gmail.com