*! version 1.0.1 26Oct2016

*! version 1.0.0 28May2016

Title

covbal -- Covariate balance statistics

Syntax

covbal treatvar varlist [if] [in]
[, wt(varname) absolute format((%fmt) saving(filename, replace) ]

wt(varname) weight used for calculating weighted statistics

absolute produce absolute standardized differences; default is signed standardized differences

format(%fmt) display format for statistics; default format is %9.0g

saving(filename) specify filename where the statistics, generated by covbal, are to be saved. The suboption "replace" is allowed

by is allowed; see [D] by.

Description

covbal produces a table of distributional test statistics (means, variances, and skewness) for each covariate specified, and assesses balance
between treatment groups in the means (using the standardized difference) and in the variances (using the variance ratio).

Remarks

While there is no empirical evidence to support the use of any particular cutpoint on the standardized difference to define imbalance, Normand et al.(2001)
suggest that a standardized difference greater than 0.10 is indicative of imbalance and Rubin (2001) suggests a cut-off of 0.25. Alternatively, since the standardized
difference is a version of Cohen’s d statistic for effect size (Cohen 1988), one could also argue for a cut-off of 0.20, which Cohen termed a "small" effect. Perhaps
the most helpful advise is to strive to achieve standardized differences as close to zero as possible (Linden & Samuels, 2013).

As balance is not only a property of the sample means of a covariate but of the overall distribution, higher-order sample moments of the distribution should be evaluated as well.
Rubin (2001) proposes the use of the ratio of treated and control variances as a balance measure of the second moment, where balance is defined by values close to 1.0 and
variables are out of balance if the variance ratio is greater than 2.0 or less than 0.5.

Covariate imbalances may occur at any point along the distribution, and may not be captured by comparisons of means and variances (Linden 2015). It is therefore highly recommended
to supplement these numerical measures of balance with graphical displays of the distribution -- in particular using quantile-quantile plots (see qqplot or qqplot3, if installed).

Options

wt(varname) specifies the weight variable used for balancing covariates.

absolute produces the absolute value of the standardized difference; the default is the signed value of the standardized difference.

format(%fmt) specifies the format to be used for all statistics. The maximum width of the specified format should not exceed
nine characters; default format is %9.0g.

saving(filename) specifies the filename where the statistics generated by covbal will be saved. replace, used with
saving, specifies that filename can be replaced if it already exists.

Examples

Load example data

. webuse cattaneo2, clear

Estimate propensity score for mbsmoke as the treatment, and generate inverse probability of treatment weights (IPTW)

. logit mbsmoke mmarried c.mage c.mage#c.mage fbaby i.medu

. predict pscore, pr

. gen iptw = cond(mbsmoke, 1/pscore, 1/(1-pscore))

Run covbal on unweighted data

. covbal mbsmoke mmarried mage fbaby medu

Rerun covbal on weighted data

. covbal mbsmoke mmarried mage fbaby medu, wt(iptw)

Same as above, but reporting the standardized differences as absolute values, and changing the format to %9.3f

. covbal mbsmoke mmarried mage fbaby medu, wt(iptw) abs for(%9.3f)

Reviewing covariate balance by strata of the propensity score

. xtile strata = pscore, nq(5)

. bys strata: covbal mbsmoke mmarried mage fbaby medu if strata !=., abs for(%9.3f)

Saving the output in file named "output" in the default directory

. covbal mbsmoke mmarried mage fbaby medu, wt(iptw) abs for(%9.3f) saving(output)

Open the saved file and review summary statistics for the standardized differences and variance ratios

. use output, clear

. sum stdiff varratio

Stored results

covbal stores the following in r(), which can be displayed by typing return list after covbal is finished (see return):

Scalars

r(varcnt) number of covariates assessed

r(masd) mean of the absolute standardized differences

r(mvr) mean of the variance ratios

Matrices

r(table) balance table

References

Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Hillsdale, NJ: Lawrence Erlbaum.

Flury, B. K. & Reidwyl, H. (1986) Standard distance in univariate and multivariate analysis. The American Statistician, 40, 249–251.

Linden, A. (2015) Graphical displays for assessing covariate balance in matching studies. Journal of Evaluation in Clinical Practice, 21, 242–247.

Linden, A. & Samuels, S. J. (2013) Using balance statistics to determine the optimal number of controls in matching studies. Journal of Evaluation in Clinical Practice,
19, 968–975.

Normand, S. L. T., Landrum, M. B., Guadagnoli, E., Ayanian, J. Z., Ryan, T. J., Cleary, P. D. & McNeil, B. J. (2001) Validating recommendations for coronary angiography
following an acute myocardial infarction in the elderly: a matched analysis using propensity scores. Journal of Clinical Epidemiology, 54, 387–398.

Rubin, D. B. (1973) Matching to remove bias in observational studies. Biometrics, 29, 159–184.

Rubin, D. B. (2001) Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services & Outcomes Research Methodology,
2, 169–188.

Citation of covbal

covbal is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden, Ariel (2016). covbal: Stata module for generating covariate balance statistics. http://ideas.repec.org/c/boc/bocode/s458188.html

Author

Ariel Linden

President, Linden Consulting Group, LLC

Ann Arbor, MI, USA

alinden@lindenconsulting.org

http://www.lindenconsulting.org

Acknowledgments

I wish to thank Nicholas J. Cox for reviewing covbal and making valuable improvements to the code.

Also see

Online: tebalance, tabstat, qqplot, pbalchk (if installed), imbalance (if installed) qqplot3 (if installed)