------------------------------------------------------------------------------- help forpstest-------------------------------------------------------------------------------

Covariate imbalance testing and graphing

pstest[varlist][ifexp] [inrange][,treated(varname)bothrawmweight(varname)support(varname)notablelabelonlysignodistgraphhistgraph_options]

Description

pstestcalculates and optionally graphs several measures of the balancing of the variables invarlistbetween two groups (ifvarlistis not specified,pstestwill look for the variables that were specified in the latest call ofpsmatch2or ofpstest). In particular it can be used to gauge comparability in terms ofvarlistbetween:1. Two matched samples (the default).

pstestcan be called directly afterpsmatch2, or it can be fed matching weights via optionmweightto assess the extent of balancing achieved on the two matched samples. A particularly useful way to usepstestis in search of a matching method and set of matching parameters that achieves good balancing;psmatch2can be called repeatedly prefixed byquietlyand the extent of corresponding balancing can each time be displayed by callingpstest.2. Any two samples (option

raw).

pstestcan be called to assess the comparability ofanytwo groups. This may be before performing matching, or completely unrelated to matching purposes. (The groups are in any case referred to as Treated and Controls, but they could be males and females, employed and non-employed etc.).3. Two samples before and after having performed matching (option

both).In this case

pstestcompares the extent of balancing between the two samples before and after having performed matching.For each variable in

varlistit calculates:(a) t-tests for equality of means in the two samples (before and after matching if option

bothis specified). T-tests are based on a regression of the variable on a treatment indicator. Before matching or onrawsamples this is an unweighted regression on the whole sample, after matching the regression is weighted using the matching weight variable _weight or user-given weight variable inmweightand based on the on-support sample;(b) the standardised percentage bias. If option

bothis specified, the standardised percentage bias is shown before and after matching, together with the achieved percentage reduction in abs(bias). The standardised % bias is the % difference of the sample means in the treated and non-treated (full or matched) sub-samples as a percentage of the square root of the average of the sample variances in the treated and non-treated groups (formulae from Rosenbaum and Rubin, 1985).It also calculates overall measures of covariate imbalance:

(a) Pseudo R2 from probit estimation of the conditional treatment probability (propensity score) on all the variables in

varlistonrawsamples, matched samples (default) orbothbefore and after matching. Also displayed are the corresponding P-values of the likelihood-ratio test of the joint insignificance of all the regressors (before and after matching if optionbothis specified);(b) the mean and median bias as summary indicators of the distribution of the abs(bias) (before and after matching if option

bothis specified);Optionally

pstestgraphs the extent of covariate imbalance in terms of standardised percentage differences using dot charts (optiongraph) or histograms (optionhist).One only need type

pstest[, both]directly afterpsmatch2to inspect the extent of covariate balancing in matched samples ifpsmatch2has been called with avarlist.If option

bothis specified,pstestreturns the following diagnostics of covariate balancing before and after matching:r(meanbiasbef)andr(meanbiasaft)the mean absolute standardised bias,r(medbiasbef)andr(medbiasaft)the median absolute standardised bias,r(r2bef)andr(r2aft)the pseudo R2 from probit estimation andr(chiprobbef)andr(chiprobaft)the P-value of the likelihood-ratio test. If the two groups are compared only once (matched samples as default or two unmatched samples if optionrawis specified),pstestreturnsr(meanbias),r(medbias),r(r2)andr(chiprob).pstestalways returns inr(exog)the names of the variables for which it has tested the extent of balancing.

Important notes

pstestonly considers balancing for the treated even if called afterpsmatch2, ate.Spline matching as in

psmatch2, splineas well as the default (tricube) local linear regression matching as inpsmatch2, llrfirst smooth the outcome and then perform nearest neighbor matching.pstestdoes not make sense in these cases since more non-treated are used to calculate the counterfactual outcome than the nearest neighbor only.

Detailed Syntax

Matched samples:

pstest[varlist][ifexp] [inrange][,treated(varname)mweight(varname)support(varname)notablenodistlabelonlysiggraphhistgraph_options]

Raw samples:

pstest[varlist][ifexp] [inrange],rawtreated(varname)[notablenodistlabelonlysiggraphhistgraph_options]

Before and after matching:

pstest[varlist][ifexp] [inrange],both[treated(varname)mweight(varname)support(varname)notablenodistlabelgraphhistgraph_options]

Options

treated(varname)Treatment (or group) indicator (0/1). If optionrawis not specified, default is _treated left behind from the latestpsmatch2call.

bothRequires comparability to be assessed both before and after matching. Default is only after matching.

rawRequires comparability to be assessed between any two (unweighted) groups. This can be before wishing to perform matching, but also unrelated to matching purposes, e.g. to quickly assess how randomisation has worked.

mweight(varname)Weight of matches. If optionrawis not specified, default is _weight left behind from the latestpsmatch2call.

support(varname)Common support indicator (0/1). If optionrawis not specified, default is _support left behind from the latestpsmatch2call.

notableDo not display the table with the individual covariate imbalance indicators (standardised percentage bias, t-tests, and if optionbothis specified achieved percentage reduction in absolute bias) for each variable invarlist.

labelDisplay variable labels instead of variable names in the variable-by-variable table.

onlysigIn the variable-by-variable table only display those variables which are significantly unbalanced (p<=0.10). This option is ignored if optionbothis specified.

nodistDo not display the distribution summary of the absolute standardised percentage bias across all variables invarlist.

graphDisplay a graphical summary of covariate imbalance via a dot chart, showing the standardised percentage bias for each covariate. If optionbothis specified, information before and after matching is displayed in the same dot chart. If more than 30 covariates are specified, they are not labelled.

histDisplay a graphical summary of covariate imbalance via a histogram, showing the distribution of the standardised percentage bias across covariates. If optionbothis specified, imbalance before and after matching is displayed in two histograms. Recommended for a large number of covariates.

graph_optionsAdditional options can be specified for the relevant graph type (dot graph or histogram). Useful examples areyscale(range(numlist)),ylabel(numlist))orlegend(off)for the former andbin(#)for the latter.

Examples. pstest age gender foreign exper, t(training) mw(_weight) onlysig graph . pstest age foreign exper if district==1, raw t(male) label hist . psmatch2 treated age gender foreign exper, outcome(wage) . pstest . pstest, both

Also seeThe commands psmatch2, psgraph.

Background ReadingRosenbaum, P.R. and Rubin, D.B. (1985), "Constructing a Control Group Using Multivariate Matched Sampling Methods that Incorporate the Propensity Score",

The American Statistician 39(1), 33-38.

AuthorEdwin Leuven, University of Oslo. If you observe any problems mailto:e.leuven@gmail.com.