*! version 1.0 16 Jun 2023
help didplacebo

Title

didplacebo -- implementation of in-time, in-space and mixed placebo tests for estimating difference-in-differences (DID) models

Syntax

didplacebo estimatename
,
treatvar(treatvarname)
[options]

Types of Placebo Test
pbotime(numlist:numlist) in-time placebo test

pbounit in-space placebo test

pbomix([1 2 3]) mixed placebo test

Optimization for In-time Placebo Test
nodrop keep all observations for the control group (never-treated group)

Optimization for In-space and/or Mixed Placebo Tests
repeat(int) the number of replications

seed(int) seed used by the random number generator

Optimization for Mixed Placebo Test
ranunitnum(int) the number of units randomly selected as fake treated units

rantimescope(t_min t_max) the range within which fake treatment times are randomly selected

Reporting
frame(framename) frame storing estimated placebo effects

nofigure do not display figures

savegraph(prefix, [asis replace]) save all produced graphs to the current path.

xtset panelvar timevar must be used to declare a panel dataset; see [XT] xtset.

estimatename should be an estimation saved by estimates store; see [R] estimates store.

Description

didplacebo implements placebo tests for estimating difference-in-differences (DID) models, where policy adoption may be synchronized or staggered.
In particular, didplacebo performs in-time placebo tests using fake treatment times,
in-space placebo tests using fake treatment units (randomly selected),
and mixed placebo tests using both fake treatment units and times (both randomly selected).
Moreover, didplacebo produces convenient graphs for visualization.

Required Settings

To perform a placebo test, it is necessary to specify the estimation name,
estimatename, and the treatment variable name, treatvarname,
as inputs to the didplacebo command.
didplacebo automatically executes estimates restore estimatename to capture the command used for DID estimation e(cmdline). The model is then estimated again with the same command using a fake treatment variable to yield an estimate of placebo effect.

treatvar(treatvarname) specifies a dummy variable indicating whether a unit is treated in a particular period.

Options

Types of Placebo Test

pbotime(numlist:numlist) specifies in-time placebo test using a series of fake treatment times prior to the actual treatment time,
while all treated observations are dropped. The fake treatment times are specified by numlist,
which contains positive integers used to generate fake treatment variables by moving the initial treatment time treatvarname numlist-periods forward.
The placebo effects are then estimated using the fake treatment variables. didplacebo uses the same method to compute the standard errors as specified by the command used to estimate the DID model.
For example, if the DID model was previously estimated by "xtreg, r", then didplacebo uses "xtreg, r" as well, which yields cluster-robust standard errors.
Similarly, if the DID model was previously estimated by "reghdfe, cluster(clustvar)", then didplacebo uses "reghdfe, cluster(clustvar)" too, again yielding cluster-robust standard errors.

Here is an example of shifting the initial treatment time 2-period forward, where "o" means untreated while "x" denotes treated or taken placebo:

t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=8 t=1 t=2 t=3 t=4
+
+ +
+
i=1 | o o o o o o o o | i=1 | o o o o |
i=2 | o o o o o o o o | drop posttreatment data i=2 | o o o o |
i=3 | o o o o o o o o |
> i=3 | o o o o |
i=4 | o o o o x x x x | 2-period forward i=4 | o o x x |
i=5 | o o o o x x x x | i=5 | o o x x |
+
+ +
+

pbounit specifies in-space placebo test where units are randomly selected as fake treatment units, while keeping the treatment time intact.
For DID models with staggered adoption, this implies that the cohort structure (i.e., how many units in each cohort with a same treatment time) is maintained.
In-space placebo test implements many repetitions, say 500 repetitions. In each repetition, a fake treatment variable is generated by randomly shuffling treatvarname blockwise along the dimension of panelvar.
The model is then estimated again using the fake treatment variable to yield an estimate of the placebo effect.
After many repetitions, a distribution of estimated placebo effects is obtained.
For statistical inference, the estimated treatment effect is then compared against the distribution of estimated placebo effects to calculate two-sided, left-sided and right-sided p-values.

Here is an example of randomly shuffling treatvarname blockwise along the dimension of panelvar:

t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=8 t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=8
+
+ +
+
i=1 | o o o o o o o o | i=1 | o o o o x x x x |
i=2 | o o o o o o o o | i=2 | o o o o o o o o |
i=3 | o o o o o o o o |
> i=3 | o o o o o o o o |
i=4 | o o o o x x x x | shuffle by panelvar i=4 | o o o o x x x x |
i=5 | o o o o x x x x | i=5 | o o o o o o o o |
+
+ +
+

pbomix([1 2 3]) specifies mixed placebo test where both treatment units and times are randomized.
Mixed placebo test implements many repetitions, say 500 repetitions.
In each repetition, a fake treatment variable is generated by randomly selecting both fake treatment units and times.
Not less than one of 1, 2 and 3 must be specified in pbomix() to choose the version of implementation,
where 1 correponds to the version suitable for standard DID, 2 and 3 correpond to the unrestricted and restricted versions suitable for staggered DID, respectively.

pbomix(1) is used for standard DID with synchronized policy adoption.
In each repetition, a certain number of fake treatment units is randomly selected, which are then assigned a same fake treatment time randomly chosen.
The number of fake treatment units and the range of fake treatment times are specified by the options ranunitnum(int) and rantimescope(t_min t_max) respectively.

Here is an example of randomizing both treatment units and times for standard DID with synchronized adoption:

t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=8 t=1 t=2 t=3 t=4 t=5 t=6 t=7 t=8
+
+ +
+
i=1 | o o o o o o o o | i=1 | o o o o o o x x |
i=2 | o o o o o o o o | randomly generate i=2 | o o o o o o x x |
i=3 | o o o o o o o o |
> i=3 | o o o o o o o o |
i=4 | o o o o x x x x | i=4 | o o o o o o o o |
i=5 | o o o o x x x x | i=5 | o o o o o o o o |
+
+ +
+

pbomix([2 3]) is used for staggered DID with staggered policy adoption, where 2 and 3 correponds to the unrestricted and the restricted versions, respectively.
The unrestricted version pbomix(2) randomly selects a fake treatment time for each unit, without maintaining the original cohort structure (i.e., how many units in each cohort).
The number of fake treatment units and the range of fake treatment periods are specified by the options ranunitnum(int) and rantimescope(t_min t_max) respectively.

On the other hand, the restricted version pbomix(3) maintains the cohort structure.
Specifically, suppose the treatment times in the sample consist of t_1, t_2,...,t_G, and the correponding cohorts contain N_1, N_2,...,N_G units respectively.
In each repetition, we randomly partition all units in the sample into G cohorts containing N_1, N_2,...,N_G units respectively,
then randomly assign G different times from rantimescope(t_min t_max) to these cohorts as their fake treatment times.
The range of fake treatment times specified by the options rantimescope(t_min t_max) must be greater than or equal to the number of periods in the actual treatment variable.

The model is then estimated again using the fake treatment variable to yield an estimate of the placebo effect.
After many repetitions, we end up with a distribution of estimated placebo effects.
For statistical inference, the estimated treatment effect is then compared against the distribution of estimated placebo effects to calculate two-sided, left-sided and right-sided p-values.

Optimization for In-space and/or Mixed Placebo Test

nodrop specifies to keep all observations for the control group (never-treated group).
If the nodrop option is not specified, didplacebo will automatically drops all posttreatment observations, including those for the control group.

Optimization for In-space and/or Mixed Placebo Test

repeat(int) specifies the number of repetitions, which defaults to 500.

seed(int) specifies the random seed for reproducible results, which defaults to 1.

Optimization for Mixed Placebo Test

ranunitnum(int) specifies the number of units randomly selected as fake treatment units in each repetition.
If ranunitnum(int) is not specified, int defaults to the actual number of treatment units in the sample.
All units in the sample have the same probability of being randomly selected as fake treatment units.

rantimescope(t_min t_max) specifies the range within which fake treatment times are randomly selected in each repetition.
All periods within the range of t_min to t_max have the same probability of being randomly assigned as a fake treatment time via a uniform distribution on this range.

For the standard DID design, if rantimescope(t_min t_max) is not specified, t_min and t_max default to the second period and the last period in the sample respectively.

For the staggered DID design, if rantimescope(t_min t_max) is not specified, t_min defaults to the earliest treatment time (or the first period in the sample, whichever is larger), and t_max defaults to the latest treatment time in the sample. Another possible choice is for users to set t_min and t_max as the first period and the last period in the sample respectively.

Reporting

frame(framename) creates a Stata frame storing placebo effects generated by in-space and/or mixed placebo tests. The frame named framename is replaced if it already exists, and created if not.

nofigure do not display figures. The default is to display all figures from placebo tests if available.

savegraph(prefix, [asis replace]) automatically and iteratively calls the graph save to save all produced graphs to the current path,
where prefix specifies the prefix added to _graphname to form a file name,
that is, the graph named graphname is stored as prefix_graphname.gph.
asis and replace are options passed to graph save; for details, see [G-2] graph save.
Note that this option only applies when nofigure is not specified.

Example 1 (Standard DID): the effects of the abandonment of China's Grand Canal (Cao and Chen, 2022)

. use cao_chen.dta, clear

. xtset county year

. reghdfe rebel canal_post, absorb(i.county i.year) cluster(county)

. estimates store did_cao_chen

* Implements in-time placebo tests using fake treatment time forwarded by 1-10 periods

. didplacebo did_cao_chen, treatvar(canal_post) pbotime(1(1)10)

* Implements in-space placebo test

. didplacebo did_cao_chen, treatvar(canal_post) pbounit

* Implements mixed placebo test suitable for standard DID

. didplacebo did_cao_chen, treatvar(canal_post) pbomix(1)

* Implements in-time, in-space and mixed placebo tests simultaneously

. didplacebo did_cao_chen, treatvar(canal_post) pbotime(1(1)10) pbounit pbomix(1)

Example 2 (Staggered DID): the impact of bank deregulation on income inequality (Beck et al., 2010)

. use bbb.dta, clear

. xtset statefip wrkyr

. global cov gsp_pc_growth prop_blacks prop_dropouts prop_female_headed unemploymentrate

. xtreg log_gini _intra $cov i.wrkyr, fe r

. estimates store did_bbb

* Implements in-time placebo test using fake treatment time forwarded by 1-10 periods

. didplacebo did_bbb, treatvar(_intra) pbotime(1(1)10)

* Implements in-space placebo test

. didplacebo did_bbb, treatvar(_intra) pbounit

* Implements the free version of mixed placebo test suitable for staggered DID

. didplacebo did_bbb, treatvar(_intra) pbomix(2)

* Implements the restricted version of mixed placebo test suitable for staggered DID

. didplacebo did_bbb, treatvar(_intra) pbomix(3)

* Implements in-time, in-space and mixed placebo tests

. didplacebo did_bbb, treatvar(_intra) pbotime(1(1)10) pbounit pbomix(2 3)

Note

To conduct placebo tests using heterogeneity-robust estimators (say, CSDID by Callaway and Sant'Anna, 2021) other than the traditional two-way fixed effects estimator (TWFE), one can make use of our companion commands xtshuffle,
xtrantreat and tofirsttreat.
For detailed illustrations, refer to Chen, Qi and Yan (2023).

Compatibility

didplacebo is confirmed to be compatible with official commands regress, xtreg, areg and xtdidregress as well as the third-party command reghdfe.
Other commands, though not specifically tested, may also be compatible with didplacebo.

Stored Results

didplacebo stores the following in e():

Macros

e(didestname) name of DID estimation generated by estimates store

e(didcmd) Stata command used for DID estimation

e(didcmdline) Stata command line used for DID estimation

e(panelvar) name of panel variable

e(timevar) name of time variable

e(treatvar) name of treatment variable

e(cmd) didplacebo

e(cmdline) command as typed

e(seed) seed used by the random number generator for reproducible results

e(graph) names of all produced graphs

e(frame) name of Stata frame storing placebo effects

Matrices

e(pbounit) matrix containg coefficient of treatment effect and p-values of placebo effects generated by in-space placebo test

e(pbotime) matrix containg coefficients and confidence intervals of placebo effects generated by in-time placebo test

e(pbomix1) matrix containg coefficient of treatment effect and p-values of placebo effects generated by mixed placebo test for standard DID

e(pbomix2) matrix containg coefficient of treatment effect and p-values of placebo effects generated by mixed placebo test for staggered DID (free version)

e(pbomix3) matrix containg coefficient of treatment effect and p-values of placebo effects generated by mixed placebo test for staggered DID (restricted version)

Resources

xtshuffle shuffles a variable in a panel dataset blockwise along the dimension of panel variable

xtrantreat randomizes a treatment variable in a panel dataset

tofirsttreat converts a treatment variable from an indicator variable to a variable specifying the first treatment period

Reference

Beck, T., Levine, R., and Levkov, A. 2010. Big bad banks? The winners and losers from bank deregulation in the United States. Journal of Finance 65(5): 1637-1667.

Callaway, B., and Sant'Anna, P.H.C. 2021. Difference-in-differences with Multiple Time Periods. Journal of Econometrics 225: 200-230.

Cao, Y., and Chen, S. 2022. Rebel on the canal: Disrupted trade access and social conflict in China, 1650–1911. American Economic Review 112(5): 1555-1590.

Chen, Q., Qi, J., and Yan, G. 2023. didplacebo: Implementing placebo tests for difference-in-differences estimations. Shandong University working paper.

Author

Qiang Chen, Shandong University, CN

www.econometrics-stata.com

qiang2chen2@126.com

Ji Qi, Shandong University, CN

miracle4556@163.com

Guanpeng Yan (correponding author), Shandong University of Finance and Economics, CN

guanpengyan@yeah.net