*! version 3.2 17oct2021

Title

pc_dd_covar -- Estimates variance and average covariances of idiosyncratic residuals for a difference-in-differences model

Syntax

pc_dd_covar depvar [if] [in] , pre(#) post(#) [i(panelvar) t(timevar) display]

Required
pre(#) number of pre-treatment periods; must be positive integer.

post(#) number of post-treatment periods; must be positive integer.

Optional
* i(panelvar) cross-sectional unit of randomization; must be numeric; will default to stored r(panelvar) if not specified.

* t(timevar) time period variable; must be numeric; will default to stored r(timevar) if not specified.

[if] [in] option to restrict observations over which program estimates idiosyncratic residual variance and covariances of depvar.

display display estimated variance and average covariances in Stata console.

* Required if r(panelvar) and r(timevar) are not stored in memory.

Description

This program estimates the average residual variance and the average within-unit
residual covariances for a difference-in-differences model with panelvar and timevar fixed effects:

- Y_it = b*D_it + fe_i + fe_t + e_it

It is designed to be called as a subprogram within pc_dd_analytic.

pc_dd_covar returns the following 4 estimates, averaged across all possible experiments with pre pre-treatment periods and post post-treatment periods:

- residual variance:
var(e_it)

- average pre-treatment residual covariance:
mean(cov(e_it,e_is)), for all pre-treatment periods t!=s

- average post-treatment residual covariance:
mean(cov(e_it,e_is)), for all post-treatment periods t!=s

- average pre/post-treatment residual covariance:
mean(cov(e_it,e_is)), for all pre-treatment periods t and post-treatment periods s

Importantly, these estimates depend on the number of pre-treatment and post-treatment observations included in the
regression specification, for two reasons.

First, residuals from time periods farther apart are likely to be less correlated,
thereby reducing the average across all off-diagonal covariances. For example,
a model with pre=3 will likely have a higher average covariance between pre-treatment
residuals than a model with pre=30 that uses the exact same
dataset. This is because the former is averaging the covariances of residuals separated by at most 2 periods, whereas the latter
averages covariances separated by up to 29 periods.

Second, residuals are only estimates of the error terms of the underlying data-generating
process. As panel length increases, residuals will more closely approximate the true
error terms (because the number of panel observations increases faster than the
degrees of freedom, allowing more precise estimation of fixed effects). For
example,
suppose a data-generating process includes an idiosyncratic error term with a variance of 10. The residuals from an estimated two-period panel fixed model will have
a variance of much less than 10, and for greater numbers of time periods this residual variance will increase approaching 10. (Residual covariances behave analogously.)

As a result, directly plugging in these estimates into
an analytic power calculation formula will produce an inaccurate minimum detectable effect. pc_dd_analytic
corrects for this estimation error in expectation using the result derived in Appendix E of:

Burlig, Fiona, Louis Preonas, and Matt Woerman (2022). "Panel Data and Experimental Design." Journal of Development Economics 144: 102548.

Appendix D.1 of this paper
outlines the full variance-covariance estimation procedure implemented by pc_dd_covar.

Stored results

pc_dd_covar stores the following scalars in r():

- r(variance)

- r(cov_pre)

- r(cov_post)

- r(cov_cross)

- r(n_units)

The first 4 terms correspond to estimates of sigma^2_[omega^hat], psi^B_[omega^hat], psi^A_[omega^hat], and psi^X_[omega^hat] using the
notation of
Burlig, Preonas, and Woerman (2020).

r(n_units) reports the average number of cross-sectional
units used to estimate each of the above 4 terms. This value is used by
pc_dd_analytic
as a correction factor to estimate analytical power calculations.

If pre=1 (post=1) then the average residual covariance between pre-treatment (post-treatment) periods is undefined, and
the stored value will be missing.

Details

This program uses the following algorithm to estimate the average residual variance and average residual covariance terms:

1. Determine the range of feasible experiments, given the panel length (pre+post) and the number of time periods present
in the dataset. For example, if timevar = {1,2,...,100}, pre=5 and post=6, then there are 90 feasible ranges for an 11-period experiment (i.e., beginning
in periods 1 thru 90). If the dataset contains over 5000 feasible ranges,
the program will sample 5000 ranges without replacement.

2. For each feasible range:

(a) Regress depvar on panelvar and timevar, storing the (idiosyncratic) residuals. (This regression includes
all cross-sectional units with non-missing data, but only time periods
within the specific pre+post range.)

(b) Calculate the variance of the stored residuals.

(c) For each pair of pre-treatment periods (i.e., the
first pre time periods in the specific pre+post range), calculate the covariance between these periods' residuals. Sum
these estimated pre-treatment covariances and divide by the total number of pre-treatment pairs, to estimate the average pre-treatment covariance.

For example, if pre=4, this step will sum
{cov(e_i1,e_i2), cov(e_i1,e_i3), cov(e_i1,e_i4), cov(e_i2,e_i3)
cov(e_i2,e_i4), cov(e_i13,e_i4)}, and divide by pre*(pre-1)/2 = 6.

(d) For each pair of post-treatment periods (i.e., the
last post time periods in the specific pre+post range), calculate the average covariance by the same method.

(e) For each pair of pre- and post-treatment periods (i.e., the
first pre and the last post time periods in the specific pre+post range), calculate the average covariance by the same method.

For example, if pre=4 and post=2, this step will sum
{cov(e_i1,e_i5), cov(e_i1,e_i6), cov(e_i2,e_i5), cov(e_i2,e_i6)
cov(e_i3,e_i5), cov(e_i3,e_i6), cov(e_i4,e_i5), cov(e_i4,e_i6)}, and divide by pre*post = 8.

3. Average the results of steps (b)-(e) across all feasible ranges.

One note of caution: using this algorithm to estimate variance and covariance
parameters from regression residuals will not recover the population parameters governing the data-generating process of the error term.
To take one simple example, suppose an AR(1) error structure
e_it = 0.8*e_i + v_it, where var(v_it)=3.6, implying that var(e_it)=10. In a model with pre=1 and post=1, the average
covariance across pre- and post-treatment periods is simply the AR(1) correlation scaled by the variance of e_it, or cov(e_ipre,e_ipost)=8. However,
partialing out fixed effects from the outcome variable recovers residuals that are not equal to e_it, due to estimation error. The residual
variance will be less than 10, and the average cross-period residual covariance will be negative.

Importantly,
pc_dd_analytic requires that the variance and covariance parameters be internally consistent -- either all must be population parameters (as with the pc_dd_analytic
options variance and ar1), or all must be
estimated from the same vector of regression residuals (via this algorithm). An
example of internally inconsistent variance and covariance parameters would
be variance estimated from a long time series of residuals and covariances estimated using residuals from a shorter (pre+post)-length panel.

The underlying analytic formula used in pc_dd_analytic will depend on whether users input population parameters or estimates
of population parameters. For population parameters specified using options variance, ar1, or trueparameters, pc_dd_analytic will
apply the serial-correlation-robust power calculation formula from Burlig, Preonas, and Woerman (2020). Otherwise,
pc_dd_analytic will automatically apply the
correction factors for estimated parameter values, as derived in
Appendix E
of Burlig, Preonas, and Woerman (2020).

Examples

. pc_dd_covar y, i(person_id) t(t) pre(2) post(8) disp

. pc_dd_covar income, i(household_id) t(year) pre(1) post(2)

. pc_dd_covar price if year>2002, i(firm_id) t(month) pre(5) post(3)

Contact

Louis Preonas

Department of Agricultural and Resource Economics

University of Maryland

Email: lpreonas@umd.edu

This program is part of the ssc package pcpanel.