help bdiff 24Nov2020, blog

Title

bdiff -- Bootstrap and Permutaion tests for difference in coffiecients between two groups

Syntax

bdiff,
group:(groupvar)
model:(string)
[options]

Main (must be specified)
group(varname) specify a dummy variable
to tag the sample into two groups

model(string) define the regression model, e.g., model(reg y x)

Other options
reps(#) set number of repetitions to #. Defult is 100.

seed(#) set random-number seed to #

bsample use bootstrap sample (sampling with replacement) to perform Fisher's Permuation test.
In default, bdiff uses the original sample to perform the Fisher's Permuation test.

surtest perform SUR test, see suest.

Reporting
first report the coeffiencts difference and empirical p-value
for the first regressor in the model

gap to add extra spacing between rows

nodots suppress replication dots

dec(#) to display # decimal places for all statistics. Defult is 3.

bdec(#) to display # decimal places for just the coefficient. Defult is 3.

pdec(#) to display # decimal places for just the empirical p-value. Defult is 3.

detail report the regression results for both groups

weights are not allowed in model().

Description

Introduction

A major focus of empirical literature is the comparison of coeffiencts of a particular variable(s)
(such as "investment-liquidity sensitivities" in corporate finance) across different groups of firms.
However, traditional tests designed to detect differences in coefficients are not
appropriate because the error terms likely violate the required assumptions.

Traditional tests are generally designed for testing changes in parameters across time series data,
where it may sometimes be reasonable to assume no heteroscedasticity
in the resulting residuals. Panel data and cross-sectional data,
are likely violate the required assumptions.
For example, the Chow test requires that the disturbance variance be the same for both regressions,
while the standard Wald test requires independence of the error terms.
These conditions are unlikely to be satisfied by panel data residuals.

As a result, conclusions regarding the existence of differences in
investment-liquidity sensitivities across groups have been largely based on observing differences
in magnitude and level of significance of the coefficient on
the liquidity variable in regression estimates.
bdiff uses simulation evidence to determine the significance of observed differences in coefficient estimates (Cleary, 1999; Lian and Cheng, 2007).

bdiff perform several tests
(Fisher's Permuation test; Seemingly Unrelated Regression test, see suest)
to determine the significance of
observed differences in coefficient estimates between two groups.

By default, bdiff performs traditional Fisher's permutation test (sampling without replacement) of differences in coefficient estimates between two groups
It uses simulation evidence to determine the significance of
observed differences in coefficient estimates between two groups.
For general introduction of this method, see Efron and Tibshirani (1993, Section 15.2, pp.202).

The Fisher's permutation test

The Fisher's permutation test can be used to test the significance of difference between two groups
of any estimator.

In case of regression coefficients difference, Cleary (1999, pp.684-685) uses this method to determine whether there is a significant
difference of "investment-cash flow sensitivity" coefficients
between financial constraint (FC) firms and non-financial constraint (NFC) firms.
Cleary states that:
A bootstrapping procedure is used to calculate empirical p-values
that estimate the likelihood of obtaining the observed differences
in coefficient estimates if the true coefficients are, in fact, equal.

Step 1: Observations are pooled from the two groups whose coefficient estimates
are to be compared.
Using n1 and n2 to denote the number of observations available from each group,
we end up with a total of n1 + n2 observations every year.

Step 2: Each simulation randomly selects n1 and n2 observations
from the pooled distribution and assigns them to group 1 and group 2,
respectively. Coefficient estimates are then determined for each group using
these observations.
The difference between coefficient estimates of group 1 and group 2 is denoted as (di)

Step 3: This procedure (Step 1 and Step 2) is repeated 5000 times.

The empirical p-value is the percentage of simulations where
the difference between coefficient estimates (di)
exceeds the actual observed difference in coefficient estimates (dSample).

This p-value tests against the one-tailed alternative hypothesis
that the coefficient of one group is greater than that of the other group (H1: d > 0).
For example, a p-value of 0.01 indicates that only 50 out of 5000
simulated outcomes exceeded the sample result,
which implies the sample difference is significant,
and supports the notion that d > 0.

Note that, the procedures in Cleary (1999, pp.684-685) is in fact a Fisher's permutation test,
because Cleary do not use samping with replacement.
So, the "A bootstrapping procedure" argument in Cleary (1999, pp.684) may be misleading.
For details, see Efron and Tibshirani (1993, Section 15.2, pp.202).)

For panel data

If the command used for Panel data is specified in model(string), e.g, xtreg, xtabond, etc.,
then the samping is clusted by id vairable specified by xtset.

SUR test

bdiff's surtest option provides a convenient way to perform test for "Do coefficients vary between groups? ". For details, see suest (example 2).

Examples

basic setting: the nlsw88.dta data

cap program drop bdiff

sysuse nlsw88, clear

global xx "ttl_exp married south hours age"

permutation test

bdiff, group(union) model(reg wage $xx) reps(100) detail

bdiff, group(union) model(reg wage $xx) reps(100) bdec(2) pdec(4) gap

Bootstrap sample + permutation test

bdiff, group(union) model(reg wage $xx) reps(500) bsample

SUR test

bdiff, group(union) model(reg wage $xx) surtest

Logit regression

bdiff, group(union) model(logit collgrad $xx) reps(500)

IV regression

bdiff, group(union) model(ivregress 2sls wage $xx (tenure=grad collg)) ///

bdec(3) pdec(3) reps(100) detail

the -first- option

bdiff, group(union) model(reg wage $xx) reps(50)

return list // what is happen?

*-we want report the empirical p-value of variable "hours"

global xx "hours ttl_exp married south age"

*-Set 1

reg wage $xx if union==0

est store m0

reg wage $xx if union==1

bdiff, group(union) model(reg wage $xx) reps(500) nodots first

estadd scalar bdiff_hours = r(bdiff)

estadd scalar pvalue= r(p)

est store m1

*-Temp report the results

esttab m0 m1, mtitle(Non-Union Union) b(%6.3f) nogap ///

s(N r2_a bdiff_hours pvalue) star(* 0.1 ** 0.05 *** 0.01)

*-Set 2

reg wage $xx if c_city==0

est store c0

reg wage $xx if c_city==1

bdiff, group(c_city) model(reg wage $xx) reps(500) nodots first

estadd scalar bdiff_hours = r(bdiff)

estadd scalar pvalue= r(p)

est store c1

*-Report the results

esttab m0 m1 c0 c1, mtitle(Non-Union Union Non-Coll Coll) ///

b(%6.3f) nogap s(N r2_a bdiff_hours pvalue) ///

star(* 0.1 ** 0.05 *** 0.01)

Campare different methods

bdiff, group(union) model(reg wage $xx) reps(500) bsample

mat p_bs = e(pvalues)

bdiff, group(union) model(reg wage $xx) reps(500)

mat p_permu = e(pvalues)

bdiff, group(union) model(reg wage $xx) surtest

mat p_sur = e(pvalues)

mat S = (p_bs, p_permu, p_sur)

mat colnames S = "Bootstrap" "Permute" "SUR"

matlist S, format(%10.3f)

Panel Data (sample by cluster(id))

webuse nlswork.dta, clear

xtset id year

global x "age south ttl_exp hours tenure"

local m "xtreg ln_wage $x, fe"

bdiff, group(collgrad) model(`m') reps(100) bs first detail

Also see

Online: help for chowtest (if installed), bsample, permute, suest.

Acknowledgements

Thanks for Jingxin Hu (jhu109@syr.edu) from Syracuse University.
Hu helps me fix the bug of seed(#) option.

Author

Yujun,Lian (arlionn) Department of Finance, Lingnan College, Sun Yat-Sen University.

E-mail: arlionn@163.com.

Blog: https://www.lianxh.cn

For problems and suggestions

Any problems or suggestions are welcome, please Email to
arlionn@163.com.

References

Cleary, S., 1999, The Relationship between Firm Investment and Financial Status,
Journal of Finance, 54(2): 673-692.
http://onlinelibrary.wiley.com/doi/10.1111/0022-1082.00121/full.

Efron, B., Tibshirani, R., 1993.
An Introduction to the Bootstrap, Chapmann & Hall.

Lian, Y.J., J. Cheng, 2007.
Investment-cash flow sensitivity: Financial constraint or agency cost?
Journal of Finance and Economics, 2007(2): 37-46. (In Chinese)

连玉君, 2016.
Stata: 如何检验分组回归后的组间系数差异？, https://www.lianxh.cn/news/051e3a01cdb19.html