help listreghttps://github.com/benjann/listreg/

Title

listreg -- Linear regression for list experiments

Syntax

Single-list design:

listreg ovar [=] tvar [indepvars] [if] [in] [weight]
[,
options
]

where ovar is the outcome variable from the list experiment, and where
tvar is a variable identifying the experimental groups
(tvar==1: long-list group; tvar!=1: short-list group).

Double-list design:

listreg ovar1 ovar2 = tvar [indepvars] [if] [in] [weight]
[,
options
]

ovar1 and ovar2 are the two outcome variables from the double-list experiment, and where
tvar is a variable identifying the experimental groups
(tvar==1: long-list group for ovar1 and short-list group for ovar2;
tvar!=1: short-list group for ovar1 and long-list group for ovar2).

options

Main
noconstant suppress constant term

controls(varlist) custom specification of short-list equation(s)

average use alternative double-list estimator

listwise use listwise deletion to handle missing values

casewise synonym for listwise

aequations include auxiliary equations in results

noisily display output from estimation process

VCE
vce(vcetype) how standard errors are computed

cluster(clustvar) synonym for vce(cluster clustvar)

nodfr report large-sample test statistics

nose omit variance estimation

ifgenerate(spec) store influence functions

replace allow replacing existing variables

Reporting
reporting_option standard reporting options

indepvars and controls() may contain factor variables; see fvvarlist.

pweights, iweights, and fweights are allowed; see help weight.

Description

listreg fits a linear model to data from a list experiment
(a.k.a. item count technique; see, e.g.,
Droitcour et al. 1991,
Blair and Imai 2012,
Glynn 2013,
Blair et al. 2020,
Ehler et al. 2021)
or to data collected by the item-sum technique
(Trappmann et al. 2014,
Krumpal et al. 2018).
Single-list and double-list designs are supported.

listreg only covers linear (least-squares) models. For a more comprehensive
package featuring alternative methods see the kict package by
Tsai (2019).

Variance estimation in listreg is based on influence functions
(see Jann 2020). Complex
survey estimation is supported, although you need to specify option
vce(svy) rather than applying the svy prefix command.

A distinct feature of listreg is that the control equation modeling
the short-list outcome can be different from the main equation of the
model. For example, when applying an intercept-only model to estimate the
overall prevalence, including predictors in the control equation may
increase statistical efficiency of the estimate. Likewise, you
may want to specify more complex effect shapes in the control equation than
in the main equation to improve the fit of the short-list
model. Important: While it is perfectly fine to use a main equation that is
less complex than the control equation, using a control equation that is
less complex than the main equation can lead to invalid results (i.e.,
results that are only due to a lack of fit in the control equation; for
example, it is usually not a good idea to include predictors in the main
equation that are not part of the control equation).

Options

noconstant suppresses constant term.

controls([varlist][, noconstant none])
specifies the variables to be included in the short-list equation. By
default, the same specification is used as for the main outcome
equation. Factor variables are allowed; see fvvarlist. Suboption
noconstant omits the constant term in the specified
short-list equation; noconstant has an effect only if varlist is
specified. Suboption none leaves the short-list equation empty; none
has no effect if varlist is specified.

In the double-list design, option controls() can be repeated to set
different specifications for the two short-list models. The first instance
of controls() determines the specification of the short-list 1
model; the second instance determines the specification of the short-list 2
model. If controls() is specified only once, the same specification
is used for both models.

average reports average coefficients from two separate outcome
equations. This is only relevant in case of the double-list design. The default
is to report coefficients from a pooled outcome model across both lists.

listwise handles missing values through listwise deletion,
meaning that the same set of observations will be used for both
outcome variables. This is only relevant in case of the double-list
design. By default, listreg determines the available observations
individually for each list.

casewise is a synonym for listwise.

aequations specifies that the coefficients from the
auxiliary equations (short-list models, separate long-list models in case of
average) be included in the results. By default, listreg
only includes the main coefficients.

noisily displays a trace of the output from the estimation
process. Note that the standard errors in
the output made visible by noisily may not be valid.

vce(vcetype) determines how standard errors are computed. vcetype may be

robust
cluster clustvar
svy [svy_vcetype] [, svy_options ]
bootstrap [, bootstrap_options ]
jackknife [, jackknife_options ]

vce(robust) computes robust standard errors; this
is the default. vce(cluster clustvar) computes standard errors
allowing for intragroup correlation within groups defined
by clustvar. vce(svy) computes standard errors taking the survey
design as set by svyset into account. The syntax is equivalent to
the syntax of the svy prefix command; that is, vce(svy) is
listreg's way to support the svy prefix. vce(bootstrap)
and vce(jackknife) compute standard errors using bootstrap or
jackknife, respectively; see help vce_option.

cluster(clustvar) is a synonym for vce(cluster clustvar).

nodfr divides variances by N rather than N-1 and reports test statistics
based on the standard normal distribution rather than the
t-distribution. Use this option to obtain results that are equivalent to
results returned by gmm. nodfr has no effect if vce(svy),
vce(bootstrap), or vce(jackknife) is specified.

nose omits variance estimation; no standard error will be reported
in this case.

ifgenerate(spec) stores the influence functions of the coefficients
in the outcome model. Either specify a list of new variables names, or
specify stub*, in which case the new variables will be named
stub1, stub2, etc. Option ifgenerate() is not
allowed with vce(bootstrap) or vce(jackknife).

replace allows to overwrite existing variables.

reporting_option are standard reporting options such as level(#),
coeflegend, noheader, or
display_options.

Examples

Data
Single-list estimate
Double-list estimate
Adding predictors and controls
Consistency test in the double-list design
Relation to GMM
Relation to treatment effect estimation

Data

Coutts et al. (2011)
report results from a list experiment on plagiarism
(intentional inclusion of text from another source without citation) by
university students. An excerpt from the data of this study is as follows.

. use http://fmwww.bc.edu/repec/bocode/l/listreg.dta, clear
. describe

The study used a double-list design. Variable plagiarism_1 contains
the response to list 1, plagiarism_2 contains the response to
list 2. Variable longlist=1 indicates that the sensitive item was
included in list 1; longlist=2 indicates that the sensitive item was
included in list 2.

Single-list estimate

The plagiarism prevalence can be estimates from list 1 as follows:

. listreg plagiarism_1 longlist

Likewise, the estimate from list 2 is:

. listreg plagiarism_2 2.longlist

Both results indicate that about 10 percent of students have ever intentionally
plagiarized in a term paper, but the estimates are rather imprecise.

Double-list estimate

A more efficient estimate can be obtained by a combined analysis of both lists:

. listreg plagiarism_1 plagiarism_2 = longlist

The equal sign indicates to listreg that the first two variables
are both outcome variables. Using wildcard notation, we can omit the equal
sign (assuming that there are only two matching variables):

. listreg plagiarism_* longlist

By default, the double-list estimator uses a pooled outcome model; to report
average coefficients from two separate outcome equations, type:

. listreg plagiarism_* longlist, average

Results are almost identical, which is not surprising since the two experimental
groups are of similar size.

Adding predictors and controls

Not all students seem to be aware of the university's plagiarism
regulations. To test whether awareness of regulations is related to
plagiarism prevalence we can add a corresponding indicator as an
independent variable to the model:

. listreg plagiarism_* longlist i.unaware

Indeed, it seems that plagiarism occurs primarily among students who are
unaware of the regulations; plagiarism prevalence is about 32 percentage points
higher than for other students. To report the plagiarism levels for both groups
(rather than the difference), we can type:

. listreg plagiarism_* longlist ibn.unaware, noconstant

The prevalence estimate for students who know the regulations is 2 percent;
for students who do not know the regulations it is 34 percent.

Here is a more complicated analysis that also tests the effect of gender and
includes further covariates in the short-list equation to improve the fit:

. listreg plagiarism_* longlist i.unaware i.female, controls(i.unaware i.female year i.working i.papers)

The results of the short-list models are not included in the listreg
output because they are not of primary interest. If you are interested in these
models, you can specify option noisily to display a trace of the estimation
process, including the short-list models. Alternatively, specify option
aequations to include the auxiliary models in the final output as separate
equations:

. listreg plagiarism_* longlist i.unaware i.female, controls(i.unaware i.female year i.working i.papers) aequations

Furthermore, it is typically easy to replicate the short-list
models outside of listreg by applying regress to the short-list
subsamples. In the current example, the two models are as follows:

. regress plagiarism_1 i.unaware i.female year i.working i.papers if longlist!=1, robust

. regress plagiarism_2 i.unaware i.female year i.working i.papers if longlist==1, robust

Note that you can specify two controls() options, if you want
to use different specifications for the two short-list equations. Example:

. listreg plagiarism_* longlist i.unaware, controls(i.unaware i.female i.working i.papers) controls(i.unaware year)

Consistency test in the double-list design

A systematic difference in results between the two lists in a double-list
design indicates that the list experiment did not work as expected: apparently,
the choice of the shortlist, or other differences between the two
experimental groups, mattered for the results, which is at odds
with the basic assumptions of the list experiment.

One approach to test the consistency of results
across lists is to specify options average and aequations so that
list-specific outcome equations are estimated and included
in the results. Then use command test to evaluate
whether the two equations are different (when calling
test, make sure to specify option constant so that the constant
is included in the test). Example:

. listreg plagiarism_* longlist, average aequations
. test [LL1 = LL2], constant

This also works if there are predictors and controls in the model:

. listreg plagiarism_* longlist i.unaware i.female, average aequations controls(i.unaware i.female year i.working i.papers)

. test [LL1 = LL2], constant

An alternative approach is to add an indicator for the experimental group
to the outcome equation of the default estimator. The effect of the
group indicator is equal to the difference in results between the two lists.

. listreg plagiarism_* longlist i.longlist

If the model contains predictors, include all interactions between the group
indicator and the predictors and then use test to perform a joint
test across the main effect and all interaction terms.

. listreg plagiarism_* longlist i.unaware i.female i.longlist i.longlist#(i.unaware i.female), controls(i.unaware i.female year i.working i.papers)

. test 2.longlist 2.longlist#1.unaware 2.longlist#1.female

The two outlined approaches lead to equivalent results.

Relation to GMM

The results computed by listreg can be replicated by gmm. Example
for the single-list design:

local Y plagiarism_1
local T longlist
local X i.unaware i.female
local Z `X' year i.working i.papers
gmm (1: (`T'!=1)*(`Y' - `Z' _cons)) ///
(2: (`T'==1)*(`Y' - - `X' _cons)) ///
, instruments(1:`Z') instruments(2:`X') winitial(identity)
listreg `Y' `T' `X', controls(`Z') nodfr

Example for the double-list design:

local Y1 plagiarism_1
local Y2 plagiarism_2
local T longlist
local X i.unaware
local Z1 `X' i.female i.working i.papers
local Z2 `X' year
gmm (1: (`T'!=1)*(`Y1' - `Z1' _cons)) ///
(2: (`T'==1)*(`Y2' - `Z2' _cons)) ///
(3: (`T'==1)*(`Y1' - ) + (`T'!=1)*(`Y2' - ) - `X' _cons) ///
, instruments(1:`Z1') instruments(2:`Z2') instruments(3:`X') ///
winitial(identity)
listreg `Y1' `Y2' = `T' `X', controls(`Z1') controls(`Z2') nodfr listwise

Relation to treatment effect estimation

The single-list estimator implemented in listreg is equivalent to a
regression-adjustment (RA) estimator of the average treatment effect on the
treated (ATET). Here is an example illustrating the equivalence between
listreg and teffects ra (treatment effect of smoking on birth weight):

. webuse cattaneo2, clear

. teffects ra (bweight prenatal1 mmarried mage fbaby) (mbsmoke), atet

. listreg bweight mbsmoke, controls(prenatal1 mmarried mage fbaby) nodfr

This means that listreg can be used for treatment effect heterogeneity
analysis. For example, we might be interested in whether the treatment effect
of smoking on birth weight varies by first-trimester exam status and
first-birth status:

. listreg bweight mbsmoke prenatal1 fbaby, controls(prenatal1 mmarried mage fbaby) nodfr

First-trimester exam status does not seem to play a role, but the results
indicate that the treatment effect is less pronounced in case of a
first baby.

Returned results

listreg stores its results in e(). Type ereturn list
after estimation for more information.

Methods and Formulas

listreg works by applying linear regression to residualized outcome
variables. Let Y be the outcome, T be the long-list indicator, X be
a vector of predictors (including constant), and Z be a vector of controls. The
estimation procedure for the single-list design then is as follows:

1. Regress Y on Z in the subsample for which T != 1.

2. Predict residuals R from this model in the subsample for which T = 1.

3. Regress R on X in the subsample for which T = 1 and report the coefficients.

For the double-list design, let Y1 be the outcome from list 1,
Y2 be the outcome from list 2, Z1 be the controls for list 1, and
Z2 be the controls for list 2. The default procedure then is as follows:

1. Regress Y1 on Z1 in the subsample for which T != 1.

2. Regress Y2 on Z2 in the subsample for which T = 1.

3. Predict residuals R from first model in the subsample for which T = 1
and from the second model in the subsample for which T != 1.

4. Regress R on X and report the coefficients.

If option average is specified, step 4 is replaced by:

4. Regress R on X subsample for which T = 1.

5. Regress R on X subsample for which T != 1.

6. Report the average of the coefficients from steps 4 and 5.

Variance estimation is conducted at the end, based on the influence functions
implied by the chain of estimation steps.

References

Blair, G., A. Coppock, M. Moor. 2020. When to Worry about Sensitivity
Bias: A Social Reference Theory and Evidence from 30 Years of List
Experiments. American Political Science Review
114(4):1297–1315. doi.org/10.1017/S0003055420000374

Blair, G., K. Imai. 2012. Statistical Analysis of List Experiments. Political
Analysis 20(1):47-77. doi.org/10.1093/pan/mpr048

Coutts, E., B. Jann, I. Krumpal, A.-F. Näher. 2011. Plagiarism
in Student Papers: Prevalence Estimation Using Special Techniques for
Sensitive Questions. Jahrbücher für Nationalökonomie und Statistik
231(5-6):749-760. doi.org/10.1515/jbnst-2011-5-612

Droitcour, J., R.A. Caspar, M.L. Hubbard, T.L. Parsely, W. Visscher,
T.M. Ezzati. 1991. The Item Count Technique as a Method of Indirect
Questioning: A Review of its Development and a Case Study Application. P. 185–210
in: P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowetz, S. Sudman (eds.). Measurement
Errors in Surveys. New York: Wiley. doi.org/10.1002/9781118150382.ch11

Ehler, I., F. Wolter, J. Junkermann. 2021. Sensitive Questions in
Surveys: A Comprehensive Meta-Analysis of Experimental Survey Studies on
the Performance of the Item Count Technique. Public Opinion Quarterly
85(1):6–27. doi.org/10.1093/poq/nfab002

Glynn, A. N. 2013. What Can We Learn with Statistical Truth Serum?: Design
and Analysis of the List Experiment. Public Opinion Quarterly
77(S1):159–172. doi.org/10.1093/poq/nfs070

Jann, B. 2020. Influence functions continued. A framework for estimating standard
errors in reweighting, matching, and regression adjustment. University of Bern Social
Sciences Working Papers No. 35. doi.org/10.7892/boris.142529

Krumpal, I., B. Jann, M. Korndörfer, S. Schmukle. 2018. Item Sum Double-List
Technique: An Enhanced Design for Asking Quantitative Sensitive Questions. Survey
Research Methods 12(2):91-102. doi.org/10.18148/srm/2018.v12i2.7247

Trappmann, M., I. Krumpal, A. Kirchner, B. Jann. 2014. Item Sum: A New
Technique for Asking Quantitative Sensitive Questions. Journal of Survey Statistics
and Methodology 2(1):58-77. doi.org/10.1093/jssam/smt019

Tsai, C. 2019. Statistical analysis of the item-count technique using Stata. Stata Journal
19(2):390-434. doi.org/10.1177/1536867X19854018

Author

Ben Jann, University of Bern, ben.jann@unibe.ch

Thanks for citing this software as follows:

Jann, B. (2024). listreg: Stata module for the analysis of list experiments
using linear regression. Available from
https://ideas.repec.org/c/boc/bocode/s459304.html.

Also see

Online: help for
regress, gmm, teffects ra