Title

itsarand -- Randomization tests for single-case and multiple-baseline AB phase designs

Syntax

itsarand depvar [indepvars] [if] [in] [weight],
treat(varname)
[ blmin(#)
trmin(#)
level
reps(#)
seed(#)
nodots
left
right
ci(#)
saving(filename, ...) ]

A dataset for a single panel must be declared to be time-series data by using tsset timevar. When the dataset contains multiple panels, a strongly balanced
panel dataset using tsset panelvar timevar must be declared. See tsset.

* treat(varname) specifies the treatment variable

blmin(#) specifies the minimum number of periods in the baseline; default is 2

trmin(#) specifies the minimum number of periods in the intervention; default is 2

level specifies that the difference in the level between treatment and baseline periods be analyzed; default is difference in trend

reps(#) perform # random permutations; default is reps(100)

seed(#) set random-number seed to #

nodots suppress replication dots

left|right compute one-sided p-values; default is two-sided

ci(#) set confidence level; default is ci(95)

saving(filename, ...) save
results to filename; or replace existing saved file

* treat(varname) must be specified.

aweights, fweights and iweights are allowed; see
weight.

Description

itsarand performs randomization tests on interrupted time series (ITS) data for a single-case AB design
and multiple-baseline AB phase designs (i.e. multiple single-case AB phase designs) and estimates p-values on
the basis of Monte Carlo simulations (see Dugard et al. [2012] and Edgington & Onghena [2007] for comprehensive discussions).

Options

treat(varname) specifies the treatment variable, whcih is typically coded as 0 for baseline observations and 1 for
observations in the intervention period. treat(varname) is required.

blmin(#) specifies the minimum number of observations to retain in the baseline period; default is 2.

trmin(#) specifies the minimum number of observations to retain in the treatment period; default is 2.

level specifies that the test statistic should be the difference between treatment and baseline period in the levels; the
default test statistic is difference between the treatment and baseline period trends.

reps(#) specifies the number of random permutations to perform. The default is 100.

seed(#) sets the random-number seed. Specifying this option is
equivalent to typing the following command prior to calling itsarand:

. set seed #

nodots suppresses display of the replication dots.

left or right requests that one-sided p-values be computed.
If left is specified, an estimate of Pr(T* < T) is produced, where
T* is the test statistic and T is its observed value. If right is
specified, an estimate of Pr(T* > T) is produced. By default, two-sided
p-values are computed; that is, Pr(|T*| > |T|) is estimated.

ci(#) specifies the confidence level, as a percentage,
for confidence intervals. The default is level(95) or as set by
set level.

saving(filename [, replace]) creates a Stata data file (.dta file) consisting of a variable containing the
replicates for the test statistic (labeled either level or trend, accordingly).

Remarks

For an effect to be interpreted as causal using a single-case AB phase design,
the intervention must be introduced at a randomly determined point in time. The permuted p-value can
then be computed by randomly reassigning the start of the intervention to all other points in time
(avoiding the first and last few time points (by specifying blmin(#) and trmin(#), respectively)
to ensure a sufficient number of observations are available in pre-intervention or intervention periods) and calculating
the desired test statistic. The permuted p-value is thus the proportion of all permutations with test statistics (i.e. difference
in the levels or trends of the outcome begtween the two periods) as extreme, or more extreme, than the test statistic of the actual sample.

The multiple-baseline design is an extension of the basic single-case ITS design in which multiple single-units each
undergo their own single-case ITSA, but the intervention is assigned in a randomly staggered way to each
unit (similar in concept to the stepped-wedge design). To compute permuted P-values, each unit is analyzed separately
(as described above for the single-case) and the results are meta-analyzed to provide an aggregate score.

The test statistic computed by itsarand is based on the basic regression model for a single-group ITSA as follows [Linden and Adams 2011; Linden 2015; Linden 2017]:

Y_t = Beta_0 + Beta_1(T) + Beta_2(X_t) + Beta_3(TX_t) (1)

Here Y_t is the aggregated outcome variable measured at each equally spaced
time point t, T is the time since the start of the study, X_t is a dummy
(indicator) variable representing the intervention (preintervention periods 0,
otherwise 1), and TX_t is an interaction term that represents the difference between
pre- and post-intervention trends.

itsarand uses the form of the regression in Equation 1 to produce the difference-in-trend
estimate (Beta_3) as the test statistic when the user specifies the trend option (this is achieved by simply not
specifying level). When there are multiple panels, Equation 1 is estimated for each unit separately
and the Beta_3 coefficients are summed across all units. When the user specifies the difference-in-levels test
statistic (by specifying level), then a regression is estimated in which the outcome Y_t is regressed on X_t alone.
In the case of multiple panels, a separate regression is estimated for each unit and the coefficients for X_t are summed.

Examples

Load data and declare the dataset as panel:

. use multibaseline, clear

. tsset id t

We run a randomization test for multiple-baselines (i.e. multiple units) and specify that there should be at least 4 observations in
both the baseline and treatment periods, and the test statistic is the difference in pre- and post-intervention levels. We save the 2000 permuted
values to a file named "test".

. itsarand y, treat( treat) saving(test) blmin(4) trmin(4) level reps(2000)

Same as above, but we specify trend as the test statistic (by not including the level option). We replace the existing file named "test".

. itsarand y, treat( treat) saving(test, replace) blmin(4) trmin(4) reps(2000)

Same as above, but we limit the analysis to a single case (ID 1), specifiy that no dots be shown, and set the seed.

. itsarand y if id==1, treat( treat) saving(test, replace) blmin(4) trmin(4) reps(2000) nodots seed(1234)

Stored results

itsarand stores the following in r():

Scalars

r(c) count when the event is true

r(n) number of nonmissing results

r(p) observed proportions

Macros

r(xstat) T(obs)

References

Dugard P, File P, Todman J. Single-case and Small-n Experimental Designs.
New York: Routledge, 2012.

Edgington ES, Onghena P. Randomization tests (4th ed.).
Boca Raton, FL: Chapman & Hall/CRC, 2007.

Linden A. 2015.
Conducting interrupted time series analysis for single and multiple group comparisons.
Stata Journal
15: 480-500.

------. 2017a.
A comprehensive set of postestimation measures to enrich interrupted time-series analysis}.
Stata Journal
17: 73-88.

------. 2017b.
Challenges to validity in single-group interrupted time series analysis.
Journal of Evaluation in Clinical Practice
23: 413-418.

------. 2017c.
Persistent threats to validity in single-group interrupted time series analysis with a crossover design.
Journal of Evaluation in Clinical Practice
23: 419-425.

------. 2018a.
A matching framework to improve causal inference in interrupted time series analysis.
Journal of Evaluation in Clinical Practice
24: 408-415.

------. 2018b.
Using permutation tests to enhance causal inference in interrupted time series analysis.
Journal of Evaluation in Clinical Practice
24: 496-501.

------. 2018c.
Combining synthetic controls and interrupted time series analysis to improve causal
inference in program evaluation.
Journal of Evaluation in Clinical Practice
24: 447-453.

------. 2018d.
Using group-based trajectory modelling to enhance causal inference in interrupted time series analysis.
Journal of Evaluation in Clinical Practice
24: 502-507.

------. 2018e.
Using forecast modelling to evaluate treatment effects in single-group interrupted time series analysis.
Journal of Evaluation in Clinical Practice
DOI: 10.1111/jep.12946

------. 2018f.
Using Randomization tests to assess treatment effects in multiple-group interrupted time series analysis.
Journal of Evaluation in Clinical Practice
DOI: 10.1111/jep.12995

Linden A, Yarnold PR. 2018
Using machine learning to evaluate treatment effects in multiple-group interrupted time series analysis.
Journal of Evaluation in Clinical Practice
DOI: 10.1111/jep.12966

Linden A, Adams JL. 2011.
Applying a propensity-score based weighting model to interrupted time series data: Improving causal inference in program evaluation.
Journal of Evaluation in Clinical Practice
17: 1231-1238.

Citation of itsarand

itsarand is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden, Ariel (2018). ITSARAND: Stata module for conducting randomization tests for single-case and multiple-baseline AB phase designs

Author

Ariel Linden

Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Help: permute, statsby, itsa (if installed)