*! version 1.0.0 12Oct2023

Title

distill -- an analytical approach to assess heterogeneous treatment effects in randomized controlled trials

Syntax

distill depvar [if] [in]
,
treat(varname)
pscore(varname)
[
nquantiles(#)
level(#)
figure[(twoway_options)]
glm_model_options
]

treat(varname) required treat() must be binary and coded 0 for the control group and 1 for the treatment group

pscore(varname) required The propensity score, indicating the probability of being in the treatment group

nquantiles(#) number of desired quantile categories of the propensity score; default is 5

level(#) set confidence level; default is level(95)

figure[(twoway_options)] produces a forest plot containing the model estimates for each strata. Specifying figure without options uses the default graph settings.

glm_model_options allows all available glm options for the outcome model

by is allowed with distill; see [D] by.

Description

distill implements the "Distillation Method" (Adams et al. 2022) to estimate treatment effects within a desired number of strata (quantiles)
of the propensity score to determine whether a randomized controlled trial (RCT) may have produced heterogeneous treatment effects. Adams et al. (2022)
found treatment effects in the higher strata, corresponding to subjects who were more likely to particate/engage in the intervention.

Remarks

The "Distillation Method" capitalizes on the frequently observed correlation between the probability of subjects' participation or
engagement in the intervention and the magnitude of benefit they experience. The method involves three stages: first, it uses baseline
covariates to generate predicted probabilities of participation. Next, these are used to produce nested subsamples of the randomized
intervention and control groups that are more concentrated with subjects who were likely to participate/engage. Finally, for the outcomes of interest,
standard statistical methods are used to re-evaluate intervention effectiveness in these concentrated subsets (Adams et al. 2022).

distill performs the second and third stages of the "Distillation Method." The user performs the first stage (estimating the propensity score)
prior to implementing distill and then specifies the propensity score in pscore(). Next, distill performs the second stage by creating
propensity score strata (quantiles) as specified in nquantiles(). Finally, distill performs the third stage by estimating
treatment effects for each of those strata using glm. The user may choose from all available options in glm to adjust the
specifications for the outcome model.

Options

treat(varname) is a binary variable indicating the treatment status of each subject. treat must be coded 0 for the
control group and 1 for the treatment group; treat is required.

pscore(varname) is the propensity score, indicating the probability of being in the treatment group; pscore is required.

nquantiles(#) desired number of quantile categories of the propensity score to be created. The variable created will be named "_xtile", and will be
replaced every time that distill is implemented; The default is 5 quantiles.

level(#) specifies the confidence level, as a percentage, for confidence intervals. The default is level(95) or as set by set level.

figure[(twoway_options)] produces a forest plot containing the model estimates for each strata.
Specifying figure without options uses the default graph settings.

glm_model_options allows all available glm options for the outcome model.

Examples

Setup

. use distill_example.dta, clear

Estimate treatment effects for a binary outcome within 10 strata of the propensity score. We observe statistically significant effects in the highest strata (10).

. distill y1, treat(treatment) pscore(_ps) nq(10) family(binomial) link(logit)

Same as above but we specify "eform" to provide the results in exponentiated form, and request a figure.

. distill y1, treat(treatment) pscore(_ps) nq(10) family(binomial) link(logit) eform fig

In order to examine the effects in the 10th strata, we first rename the existing strata variable ("_xtile") and then rerun distill
on only that strata (creating a new strata variable with only 5 quantiles).

. rename _xtile orig_xtile

. distill y1 if orig_xtile==10 , treat(treatment) pscore(_ps) nq(5) family(binomial) link(logit) eform fig

Estimate treatment effects for a count outcome within 10 strata of the propensity score.

. distill y2 , treat(treatment) pscore(_ps) nq(10) family(nbinomial) link(log) fig

Same as above, but specify eform and a 99% confidence interval

. distill y2 , treat(treatment) pscore(_ps) nq(10) family(nbinomial) link(log) fig eform level(99)

Stored results

distill stores the following in r():

Scalars

r(nq) number of quantiles of the propensity score

r(N) number of observations

Matrices

r(table) matrix containing the coefficients with their standard errors, test statistics, p-values, and confidence intervals

References

Adams, J. L., Davis, A. C., Schneider, E. C., Hull, M. M. and E. A. McGlynn. (2022). The distillation method:
A novel approach for analyzing randomized trials when exposure to the intervention is diluted.
Health Services Research 57: 1361-1369.

Citation of distill

distill is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden, A. and J. L. Adams (2023). DISTILL: Stata module to assess heterogeneous treatment effects in randomized controlled trials

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

John L. Adams

Center for Effectiveness and Safety Research,

Kaiser Permanente, Pasadena, California, USA

Also see

Online: xtile, glm