*! version 1.0.2 05August2017
help stbrier

Title

stbrier -- Brier score for censored time-to-event (survival) data

Syntax

stbrier varlist [if] [in]
,
btime(#)
[ distribution(string)
compete(crvar[==numlist])
ipcw(varlist)
gen(string)
model_options
]

btime(#) specify the timepoint at which the Brier score should be computed; required

distribution(string) specify the survival distribution for an streg model; see streg for available distributions

compete(crvar[==numlist]) specify competing-risks event(s) for a competing-risk model; see stcrreg

ipcw(varlist) specify covariates for estimating the inverse probability of censoring weights (IPCW)

gen(string) generate a variable containing the observation-level stbrier scores

model_options specify all available options for streg when the model option is chosen;
all available options for stcrreg when the compete option is chosen; otherwise all available options for stcox

You must stset your data before using stbrier; see [ST] stset.

varlist may contain factor variables; see fvvarlist.

fweights, iweights, and pweights may be specified using
stset; see [ST] stset. Weights are not supported with
efron and exactp.

Description

stbrier computes the Brier score for risk prediction
models in survival analysis based on right censored data by weighting
individuals by their inverse probability of being uncensored [Graf et
al. 1999; Gerds & Schumacher 2006; Binder et al. 2009] (see brier
for non-censored data). In settings without competing risks
stbrier can compute the Brier score of a parametric
(streg) or semi-parametric Cox regression model (stcox),
and in the presence of competing risks, the Brier score of a Fine-Gray
regression model (stcrreg). To serve as a benchmark (null model),
stbrier can be computed without covariates [Gerds et
al. 2008]. Additionally, the user can use a simple wrapper program to
compute the integrated Brier score [Graf et al.1999] across all values
of btime (see the example below).

The Brier score is a quadratic scoring rule defined as the
average of the squared differences between actual binary outcomes Y at
the prediction horizon btime and probabilities p. For
each individual the squared difference is calculated as (Y -
p)^2, where p is the predicted risk of the event until
btime based on the baseline covariates of this individual and Y
is 1 if the event occured before btime for this individual, and
0 otherwise. The Brier score can range from 0 to 1, the lower the
better. A useful model should have a value lower than 0.25, because
this can be achieved by a non-informative model which predicts a 50%
risk of the event for all subjects, i.e. (Y - 0.50)^2. When the
population average outcome risk at the prediction horizon btime
is lower or higher than 50%, then the Brier score of the benchmark
null model (without covariates) will be lower than 0.25 and a good
regression model which uses the covariates should outperform the null
model.

Options

btime(#) specifies the timepoint (prediction horizon) at
which the Brier score should be computed. Any btime value within
the range of the follow-up times can be specified. An error is
produced if btime is either not specified, or if the specified btime falls
beyond the range of the data.

distribution(string) specifies the survival distribution for an streg model;
see streg for available distributions. stcox is implemented when
distribution() and compete() are not specified.

compete(crvar[==numlist]) specifies the events
that are associated with failure due to competing risks (see stcrreg).

If compete(crvar) is specified, crvar is interpreted as an
indicator variable; any nonzero, nonmissing values are interpreted as
representing competing events.

If compete(crvar==numlist) is specified, records with crvar taking
on any of the values in numlist are assumed to be competing
events.

The syntax for compete() is the same as that for
stset's failure() option. Use stset, failure() to
specify the event of interest, that is, the event you wish to predict.
Use stcrreg, compete() to specify the event or events that
compete with the failure event of interest.

ipcw(varlist) specifies the covariates
to include in estimating the inverse probability of censoring weight
model.

gen generates a variable containing the observation-level stbrier scores.

model_options specifies all available options for streg when the
model option is chosen; stcrreg when the compete option is chosen;
otherwise all available options for stcox.

Examples with semi-parametric Cox regression

Setup

. use GBSG2.dta, clear

Declare data to be survival-time data

. stset time, fail(cens)

Estimate the stbrier score with no covariates (and efron option for ties) at time 1990, to serve as a benchmark for other models (this
is not equal, but asymptotically equivalent, to the Kaplan-Meier estimator)

. stbrier , bt(1990) efron

Estimate the stbrier score at the median survival time, specifying the efron option for ties, and generate a variable for
individual-level stbrier scores

. stsum

. stbrier i.horth ib2.menostat i.tgrade age tsize pnodes, efron ipcw(i.horth ib2.menostat i.tgrade age tsize pnodes) gen(brier) bt(1807)

Estimate the stbrier score with strata

. stbrier ib2.menostat i.tgrade age tsize pnodes, bt(1990) strata(horth) efron ipcw(ib2.menostat i.tgrade age tsize pnodes)

Example with parametric survival regression

Estimate the stbrier score at time 2000 using an exponential distribution for the event time and no covariates in either the risk model or IPCW model

. stbrier , d(expon) btime(2000)

Estimate the stbrier score using survival regression with an exponential distribution, specifiying certain covariates in the IPCW model, and
generating a variable for individual-level stbrier scores

. stbrier i.horth ib2.menostat i.tgrade age tsize pnodes, d(expon) ipcw(i.tgrade age tsize) gen(brier) btime(2000)

Estimate the stbrier score using a Weibull regression model, and generating a variable for individual-level stbrier
scores

. stbrier i.horth ib2.menostat i.tgrade age tsize pnodes, d(weib) btime(2000) ipcw(i.tgrade age tsize) gen(brier)

Example with competing-risk regression

Setup

. use melanoma.dta, clear

Declare data to be time-to-event data where the event of interest is status==1

. stset time, fail(status==1)

Estimate the stbrier score at time 1000 using Fine-Gray regression, where the competing risk is status==2 without covariates in
the risk prediction model. We use covariates for the inverse probability of censoring weights

. stbrier , comp(status==2) btime(1000) ipcw(age i.sex logthick i.ulcer) gen(brier)

Same as above but now with covariates

. stbrier age i.sex logthick i.ulcer, comp(status==2) btime(1000) ipcw(age i.sex logthick i.ulcer) gen(brier)

Computing the integrated Brier score

Setup

. use GBSG2.dta, clear

Declare data to be survival-time data

. stset time, fail(cens)

Run loop over all time values in the data and generate a line graph

levelsof _t, local(times)
foreach i of local times {
stbrier i.horth ib2.menostat i.tgrade age tsize pnodes, efron ipcw(i.horth ib2.menostat i.tgrade age tsize pnodes) btime(`i') gen(br`i')
}
collapse (mean) br*
gen id = _n
reshape long br , i(id) j(_t)
line br _t if _t <2100

Saved results

By default, stbrier returns the following results, which can be displayed by typing return list after
stbrier is finished (see return).

Scalars

r(brier) mean Brier score

Matrices

r(table) estimates from mean table

References

Binder H, Allignol A, Schumacher M, Beyersmann J. Boosting for high-dimensional time-to-event data with competing risks.
Bioinformatics 2009;25(7):890-896.

Gerds TA, Cai T, Schumacher M. The performance of risk prediction models.
Biom J 2008;50:457-479.

Gerds TA, Scheike TH, Blanche P, Ozenne B. riskRegression: Risk Regression Models and Prediction Scores
for Survival Analysis with Competing Risks. R package version 1.3.7. (2017). https://CRAN.R-project.org/package=riskRegression

Gerds TA, Schumacher M. Consistent estimation of the expected Brier score in general survival models
with right-censored event times.
Biom J 2006;48:1029-1040.

Graf E, Schmoor C, Sauerbrei W, Schumacher M. Assessment and comparison of prognostic classification
schemes for survival data.
Stat in Med 1999;18:2529-2545.

Schumacher M, Graf E, Gerds T. How to assess prognostic models for survival data: a case study in oncology.
Methods Inf Med 2003;42:564-571.

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW.
Assessing the performance of prediction models: a framework for some traditional and novel measures.
Epidemiology 2010;21:128-138.

Citation of stbrier

stbrier is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden A, Gerds TA, Huber C. (2017). STBRIER: Stata module for estimating the Brier score for survival (censored) data. https://ideas.repec.org/c/boc/bocode/s458368.html

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

http://www.lindenconsulting.org

Thomas A Gerds

Professor, University of Copenhagen

tag@biostat.ku.dk

Chuck Huber

Senior Statistician, StataCorp, LLC

chuber@stata.com

Acknowledgments

We wish to thank Isabel Canette, Yulia Marchenko, and Nick J Cox for their support while developing stbrier.

Also see

Online: stcox, stcox postestimation, streg, streg postestimation,
stcrreg, stcrreg postestimation, sts list, stsum, brier