*! version 1.0.0 26Sep2024

Title

csumchart -- Cumulative sum (CUSUM) charts for monitoring clinical performance

Syntax

Non-risk-adjusted CUSUM

csum
yvar
xvar
[if] [in]
,
blprob(#)
[
odds(#)
limit(#)
reps(#)
seed(#)
centile(#)
wt(varname)
replace
nograph
twoway_options
]

Compute control limit for risk-adjusted CUSUM

csumralimit
riskscore
[if] [in]
[,
odds(#)
reps(#)
seed(#)
centile(#)
local(macname)
]

Risk-adjusted CUSUM

csumra
yvar
xvar
[if] [in]
,
risk(varname)
limit(#)
[
odds(#)
wt(varname)
replace
nograph
twoway_options
]

yvar variable must be binary

riskscore variable must contain values between 0 and 1

csum
blprob(#) baseline probability of failure (e.g. death, hospitalization); blprob() is required

odds(#) odds of failure; default is to detect a doubling of the odds of failure odds(2)

limit(#) control limit for signaling a change in performance; default is for csum to compute the limit using simulation

reps(#) number of replications to be performed when computing limit(); default is reps(50)

seed(#) set random-number seed to # when computing limit() using simulation

centile(#) set centile level for computing limit() using simulation; default is centile(95)

wt(varname) weights used for computing CUSUM; default is for csum to compute the weights

replace replace variables created by csum

nograph suppresses the plot

twoway_options any options documented in [G-3] twoway_options

csumralimit
odds(#) odds of failure; default is to detect a doubling of the odds of failure odds(2)

reps(#) number of replications to be performed when computing limit(); default is reps(50)

seed(#) set random-number seed to # when computing limit() using simulation

centile(#) set centile level for computing limit() using simulation; default is centile(95)

local(macname) stores the control limit in local macro macname, making it accessible for later use.

csumra
risk(varname) variable containing the risk-adjusted predictions of the yvar; risk() is required

limit(#) control limit for signaling a change in performance; the limit(#) can be computed using
csumralimit; limit(#) is required

odds(#) odds of failure; default is to detect a doubling of the odds of failure odds(2)

wt(varname) weights used for computing CUSUM; default is for csumra to compute the weights

replace replace variables created by csumra

nograph suppresses the plot

twoway_options any options documented in [G-3] twoway_options

Description

The csumchart package produces risk-adjusted and non-risk-adjusted cumulative sum (CUSUM) charts for monitoring clinical
performance over time, by signaling if sufficient evidence has accumulated to detect a change in the failure rate of
the outcome (Steiner et al. 2000). The CUSUM procedure requires a binary yvar (where a value of 1 indicates failure
and 0 indicates success), and a sequential xvar which typically represents sequential patient numbers or the passage
of time. In the case of risk-adjusted outcomes, an additional variable containing the riskscores is required.
Process deterioration is detected when the CUSUM exceeds the a priori control limit in the positive direction.
Similarly, process improvement is detected when the CUSUM exceeds the a priori control limit in the negative direction. See
Steiner et al. (2000) for a comprehensive discussion about how CUSUM charts are produced and interpreted.

Options

blprob(#) specifies the baseline rate of failure; blprob() is required for csum.

risk(varname) specifies the risk-adjusted predictions of the yvar; risk(varname) is required for csumra.

odds(#) specifies the odds ratio used to detect a change in the outcome (e.g. death). For example, odds(2) is designed to detect
a doubling in the odds of failure (process deterioration), whereas odds(.50) is designed to detect halving the odds of failure
(process improvement).

limit(#) specifies the control limit. The point on the xvar where the CUSUM first exceeds the control limit is the start of
the process being "out of control". The control limit may be guided by external expectations or regulatory requirements, or it can be
determined empirically using simulation. When limit(#) is not specified in csum, or when implementing csumralimit to
compute the limit for risk-adjusted data (to be used in csumra), random data are generated based on existing sample
size, specified odds ratio, and baseline probability of failure (or riskscores). The CUSUM is computed and the maximum CUSUM value is stored. This
process is repeated the number of times specified in reps(). Finally, the value corresponding to the specified centile()
is computed. This value serves as the control limit. While a positive control limit is used to detect an increase in the failure rate,
a negative control limit is used to detect a decrease in the failure rate (a negative control limit is used in congunction
with a odds() below 1.0).

reps(#) the number of replications to be performed when computing limit() or csumralimit(); default is reps(50).

seed(#) set random-number seed to # when computing limit() or csumralimit() using simulation.

centile(#) set centile level for computing limit() or csumralimit() using simulation. The default is centile(95)
for assessing process deterioration and centile(5) for assessing process improvement.

local(macname) stores the control limit computed by csumralimit in local macro macname within the calling program's space,
thereby making the control limit accessible after csumralimit has finished. This is helpful for later use with csumra.

wt(varname) specifies the weights used for computing CUSUM. The default is for csum and csumra to compute the weights.

replace replaces the variables in the file created by csum and csumra. Three variables are created: _wt which is the
weight generated by csum and csumra or a replicate of those specified by the user in wt(); _ct which is the CUSUM; and
_signal which indicates at which point on the xvar the CUSUM exceeds the control limit.

nograph suppresses the plot.

twoway_options any options documented in [G-3] twoway_options

Examples

(1) Unadjusted CUSUM

Detecting process deterioration (increase in failure rate):

create synthetic data

. clear

. set obs 1000

. gen ptid = _n

. label var ptid "Patient number"

. set seed 1234

. gen y = rbinomial(1,.40)

. label var y "Deaths"

We generate a CUSUM chart to assess whether there was an increased failure
rate over a sequential number of observations. We set the baseline failure rate
to 0.20 and we set the odds ratio to 2 in order to detect a doubling in the odds of
failure. We allow csum to estimate the control limit at the 95th centile using 100
repetitions. We see that the failure rate exceeds the control limit at patient number 42.

. csum y ptid, blp(0.20) seed(1234) reps(100) centile(95) replace odds(2)

Detecting process improvement (decrease in failure rate):

create synthetic data

. clear

. set obs 1000

. gen ptid = _n

. label var ptid "Patient number"

. set seed 1234

. gen y = rbinomial(1,.20)

. label var y "Deaths"

We generate a CUSUM chart to assess whether there was an decreased failure
rate over a sequential number of observations. We set the baseline failure rate
to 0.40 and we set the odds ratio to 0.50 in order to detect a halving in the odds of
failure. We allow csum to estimate the control limit at the 95th centile using 100
repetitions. We see that the failure rate decreases beyond the control limit at patient
number 81.

. csum y ptid, blp(0.40) seed(1234) reps(100) centile(95) replace odds(0.50)

Here we manually set the limt to -20.

. csum y ptid, blp(0.40) limit(-20) replace odds(.5)

(2) Adjusted CUSUM

Compute control limit using csumralimit on baseline data for assessing process deterioration (increase in failure rate):

create synthetic baseline data

. clear

. set obs 1000

. set seed 1234

. gen ptid = _n

. label var ptid "Patient number"

. gen y = rbinomial(1,0.40)

. label var y "Deaths"

. gen riskscore = cond(y,runiform(0.51,1),runiform(0,0.50))

. label var riskscore "Risk-adjusted probability of death"

we compute the control limit for a doubling of the odds of death on these baseline data (this
produces a control limit of 6.260). We save the control limit as a local macro named limit1 so
that it can be used afterwards in csumra.

. csumralimit riskscore , seed(1234) odds(2) reps(1000) local(limit1)

Detecting process deterioration (increase in failure rate) on followup data:

create synthetic followup data

. clear

. set obs 1000

. set seed 1234

. gen ptid = _n

. label var ptid "Patient number"

. gen y = rbinomial(1,.70)

. gen riskscore = cond(y,runiform(0.51,1),runiform(0,0.50))

We generate a CUSUM chart to assess whether there was an increased failure
rate over a sequential number of observations. We set the odds ratio to 2 in order
to detect a doubling in the odds of failure. We specify the control limit using the
local macro (named limit1) that was computed in the previous step. We see
that the failure rate exceeds the control limit at patient 249.

. csumra y ptid , risk(riskscore) limit(`limit1') odds(2) replace

Compute control limit using csumralimit on baseline data for assessing process improvement (decrease in failure rate):

create synthetic followup data

. clear

. set obs 1000

. set seed 1234

. gen ptid = _n

. label var ptid "Patient number"

. gen y = rbinomial(1,.60)

. gen riskscore = cond(y,runiform(0.51,1),runiform(0,0.50))

we compute the control limit for a halving of the odds of death on these baseline data which
produces a control limit of -6.300. We save this value as a local macro named limit2

. csumralimit riskscore , seed(1234) odds(0.50) reps(1000) local(limit2)

Detecting process improvement (decrease in failure rate) on followup data:

create synthetic followup data

. clear

. set obs 1000

. set seed 1234

. gen ptid = _n

. label var ptid "Patient number"

. gen y = rbinomial(1,.30)

. gen riskscore = cond(y,runiform(0.51,1),runiform(0,0.50))

We generate a CUSUM chart to assess whether there was a decreased failure
rate over a sequential number of observations. We set the odds ratio to 0.50 in order
to detect a halving in the odds of failure. We specify the control limit using the
local macro named limit2, which was computed in the previous step.
We see that the failure rate exceeds the control limit at patient 164.

. csumra y ptid , risk(riskscore) limit(`limit2') odds(0.50) replace

Stored results

csum, csumra and csumralimit store the following in r():

Scalars

r(limit) the control limit

References

Steiner, S. H., Cook, R. J., Farewell, V. T. & T. Treasure. 2000. Monitoring surgical performance using risk-adjusted cumulative sum charts.
Biostatistics 1: 441-452.

Citation of csumchart

csumchart is not an official Stata command. It is a free contribution
to the research community, like a paper. Please cite it as such:

Linden A. (2024). CSUMCHART: Stata module to compute cumulative sum (CUSUM) charts for monitoring clinical performance.

Authors

Ariel Linden

President, Linden Consulting Group, LLC

alinden@lindenconsulting.org

Also see

Online: cusum