*! version 1.0.1, 2021-02-15

Title

mcwr -- Markov chain with rewards calculations

Syntax

Check that data set is set up correctly

mcwr check [, ij skipexit noexit]

Change from-state and target state index convention of variable names

mcwr switch [, ij noisily]

Edit or check last (exit) row of data set

mcwr exit , age(ageval, [replace update]) [rewards(rwval)]

Generate (additional) transition probability or rewards variables

mcwr genvars [, timing(timespec, ) nop nor order]

Calculate state and overall expectancies

mcwr expectancies [, initprop(initspec) timing(timespec, ) keep]

View matrices involved in the calculations in the data browser

mcwr matbrowse {e|P|F|R#} [, format(%fmt) norestore noshow]

List matrices involved in the calculations

mcwr matlist [, matlistopts]

subcmd check

ij data set is in ij-format

skipexit do not check last row of data set

noexit exit row is missing in data set

subcmd switch

ji data set will always be in ji-format after command conclusion

noisily display verbose error messages

subcmd exit

age(ageval, [replace update]) age of exit row of data set

rewards(rwval) rewards value to use for exit transitions

subcmd genvars

timing(timespec, ) specify rewards for from and target states

order order variables alphabetically according to varlist 'age p* r*'

nop do not generate any p-variables

nor do not generate any r-variables

subcmd expectancies

initprop(initspec) initial state fractions at baseline age

timing(timespec, ) see above

keep keep newly generated p/r-variables necessary for the computation in the data set

subcmd matbrowse

format(%fmt) format numbers accoding to %fmt

norestore do not restore data set but keep matrix data in memory

noshow do not open browser with matrix data

subcmd matlist

matlistopts all options allowed by matlist

Description and Overview

mcwr implements Markov chain with rewards calculations. It accompanies the article

Schneider / Myrskylä / van Raalte (2021):
Flexible Transition Timing in Discrete-Time Multistate Life Tables
Using Markov Chains with Rewards. MPIDR Working Paper, February 2021.
Available here.

Below, this article will be referred to as "the paper" and its appendix as "the appendix".

The main subcommand, which does the actual calculations, is mcwr expectancies. The other subcommands
either check the data set for consistency, carry out data management tasks, or display results.

Running mcwr demands a very specific data set in memory (called mcwr data set,
laid out in section Data setup).
Several data management commands aide in creating such a specific data set.

The main data management subcommand is mcwr genvars, which generates
p- and r- variables that are missing from the data set.
These variables specify transition probabilities and rewards.
Other data management commands check the data set for consistency (mcwr check),
transform variable names (mcwr switch), or check or create an exit row (mcwr exit).

Two commands display results: mcwr matbrowse grabs results matrices and displays them
in Stata's data browser. This greatly facilitates looking at large matrices. mcwr matlist
is a simple wrapper for matlist and displays results matrices in a nice format.

Abbreviations and definitions used in this help entry

Abbreviations

MCWR Markov chain with rewards

Definitions

p-variables Variables whose name is of the format 'p##'.
They contain transition probability data.
The numbers encode from and target state.
The default for this number pair is the ji-format (see below).
For example, variable 'p12' has transition probabilities from-state 1 to state 2.

r-variables Variables whose name is of the format 'r#_##'.
They contain rewards data.
The first number encodes the state that receives the reward.
The second and third numbers encode from and target state.
The default for this number pair is the ji-format (see below).

ji/ij-format Variable (names) are said to be in ji-format if the first number of the variable name
specifies the from-state and the second number specifies the target state.
The index j always denotes the from-state.
Consequently, variable (names) in ij-format state the target state first and then the from-state.

from/target/initial states From-state and target state are the two states that are connected
by a state transition. The initial state is the state at baseline age.

mcwr data set A data set that fulfills all requirements for mcwr to run on it
without error. The structure of the data set is set forth in section Data setup.

Subcommand Detail and Options

Auxiliary data management subcommands

mcwr check

Run mcwr check to check whether your data set is a valid mcwr data set or whether you have
to modify something. It is also run internally by the other subcommands, so you are always safe
not to be using incorrectly set up data.

A second use of the subcommand is to gather comprehensive information about existing and missing model
variables, and more. See section Saved results.

Options

ij data set is in ij-format.

skipexit do not check the last row of the data set.

noexit the exit row is missing in the data set.

mcwr switch

The appendix notation follows the ij-notation, where the first index refers to the
target state and the second index to the from-state. This has the advantage of conforming
with the conventions of matrix algebra.
p- and r-variables of mcwr data sets, however, generally follow the ji-convention.
The advantage of this is that sorting the variables alphabetically results in a sensible
and intuitive ordering. Therefore, your data set variables are required to follow the ji-convention.
The convenience command mcwr switch allows you to switch between the two conventions.
If you have a consistent data set in ij-format, running this command will rename
variables according to the ji-convention.
Your data must be in ji-format before you can run any other of the mcwr subcommands that rely on
the data set in memory.
As a brief example, we load data in ji-format, then switch to ij-format and back:

(learn how to download the example data; after the download:)

(click to load example data)

. list in 1/3

. mcwr switch

. list in 1/3

. mcwr switch

. list in 1/3

Options

ji data set will always be in ji-format after command conclusion.
That is, if the data are in ij-format they will be converted to ji-format, and will be left
untouched otherwise.

noisily will display verbose error messages. This is useful if the command tells you that
it can neither find a consistent ji data set nor a consistent ij data set. The error messages
under option noisily may give you a clue about the source of the error.

mcwr exit

mcwr requires that all data points that do not enter matrix calculations be set to missing
in order to avoid incorrectly set up data. This rule makes the last (exit) row of the mcwr data set
somewhat tedious to manage.
The convenience command mcwr exit makes it easier to create or edit the last (exit) row
of the data set.

Options

age(ageval, [replace update]) specifies the age of the exit row in the data set. It may or may not exist.
It may not be smaller than the largest age in the data set.

If ageval corresponds to the largest age in the data set,
suboption replace must be specified. The values of the corresponding row are replaced.
Exit transition values for p-variables are set to 1.
Exit transition values for r-variables are left as-is if they are non-missing and suboption update
is not used. Otherwise they are set to rwval.
Values of all other transitions are set to missing.

If ageval is larger than the largest age in the data set, a new row will be inserted.
Exit transitions are set to 1 for p-variables and to rwval for r-variables.
Values of all other transitions are set to missing.

rewards(rwval) determines the rewards value for exit transitions.
Its default value (when left unspecified) is 0.

Main subcommands

mcwr genvars

The main purpose of this subcommand is to generate rewards variables (r-variables).
It examines existing p- and r-variables, determines the implied full set of states,
and generates any missing variables that are missing from the data set.
It interacts flexibly with existing r-variables:
You can leave them unchanged or have them replaced.

An effective way to create r-variables may be to generate a full set of r-variables
using mcwr genvars and then edit them where necessary.
This is illustrated under Examples.

Options

timing(timespec, ) specifies how rewards are distributed to
from and target states. It is required if option nor is not used.
timespec can be one of 'bop', 'mid', and 'eop', which stands for
'beginning-of-period', 'mid-period', and 'end-of period', respectively.
Alternatively, it can also be a number in the interval [0 1] that specifies
the fraction of the interval that goes to the from-state.
Values of 0, 0.5, and 1 correspond to 'beginning-of-period', 'mid-period', and
'end-of-period', respectively.

add will leave existing r-variables unchanged.

replace will replace them.

nop does not generate any p-variables. By default, all missing p-variables are
generated. Since existing p-variables must satisfy the sums-to-unity condition, only
p-variables that are (by implication) all-zero can be missing. mcwr expectancies will
run whether such redundant variables exist or not.

nor does not generate any r-variables.

order orders variables alphabetically according to the varlist 'age p* r*'.

mcwr expectancies

This is the subcommand that does the actual calculations.
It returns its results in r(), most notably in r() matrices.

Options

initprop(initspec) supplies information about the initial state fractions at baseline age.
initspec can either be a numlist of values in the interval [0 1] which sum to 1.
Numbers are taken to correspond to from-states in an sequential order. For example,
If your model contains from-states 1 2 7, initspec must consist of three numbers,
specifying the initial proportion of each state in turn.
Alternatively, initspec may also be the name of a Stata matrix with one row
and columns according to the number of from-states in the model.
Its column names must either be the numeric, sequential from-states ('1 2 7' in the
above example) or their label equivalents, determined by value label MCWR
(see section The MCWR value label).

timing(timespec, ) - see mcwr genvars above.

In most cases, you do not have to create a full set
of r-variables using subcommand genvars.
r-variables that correspond to timings that can be accommodated
by the timing() option can be created automatically, behind the scenes.
You do so by specifying the timing() option in subcommand expectancies
instead in the subcommand genvars.
Any missing r-variables (and p-variables, for that matter) will then be created behind
the scenes before calculations are done.
They will get deleted before the command concludes.

r-variables with more complicated timings have to be created explicitly before running
subcommand expectancies.

keep will keep any temporarily generated p- and r-variables that are
necessary for the computation. The default is to drop them
before command conclusion.

Results display subcommands

mcwr matbrowse

This subcommand exists to facilitate looking at large matrices. The best way in
Stata to look at large matrices is to convert them to a data set and then
browse the matrix data. mcwr matbrowse does this, and upon
any keystroke restores the data that had been in memory before.

It only works on mcwr results matrices (e, P, F, R#) and you must specify
one of them in the command statement.
It first looks for the corresponding matrix in r() (e.g., r(P)) and, if it
cannot find the matrix there, under its regular matrix name (e.g., P).

Options

format(%fmt) formats variables accoding to %fmt.

norestore does not restore the previous data set but keeps the matrix data in memory.

noshow transfers the matrix data to the data in memory but does not open
the data browser.

mcwr matlist

This subcommand is a simple wrapper for matlist and separates states by lines
when displaying results matrices.

Options

matlistopts all options allowed by matlist

Remarks

Remarks are presented under the following headings:

Data setup
The MCWR value label
Limitations
Life table and open age intervals
Other community-contributed commands related to multistate models

Data setup

mcwr requires the transition data to be in a very specific format, called 'mcwr data set' in this help entry. To give you a quick idea of what
is required, clear your data in memory and click here to load an example data set. Then list its
beginning and ending rows:

list if !inrange(age, 56, 105), noobs sep(6)

In general, the following rules and conventions apply:

- Transition probabilities are specified in p-variables.
The default convention for specifying from and target states is the ji-format:
The first number encodes the from-state, the second number the target state.

- Rewards are specified in r-variables.
The numbers occuring in the variable names specify the rewarded state, the from-state, and the
target state, respectively.

- A maximum number of 9 states (including the absorbing state) is allowed.
States must be encoded using numbers 1-9. 0 is not allowed.

- Only a single absorbing state is allowed. It is encoded by the highest
number occuring for all states. In the example, 5 is the aborbing state.

- States can be non-contiguous. In the example, states 2 and 3 are missing
from the model. The states of the model are 1, 4, 5.

- Transition probabilities must sum to 1. For example, columns p11, p14, p15 sum to 1.

- As long as the sums-to-unity condition is satisfied, not all p-variables must be present.
In the example, variable p41 is missing, and all-zero by implication.

- Age (or, more general, time) is specified in a variable called 'age'.

- Irregular age intervals are allowed. The Examples section
illustrates this with a life table on a 'demographic' 5-year age grid (childhood age intervals are shorter).

- The first row of the data set specifies the baseline age. All p- and r-variables must have
missing values in the first row.

- The last row of the data set corresponds to the exit age. At this age, all subjects
are required to die (enter the absorbing state). In the example, variables p15 and p45 are set
to 1 at age 111. All other transitions at the exit age must be set to missing.

- In general, whenever values of a mcwr data set are certain to not enter matrix calculations,
they must be set to missing in the data set. Conversely, data points that are certain to enter
matrix calculations must never be missing.

- The first transition takes place in row 2 (age 51 in the example).
It is important to keep in mind that age specifies a point in time, not an interval.
It is the point in time when the subject turns 51.

- At this point, rewards can be distributed for the previous age (interval).
Standard Markov chain calculations would distribute occupancy times end-of-period
(the transition takes place at exact age 51).
By contrast, in the example we distribute time rewards and assume mid-period transitions, i.e. we assume
that state transitions take place at ages 50.5, 51.5, etc. The reward for state 1 of the 1->4 transition
at age 51 is specified to be 0.5. This covers the period [50 50.5).
The reward for state 4 of the 1->4 transition
at age 51 is also specified to be 0.5. This covers the period [50.5 51). From the same logic,
staying in the same state carries a reward of 1. It is made up of rewards of 0.5 for each one
of the periods [50 50.5) and [50.5 51), respectively.

- Rewards can only flow to from or target states.
For example, a variable 'r3_12' in the data set will be flagged as an error.

- Rewards can only flow to transient states.
In the example, where the absorbing state is 5, a variable 'r5_45' will generate an error.

- Rewards can take on any numeric value, including negative ones.

- You may have additional variables (variables other than p- and r-variables) in the data set.
They will be ignored.

As a minor remark on the example data set,
it is derived from the SHARE application in the paper. In that
application, state 1 is left with certainty when turning 71, with no return possible.
The values shown in the data set for p14 and p15 for ages 71 and higher are therefore
immaterial, given that this state is never occupied at these ages.

The MCWR value label

If the value label 'MCWR' is present, its mappings are used for more convenient input options,
labelling r()-output, and variable names.
No consistency checks on the value label definition are performed, however.

Limitations

At the risk of redundancy, here are the limitations that the command places on the model setup:

- The maximum number of states is 9 (including the absorbing state).

- Only a single absorbing state is allowed.

- Rewards can only flow to from or target states.

Moreover, the code of the command has been written with user convenience in mind, which
took its toll on efficiency. So, it is somewhat slow.
To give you a rough idea, using a 4GHz single core computer to run the command
on 10,000 mcwr data sets (110 observations, 3 transient states) took around 20 minutes
(0.12 seconds per iteration).

Life tables and open age intervals

The last age group of life tables is frequently an open interval.
What does this mean for mcwr data sets?
In the present context, the important thing to recognize is that mcwr expectancies
must have access to the proper ax value (Chiang's a).
The best way to achieve this is to set exit age (certain transition into
death) to some number greater than the previous age.
It does not really matter which age you choose as long as you assign the
ax value to the correct rewards variable.
The Examples section below illustrates this.

Other community-contributed commands related to multistate models

Recent commands related to multistate modelling include
multistate (Crowther, Michael J., and Paul C. Lambert (2017)),
mstatecox (Metzger, Shawna K., and Benjamin T. Jones (2018)), and
mslt (Muniz, Jerônimo Oliveira (2020)).

Examples

Downloading example data sets: You need to download the example
data sets first in order to
execute the examples below. Do

so

by

. net get mcwr

This will download three small Stata files (total 8KB) to your working directory.
The examples will either be based on these files
directly or on small modifications of them.
The appropriate example data will be stored in memory by the links

below

that read "(click to load example data)".

The first example uses a subset of the data from the retirement example in the paper.
The example data set only contains transition variables.
We verify that the data set is suitable for mcwr and gather some model information

(click to load example data)

. mcwr check

. return list

r(s_frm) tells us that the from-states are 1 2 4, and r(s_abs) indicates 5 to be
the absorbing state. There are 61 age classes in the model, which we can see from r(numages).
Given initial proportions, we can immediately calculate expectancies with different timing assumptions:

. mcwr expectancies , timing(mid) initprop(0.95 0.04 0.01)

. mcwr expectancies , timing(eop) initprop(0.95 0.04 0.01)

Since we have a regular age grid of 1, the corresponding magnitudes in the
'end-of-period' specification are higher by 0.5.
The output also tells us that total life expectancy does not depend on the initial state and that
some occupation times are zero.
This comes from the particular data restrictions and assumptions of the application in the paper:
Mortality does not differ across states,
and transitions from state 4 (retirement) to states 1 (working) or 2 (unemployed) never occur (by assumption).
See the paper for more details.

Since we are talking about the meaning of the state encodings:
Let's label the numeric values with meaningful descriptions.
If we define a value label called 'MCWR', mcwr will use it to make the output a little nicer:

. label define MCWR 1 work 2 unem 4 retr 5 dead , modify

. mcwr expectancies , timing(eop) initprop(0.95 0.04 0.01)

This is also useful when browsing matrices (this will open the data browser):

. mcwr matbrowse F

(resume Stata execution)

Let's now assume that retirement occurs on average 3 months into the retirement
year. The easiest way to specify this is to generate all rewards variables first
as mid period and then edit them as needed:

. mcwr genvars , timing(mid) order

. describe

. replace r1_14 = 0.25 if !mi(r1_14)

. replace r4_14 = 0.75 if !mi(r4_14)

. replace r2_24 = 0.25 if !mi(r2_24)

. replace r4_24 = 0.75 if !mi(r4_24)

We do not have to worry about transitions out of retirement since their probability
is zero.

At this point it is useful to browse the data.
In particular, take note of how the first and last few rows are structured.

We recalculate:

. mcwr expectancies, initprop(0.95 0.04 0.01)

As an alternative to the above replace statements,
we could have solely used subcommand genvars in conjunction
with the timing() option of mcwr expectancies.
This has the advantage of not having to worry about getting the
baseline and exit age rows right (the "if !mi()" part of the statements).

. drop r*

. mcwr genvars , timing(0.25) order

. keep age p?? r?_14 r?_24

. mcwr expectancies , timing(mid, add) initprop(0.95 0.04 0.01)

After the creation of the 'timing(0.25)' rewards variables, we could
have generated the remaining rewards variables explicitly:

. mcwr genvars , timing(mid, add) order

. mcwr expectancies, initprop(0.95 0.04 0.01)

Next, we illustrate the equivalence to standard life table calculations.
We first look at a regularly spaced 1-year life table.

(click to load example data)

. describe

Checking whether we have a proper mcwr data set reveals problems:

. mcwr check // <- generates error

We look at the beginning and ending rows of the data set:

. list if !inrange(_n, 6, 106) , sep(5) noobs

We see that a number of mcwr data requirements, in particular those concerning the first and last
row of the data set, are not met.

(click to load the same data as a mcwr data set)

. list if !inrange(_n, 6, 107) , sep(5) noobs

The correct life table value for e0 is 80.16. Mid-period mcwr calculations yield

. mcwr expectancies , timing(mid)

This ignores the values in the ax variable, which are different from 0.5
for the first and last age.
We take them into account be generating an r-variable with corresponding values.

. gen r1_12 = ax

. mcwr expectancies , timing(mid, add)

The result is a little more accurate than the previous calculation.

Let's look at the corresponding 5-year life table.

(click to load example data)

. mcwr check // <- generates error

. list, sep(0) noobs

The data set must again be first transformed to a valid mcwr data set.

(click to load the modified data)

. mcwr check

. list, sep(0) noobs

We again perform calculations with and without taking ax into account:

. mcwr expectancies , timing(mid)

. ren ax r1_12

. mcwr expectancies , timing(mid, add) keep

This time we also specified the keep option.
This option kept the rewards variable r1_11, which was generated by mcwr expectancies
behind the scenes, from being dropped before the command concluded:

. list, sep(0) noobs

We did this to illustrate how mcwr uses rewards to handle irregularly spaced age intervals.

Saved results

Below, all returned lists are sorted.

mcwr check saves the following in r():

Macros
r(p_exi) p-variables in the data set

r(p_ful) full set of p-variables implied by states

r(p_new)) p-variables that are implied by states but not in the data set

r(s_trn)) list of transitions occuring in data set

r(s_frm)) list of from-states

r(s_trg)) list of target states

r(s_abs)) absorbing state

r(s_omt)) states omitted from the model

r(r_exi)) r-variables in the data set

r(r_ful)) full set of p-variables implied by states

r(r_new)) r-variables that are implied by states but not in the data set

r(r_trn)) list of transitions covered by existing r-variables

r(s_rcv)) states receiving rewards

r(s_nrc)) states not receiving rewards

r(s#desc)) description / value label of state #

r(numages)) number of age classes in the model

r(agelist)) list of age classes of the model

r(ageintervals)) list of lengths of age intervals

r(hasexit)) 0/1: whether data has an exit row

mcwr switch saves the following in r():

Macros
r(oldorder) the previous order: one of 'ji' or 'ij'

r(neworder) the new order: one of 'ji' or 'ij'

r(oldvarnames) list of p- and r-variables before the switch

r(newvarnames) list of p- and r-variables after the switch

mcwr genvars saves the following in r():

Macros
r(vars_existed) p- and r-variables that already existed

r(vars_filled) p- and r-variables that were newly created

mcwr expectancies saves the following in r():

Scalars
r(total) total weighted expectancy

Matrices
r(e) state expectancies

r(P) transition probabilities

r(F) fundamental matrix

r(R#) rewards matrix for state #

r(initprop) matrix of initial proportions, if supplied to the command

Author

Daniel C. Schneider, Mikko Myrskylä, Alyson van Raalte

Max Planck Institute for Demographic Research (www.mpidr.de)

Support: schneider@demogr.mpg.de

References

Schneider, Daniel C., Myrskylä, M., and Alyson van Raalte (2021):
Flexible Transition Timing in Discrete-Time Multistate Life Tables
Using Markov Chains with Rewards. MPIDR Working Paper, February 2021.

Crowther, Michael J., and Paul C. Lambert (2017): Parametric Multistate Survival Models: Flexible Modelling Allowing Transition-Specific
Distributions with Application to Estimating Clinically Useful Measures of Effect Differences.
Statistics in Medicine 36 (29): 4719–42. https://doi.org/10.1002/sim.7448.

Metzger, Shawna K., and Benjamin T. Jones (2018): Mstatecox: A Package for Simulating Transition Probabilities from
Semiparametric Multistate Survival Models. The Stata Journal 18 (3): 533–63. https://doi.org/10.1177/1536867X1801800304

Muniz, Jerônimo Oliveira (2020): Multistate Life Tables Using Stata. The Stata Journal 20 (3): 721–45.
https://doi.org/10.1177/1536867X20953577.

Also see

User-written, if installed:

mslt -help- -install- -remote help-

mstatecox -help- -install- -remote help-

multistate -help- -install- -remote help-