Help for xtoos_bin_t

Description

xtoos_bin_t evaluates the out-of-sample prediction performance of a specified panel-data model with a binary dependent variable in a time-series dimension.

The procedure excludes a number of time periods defined by the user from the estimation sample for each individual in the panel.
Then for the remaining subsample it fits the specified model and uses the resulting parameters to forecast the probability of ocurrence of the dependent variable in the unused periods (out-of-sample).

The unused time-period set is then recursively reduced by one period in every subsequent step and the estimation and forecasting procedures are repeated,
until there are no more periods ahead that could be evaluated.

The procedure evaluates the prediction performance based on the area under the receiver operator characteristic (ROC) statistic evaluated in both the training sample (labeled as _in) and the out-of-sample (labeld as _oos).
The procedure reports both the estimated ROC and also its standard error (SE).

xtoos_bin_t allows to choose different estimation methods different estimation methods (e.g. logit, probit, xtprobit) and could also be used in a time-series dataset only.

xtoos_bin_t allows to evaluate the model's forecasting performance for one particular individual or for a defined group of individuals instead of the whole panel.

The performance results are broken down and reported in two different ways:

1) According to the last period included in the estimation sample.

2) According to the length of the forecasting horizon.

Proceedings Spain Stata Conference 2019

https://www.stata.com/meeting/spain19/slides/Spain19_Ugarte-Ruiz.pdf

Syntax

xtoos_bin_t depvar [indepvars] [if], *indate(string) *cdate(string)
[mprob(string)] [evalopt(varname)] [met(string)] [fe] [dum]
[model_options]

Options

*indate() It is always required. Specifies the last date (time-period) included in the estimation sample in the first step of the recursive procedure,
i.e. it defines the date at which the performace evaluation starts. It allows Yearly, Quarterly and Monthly dates, or integer numbers, only. Quarterly and Monthly dates should be written as 1990q1 or 1990m1.

*cdate() It is always required. Specifies the date at which the evaluation performance ends, i.e. the current date or the date at which the sample ends.
It allows Yearly, Quarterly and Monthly dates, or integer numbers, only. Quarterly and Monthly dates should be written as 1990q1 or 1990m1.

mprob() Specifies the method of estimating the probability of a positive outcome that depends on the estimation method used (e.g. prob, pu0, pc1).

evalopt() Specifies a particular individual or group of individuals for which the prediction performance is evaluated.
The particular individual or group must be defined by a dummy variable equal to one for those individuals.

met() Specifies the estimation method. The default is xtlogit.

fe Estimates using Fixed-Effects (within) estimator.

dum Estimates including one dummy variable per individual in the specification and in the prediction.

model_options Specifies any other estimation options specific to the method used and not defined elsewhere.

Example 1. Defining the evaluation (out-of-sample) period

Use of xtoos_bin_t to evaluate the prediction perfomance based on ROC between the first quarter of year 2010 and the fourht quarter of year 2018

. xtoos_bin_t y x1 x2, indate(2010q1) cdate(2018q4) mprob(pr)

Example 2. Fixed Effects option and probability method

Use of xtoos_bin_t using Fixed-Effects (within) estimator and the estimation option pc1 for the predicted probability

. xtoos_bin_t y x1 x2, indate(2010q1) cdate(2018q4) mprob(pc1) fe

Example 3. Changing the estimation method

Use of xtoos_bin_t using as estimation method the command xtprobit

. xtoos_bin_t y x1 x2, indate(2010q1) cdate(2018q4) mprob(pr) met(xtprobit)

Example 4. Restricting the evaluation to a group of individuals

Use of xtoos_bin_t to evaluate the prediction perfomance based on ROC, but restricting the evaluation only to individual # 1

. gen ind1=individual==1

. xtoos_bin_t y x1 x2, indate(2010q1) cdate(2018q4) mprob(pr) evalopt(ind1)

Example 5. Including dummy variables

Use of xtoos_bin_t using dummy variables per individual

. xtoos_bin_t y x1 x2, indate(2010q1) cdate(2018q4) mprob(pr) dum

Stored results

xtoos_bin_t stores the following in r():

Matrices

r(ev_last) evaluation according to the last in-sample period

r(ev_hor) evaluation according to the forecasting horizon

Author

Alfonso Ugarte-Ruiz
alfonso.ugarte@bbva.com