Help for xtoos

Description

The package XTOOS includes four commands that allow to evaluate the out-of-sample prediction performance of panel-data models in their time-series and cross-individual dimensions separately,
also with separate procedures for different types of dependent variables, either continuous or dichotomous variables.

The name of the commands are { help xtoos_t}, xtoos_i, xtoos_bin_t, and xtoos_bin_i.

The time-series procedures (xtoos_t and xtoos_bin_t) exclude a number of time periods defined by the user from the estimation sample for each individual in the panel.

Similarly, the cross-individual procedures (xtoos_i and xtoos_bin_i) exclude a group of individuals (for example, countries) defined by the user from the estimation sample (including all their observations throughout time).

Then, for the remaining subsamples (training-sample), they fit the specified models and use the resulting parameters to forecast the dependent variable (or the probability of a positive outcome)
in the unused time-periods or the unused individuals (testing-sample).

The unused time-period or individual sets are then recursively reduced by one period in every subsequent step in the time-series dimension,
or, either in a random or in an ordered fashion for the cross-individual dimension.

The estimation and forecasting evaluation are repeated until there are no more periods ahead or more individuals that could be evaluated.

In the continuous cases, the model's forecasting performance is reported both in absolute terms (RMSE) and relative to a naive prediction, by means of a U-Theil ratio.

In the dichotomous case, the prediction performance is evaluated based on the area under the receiver operator characteristic (ROC) statistic evaluated in both the training sample and the out of sample.

The procedures allow to choose different estimation methods, including some dynamic methodologies, and could also be used in a time-series or a cross-section dataset only.
They also allow evaluating the model's forecasting performance for one particular individual or for a defined group of individuals instead of the whole panel.

See xtoos_t for specific help about xtoos_t command.

See xtoos_i for specific help about xtoos_i command.

See xtoos_bin_t for specific help about xtoos_bin_t command.

See xtoos_bin_i for specific help about xtoos_bin_i command.

Proceedings Spain Stata Conference 2019

https://www.stata.com/meeting/spain19/slides/Spain19_Ugarte-Ruiz.pdf

Author

Alfonso Ugarte-Ruiz
alfonso.ugarte@bbva.com