Help for xtoos_t

Description

xtoos_t evaluates the out-of-sample prediction performance of a specified panel-data model in a time-series dimension.

The procedure excludes a number of time periods defined by the user from the estimation sample for each individual in the panel.
Then for the remaining subsample it fits the specified model and uses the resulting parameters to forecast the dependent variable in the unused periods (out-of-sample).
The unused time-period set is then recursively reduced by one period in every subsequent step and the estimation and forecasting procedures are repeated,
until there are no more periods ahead that could be evaluated.

xtoos_t allows to choose different estimation methods including some dynamic methodologies and could also be used in a time-series dataset only.

In the case the specification includes lags of the dependent variable, the procedure is able to automatically generate dynamic forecasts for the evaluation performance.

xtoos_t allows to evaluate the model's forecasting performance for one particular individual or for a defined group of individuals instead of the whole panel.

xtoos_t reports the specified model's forecasting performance, both in absolute terms (RMSE) and also relative to an alternative model by means of an U-Theil ratio, which is the ratio
between the RMSE of the specified model and the RMSE of the alternative model. By default, the alternative model is a "naive" prediction in which
the last observation of the in-sample period is used directly as a forecast without any change. The procedure also allows to use an AR1 model as the alternative model for the comparison.
The procedure reports the RMSE of the specified model (RMSE_oos), the alternative model (RMSE_Alt_oos) and the U-Theil ratio (Utheil).

The performance results are broken down and reported in two different ways:

1) According to the last period included in the estimation sample.

2) According to the length of the forecasting horizon.

Additionally, it allows to draw a graph with the model forecasts at each forecasting horizon for a selected group of individuals in the panel.

Proceedings Spain Stata Conference 2019

https://www.stata.com/meeting/spain19/slides/Spain19_Ugarte-Ruiz.pdf

Syntax

xtoos_t depvar [indepvars] [if], *indate(string) *cdate(string)
[met(string)] [mcomp(string)] [evalopt(varname)]
[fe] [xbu] [dum] [opar] [lags(numlist)] [hgraph(numlist)]
[model_options]

Options

*indate() It is always required. Specifies the last date (time-period) included in the estimation sample in the first step of the recursive procedure,
i.e. it defines the date at which the performace evaluation starts. It allows Yearly, Quarterly and Monthly dates, or integer numbers, only. Quarterly and Monthly dates should be written as 1990q1 or 1990m1.

*cdate() It is always required. Specifies the date at which the evaluation performance ends, i.e. the current date or the date at which the sample ends.
It allows Yearly, Quarterly and Monthly dates, or integer numbers, only. Quarterly and Monthly dates should be written as 1990q1 or 1990m1.

evalopt() Specifies a particular individual or group of individuals for which the prediction performance is evaluated.
The particular individual or group must be defined by a dummy variable equal to one for those individuals.

met() Specifies the estimation method. The default is xtreg when using panel data and reg when using time-series data.

mcomp() Specifies the estimation method for the AR1 model used as a comparison and for estimating the U-Theil, when an AR1 is preferred to a "naive" prediction.
The AR1 could be estimated by dynamic panel methods such as xtabond or xtdpdsys, or simply by xtreg or reg.

fe Estimates using Fixed-Effects (within) estimator.

xbu Specifies that the prediction including the fixed or random component should be used.

dum Estimates including one dummy variable per individual in the specification and in the prediction.

opar Specifies that the contribution of the dummy variables per individual should not be included in the prediction.
It can only be used together with the option dum.

lags() Specifies the number of lags of the dependent variable that must be included the specification, the default is zero lags.
This option should be used when using a dynamic panel methods such as xtabond or xtdpdsys. If other estimation method is being used,
the command xtoos_t also allows to write down the desired lagged dependent variables terms simply as other explanatory variables.

hgraph() Specifies the individuals for which to draw a "hair" graph, with all the model forecasts at each forecasting horizon.
The graph displays the actual dependent variable together with the model's prediction for each forecasting horizon and the alternative (comparison) prediction, which by default
corresponds to the prediction of an AR(1) model. The option requires to specify a list of numbers (panel code number) corresponding to the individals that you want to draw the hair graph.

model_options Specifies any other estimation options specific to the method used and not defined elsewhere.

Example 1. Defining the evaluation (out-of-sample) period

Use of xtoos_t to evaluate the prediction performance between periods 15 and 20 (out of 20 total periods in the sample)

. webuse invest2, clear

. xtset company time

. xtoos_t invest market stock, indate(15) cdate(20)

Example 2. Restricting the evaluation to a group of individuals

Use of xtoos_t to evaluate the prediction performance between periods 15 and 20, but restricting the evaluation only to company # 1

. gen company1=company==1

. xtoos_t invest market stock, indate(15) cdate(20) evalopt(company1)

Example 3. Changing the estimation method

Use of xtoos_t using as estimation method the command xtregar

. xtoos_t invest market stock, indate(15) cdate(20) met(xtregar)

Example 4. Changing the comparison method

Use of xtoos_t using OLS (reg) as the estimation method for an AR(1) comparison model

. xtoos_t invest market stock, indate(15) cdate(20) mcomp(reg)

Example 5. Fixed Effects and individual components options

Use of xtoos_t using Fixed-Effects (within) estimator including the estimated individual components in the prediction

. xtoos_t invest market stock, indate(15) cdate(20) fe xbu

Example 6. No individual component in the prediction

Use of xtoos_t using Fixed-Effects (within) estimator but without including the estimated individual components in the prediction

. xtoos_t invest market stock, indate(15) cdate(20) fe

Example 7. Use of dummy variables

Use of xtoos_t using dummy variables per individual and including their estimated values in the prediction (equivalent to options fe xbu)

. xtoos_t invest market stock, indate(15) cdate(20) dum

Example 8. Use of dummy varibles in the specification but not in the prediction

Use of xtoos_t using dummy variables per individual but without including their estimated values in the prediction (equivalent to option fe)

. xtoos_t invest market stock, indate(15) cdate(20) dum opar

Example 9. Defining lags of the dependent variable

Use of xtoos_t including lags of the dependent variable in the specification. The following three specifications are equivalent:

xtoos_t invest market stock, indate(15) cdate(20) lags(3)

xtoos_t l(0/3).invest market stock, indate(15) cdate(20)

xtoos_t invest l.invest l2.invest l3.invest market stock, indate(15) cdate(20)

Example 10. Using a dynamic model method

Use of xtoos_t using a dynamic model method, either xtabond or xtdpdsys. In this case, the default specification includes one lag
of the dependent variable

. xtoos_t invest market stock, indate(15) cdate(20) met(xtabond)

. xtoos_t invest market stock, indate(15) cdate(20) met(xtdpdsys) lags(2)

Example 11. Drawing a graph with the different out-of-sample forecasts

Use of xtoos_t to draw a "hair" graph with all the model forecasts at each forecasting horizons for individuals 1 to 5.

xtoos_t invest market stock, indate(15) cdate(20) hgraph(1/5)

Stored results

xtoos_t stores the following in r():

Matrices

r(ev_last) evaluation according to the last in-sample period

r(ev_hor) evaluation according to the forecasting horizon

Author

Alfonso Ugarte-Ruiz
alfonso.ugarte@bbva.com