*! version 1 04jul2026
help mcset

Title

mcset -- The Model Confidence Set (MCS)

Syntax

mcset varlist [if] [in],
typeboot(string)
typeloss(string)
lblock(integer)
[ bootdraws(integer)
alpha(real)
stat(string)
seed(integer)
]

You must tsset your data before using mcset; see [TS] tsset. There must not be gaps in the time series. Time-series operators are not supported.

varlist is a list of variables, the first of which is the actual series, and the
following variables are forecasts from alternative models. At least two forecast series must be provided.

Description

The mcset command implements
the Model Confidence Set (MCS) procedure of Hansen, Lunde, and Nason (2011), which
provides a formal way to ask which forecasting models are statistically distinguishable
from the best-performing alternatives. The main output of the procedure is not a single
“winner”, but a set of models that cannot be rejected as having superior predictive
ability at a chosen confidence level. The algorithm is based on the implementation in R provided by
Catania (2026), first described by Bernardi and Catania (2014) and the block bootstrap schemes of Baum and Otero (forthcoming).

mcset implements three block bootstrap schemes for dependent time series data:

mbb: Moving block bootstrap of observations; see Künsch (1989) and Liu and Singh (1992)

cbb: Circular block bootstrap of observations; see Politis and Romano (1992)

sbb: Stationary block bootstrap of observations; see Politis and Romano (1994)

Options

typeboot must be used to choose among the three block bootstrap schemes implemented by mcs.

typeloss must be used to choose among the four loss functions: rmse for root mean square
error, mae for mean absolute error, mape for mean absolute percentage error, and
qlike for the quasi-likelihood loss, log h + sigma^2/h.

lblock sets the length of the block of observations to bootstrap. Note that in the case of sbb, lblock corresponds to the average length of the blocks.
Setting lblock equal to 1 yields the simple bootstrap, suitable for time independent processes, while lblock greater than 1 is suitable for time dependent processes.
If lblock is set to zero, the auto-blocklength mechanism following Politis and White (2004), Patton et al. (2009) and Politis
and Romano (1995) is employed. In that case, an optimal blocksize for each forecast series is computed, but the maximum of hose
values is used in the bootstrap for all forecast series. The optimal blocksize values are returned in the blklen matrix.

bootdraws specifies the number of block bootstrap repetitions. If not specified, it is set to 100.

alpha specifies the significance level to be used in defining the MCS. If not specified, it defaults to 0.10.

stat specifies the test statistic, which can be TR or Tmax. If not specified, the routine computes TR.

seed sets the seed for random number generation in the bootstrap results to produce reproducible results.

Stored results

Scalars

r(N_included) Number of models included in MCS

r(N_excluded) Number of models excluded from MCS

r(N_models) Number of models evaluated

r(lblock) Blocksize

r(bootdraws) Number of bootstrap repetitions

r(N) Number of observations

Macros

r(excluded) Model names for those excluded from MCS

r(included) Model names for those included in MCS

r(typeboot) Bootstrap scheme

r(typeloss) Loss function

r(stat) Test statistic

r(predicted) Model names of the predicted series, in reverse order of average loss

r(actual) Name of the actual series

r(cmdname) Command name

Matrices

r(mcsres) Result matrix

r(blklen) Block length

Example of use

We illustrate the use of the mcs command using generated data. ssc install bcuse if needed.

. bcuse mcsdata2

. summarize

In these data, y is the actual series, and the m* series are forecasts of y from
different models. By construction, models A, C, E and G are more divergent from y.

. mcset y m*, typeboot(cbb) typeloss(rmse) lblock(12) bootdraws(1000)

The MCS--containing models with similar accuracy--includes the six other models, and excludes models
A, C, E and G.

References

Baum, C. F., and J. Otero (2026). Bootstrapping time-dependent stationary processes.
The Stata Journal 26 (forthcoming).

Bernardi, M., and L. Catania (2014). The Model Confidence Set package for R.
https://arxiv.org/abs/1410.8504.

Catania, L. (2026). Package ‘MCS’. Technical report, CRAN.
https://search.r-project.org/CRAN/refmans/MCS/html/MCS-package.html

Hansen, P. R., A. Lunde, and J. M. Nason (2005). Model confidence sets for forecasting
models. Working Paper 2005-07, Federal Reserve Bank of Atlanta, Atlanta, GA.

Hansen, P. R., A. Lunde, and J. M. Nason (2011). The model confidence set. Econometrica 79(2): 453–497.

Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics 17, 1217-1261.

Liu, R. Y. and Singh, K. (1992). Moving blocks jackknife and bootstrap capture weak dependence, in R. Lepage and L. Billard, eds, 'Exploring the Limits of the Bootstrap', Wiley, New York, pp. 225-248.

Patton, A., Politis, D. N. and White, H. (2009). Correction to "automatic block-length selection for the dependent bootstrap"
by D. Politis and H. White. Econometric Reviews 28(4), 372–375

Politis, D. N. and Romano, J. P. (1992). A circular block resampling procedure for stationary data, in R. Lepage and L. Billard, eds, 'Exploring the Limits of Bootstrap', Wiley, New York, pp. 263-270.

Politis, D. N. and Romano, J. P. (1994). The stationary bootstrap. Journal of the American Statistical Association 89(428), 1303-1313.

Politis, D. N. and Romano J. P. (1995). Bias-corrected nonparametric spectral estimation. Journal of Time Series Analysis
16(1), 67–103.

Politis, D. N. and White, H. (2004). Automatic block-length selection for the dependent bootstrap. Econometric Reviews 23(1),
53–70.

Authors

Christopher F Baum

Boston College

Chestnut Hill, MA USA

baum@bc.edu

Jesús Otero

Universidad del Rosario

Bogotá, Colombia

jesus.otero@urosario.edu.co