*! version 1.1 oct 30 2022

Title

cv_kfold -- Module to implement k-fold cross-validation procedures

Syntax

cv_kfold
[,
options]

Optional
k(#) Indicates the number of equal sizes subsamples will be used for the estimation of the k-fold cross validation.
Default value is 5.

reps(#) Indicates times the cross validation procedure will be implemented. Default value is 1.

seed(str) The author can provide a seed number for the generation of the random groups.

Description

This cv_kfold is a post estimation command that implements k-fold crossvalidation for various stata commands.

The current version of this command can be used after: regress, logit, probit, logit, cloglog,
poisson, nbreg, mlogit, mprobit, ologit, and oprobit.

When used after regress, cv_kfold estimates and reports the average unweighted Root Mean Squared error (RMSE) across all repetitions.
For all other estimation commands, it reports the unweighted model loglikelihood function (AvLL).

Internally, cv_kfold uses the syntax from the previously estimated command for the k-fold cross validation producedure.
Using the overall estimation sample, k random groups of equalsize are created, and the same previously model syntax is used to re-estimate the model.

For example, if one uses a 5-folds, 4 of the 5 subsamples are used to estimate the model, leaving the 5th subsample to make an out-of sample prediction
and evaluate the model using the RMSE or the AvLL. when the option reps() is used, the command repeats the k-fold procedure N times, and reports the average RMSE and AVLL
across all repetitions, but stores the estatistics of each individual repetition in a separate matrix.

If you are interested in a leave-one-out cross validation procedure
for regress, see cv_regress available from ssc.

The command has been tested under Stata 14. But it does not work with version control.

Examples

Set up

ssc install frause

set seed 10101

frause oaxaca, clear

Leave on out cross validation

ssc install cv_regress

regress lnwage educ exper tenure female age agesq

cv_regress

k-fold cross validation

regress lnwage educ exper tenure female age agesq

cv_kfold

k-fold cross validation, with 5 repetitions

regress lnwage educ exper tenure female age agesq

cv_kfold, reps(5)

matrix list r(msqr)

k-fold for other type of models. Logit, poisson and mlogit

drop if lnwage==.

gen dwage=lnwage>3.4

gen wage=round(exp(lnwage))

xtile qwage=lnwage, n(5)

logit dwage educ exper tenure female age agesq

cv_kfold, reps(5)

matrix list r(msqr)

Currently, Poisson model only works if the Dep variable is

poisson wage educ exper tenure female age agesq

cv_kfold, reps(5)

matrix list r(msqr)

mlogit qwage educ exper tenure female age agesq

cv_kfold, reps(5)

matrix list r(msqr)

ologit qwage educ exper tenure female age agesq

cv_kfold, reps(5)

matrix list r(msqr)

Author

Fernando Rios-Avila

Levy Economics Institute of Bard College

Annandale-on-Hudson, NY

friosavi@levy.org

Acknowledgement

Many thanks to Morteza Saharkhiz for suggesting extending he command to ologit and oprobit models.

Also see

Help: cv_regress