*! version 0.0.3 15feb2024

Syntax

cmdmod "estimation command" , split(varname) [
kfold(integer) ]

Required
split name of the variable that identifies the training split(s)

Optional
kfold specifies the number of folds in the training set; default is kfold(1).

Description

cmdmod is part of the crossvalidate suite of tools to implement
cross-validation methods with Stata estimation commands. cmdmod is used
internally by the fitit and predictit commands for metaprogramming
tasks. Specifically, cmdmod is used to modify the user supplied
estimation command to include the necessary if expression to fit the data to
the appropriate training set or K-Fold. It is also used to ensure that
predictions are made on the appropriate out-of-sample portion of the data set.

Options

Required

split must contain the name of the variable that stores the test,
validation, and test splits. There will only be a single variable if the splits
were created using splitit.

Optional

kfold defines the number of K-Folds used for the training set. In other
places, we reference using K-Fold cross-validation in the more common form,
where the training set consists of multiple subsets of data. However, standard
train/test and train/validation/test splits are simply a special case of K-Fold
cross-validation where there is only a single fold.

Examples

Binary Classification Example

Non-K-Fold Example

cmdmod "ivreg price (mpg i.foreign)", spl(spvar)

K-Fold Example

cmdmod "ivreg price (mpg i.foreign)", spl(spvar) kf(5)

Returned Values

The following lists the names of the r-macros and their contents.

r(modcmd) the modified command for test/train or test/validate/train splits

r(predifin) the if expression to use when predicting on validation/test split.

r(kfmodcmd) the modified command for K-Fold splits

r(kfpredifin) the if expression to use when predicting on the K-Fold hold out set.

cmdmod also stores the modified command strings and modified if
expressions in data set characteristics. This appears to be the most stable way
to ensure that the values in the string are not evaluated prematurely when
assigning the -r()- macros to another local. The data set characteristics that
are generated by cmdmod are automatically cleaned up by xt and
xtloo.

Additional Information

If you have questions, comments, or find bugs, please submit an issue in the crossvalidate GitHub repository.

Contact

William R. Buchanan, Ph.D.

Sr. Research Scientist, SAG Corporation

SAG Corporation

wbuchanan at sagcorp [dot] com

Steven D. Brownell, Ph.D.

Economist, SAG Corporation

SAG Corporation

sbrownell at sagcorp [dot] com