help r_ml_stata_cv

Title

r_ml_stata_cv -- Machine learning regression in Stata

Syntax

r_ml_stata_cv
outcome
features
[if] [in],
mlmodel(modeltype)
data_test(filename)
seed(integer)
[learner_options
cv_options
other_options]

Inputs

outcome: numerical variable

features: list of numerical variables representing the features. When a feature is categorical,
please generate the categorical dummies related to this feature. As the command does not do it by default,
it is user's responsibility to generate the appropriate dummies

Description

r_ml_stata_cv is a command for implementing machine learning regression algorithms in Stata 16.
It uses the Stata/Python integration (sfi)
capability of Stata 16 and allows to implement the following regression
algorithms: ordinary least squares, elastic net, tree, boosting, random forest, neural network, nearest neighbor,
support vector machine. It provides hyper-parameters' optimal tuning via K-fold cross-validation using greed search.
This command makes use of the Python Scikit-learn API to carry out both cross-validation and prediction.

Main options

options

Main
mlmodel(modeltype) where modeltype indicates the ML model to fit

data_test(filename) where filename is a Stata dataset used as testing dataset

seed(#) where # is an integer number used as random seed

Model options

modeltype_options

Model
ols Ordinary least squares

elasticnet Elastic net

tree Tree regression

randomforest Bagging and random forests

boost Boosting

nearestneighbor Nearest neighbor

neuralnet Neural network

svm Support vector machine

Learner options

learner_options

Tree
tree_depth(#) Maximum tree depth. # = integer

Elastic net
alpha(#) Penalization parameter. # = float

l1_ratio(#) Elastic parameter. # = float

Random forest
tree_depth(#) Maximum tree depth. # = integer

max_features(#) Maximum number of splitting features. # = integer

n_estimators(#) Number of bootstrapped trees. # = integer

Boosting
tree_depth(#) Maximum tree depth. # = integer

learning_rate(#) Learning rate. # = float

n_estimators(#) Number of sequential trees. # = integer

Nearest neighbor
nn(#) Number of nearest neighbors. # = integer

Neural network
n_neurons_l1(#) Number of neurons in the first layer. # = integer

n_neurons_l2(#) Number of neurons in the second layer. # = integer

alpha(#) L2 penalization parameter. # = float

Support vector machine
c(#) Margin parameter. # = float

gamma(#) Inverse of the radius of influence of observations selected as support vectors. # = float

Cross-validation options

cv_options

CV options
n_folds(#) Number of cross-validation folds. # = integer

cross_validation(name) name specifies the name of the dataset containing cross-validation results

Other options

other_options

Other options
prediction(name) Generating predictions of the outcome variable of name name

default Running the specified model with parameters equal to the Scikit-learn's default values

graph_cv Displaying the cross-validation graph

save_graph_cv(name) Saving the cross-validation graph with name name

Returns: general

Scalars
e(N_train_all) Total number of observations in the initial training dataset

e(N_train_used) Number of training observations used

e(N_test_all) Total number of observations in the initial testing dataset

e(N_test_used) Number of testing observations used

e(N_features) Number of features

e(TRAIN_ACCURACY) K-fold cross-validation training average accuracy (= explained variance)

e(TEST_ACCURACY) K-fold cross-validation testing average accuracy (= explained variance)

e(SE_TEST_ACCURACY) K-fold cross-validation stad. err. of the testing average accuracy (= explained variance)

e(BEST_INDEX) Best cross-validation index

e(N_FOLDS) Number of folds used for cross-validation

e(Train_mse) Mean squared error on training data

e(Test_mse) Mean squared error on testing data

e(Train_mape) Mean absolute prediction error on training data

e(Test_mape) Mean absolute prediction error on testing data

Returns: by learner

Boosting
Scalars
e(OPT_LEARNING_RATE) Optimal coefficient that shrinks the contribution of each tree to the Boosting's prediction

e(OPT_N_ESTIMATORS) Optimal number of sequential trees for the Boosting ensemble prediction

e(OPT_TREE_DEPTH) Optimal depth of the single tree

Elastic net
Scalars
e(OPT_ALPHA) Optimal shrinkage parameter: constant multiplying the penalty term

e(OPT_L1_RATIO) Optimal elastic net parameter: 0=Lasso, 1=Ridge regression

Nearest neighbor
Scalars
e(OPT_NN) Optimal number of nearest neighbors to use

Macro
e(OPT_WEIGHT) Optimal kernel weighting scheme: (1) uniform: uniform weights (observations equally weighted);
or (2) distance (observations weighted by the inverse of their distance from the point of imputation)

Neural network
Scalars
e(OPT_NEURONS_L_1) Optimal number of neurons (or hidden units) in the first (hidden) layer

e(OPT_NEURONS_L_2) Optimal number of neurons (or hidden units) in the second (hidden) layer

e(OPT_ALPHA) Optimal L2 penalization parameter

Random forests
Scalars
e(OPT_N_ESTIMATORS) Optimal number of bootstrapped trees for the Boosting ensemble prediction

e(OPT_TREE_DEPTH) Optimal depth of the single tree

e(OPT_MAX_FEATURES) Optimal number of features to consider at each tree best split

Support vector machine
Scalars
e(OPT_C) Optimal regularization parameter, where the strength of the regularization is inversely proportional to C

e(OPT_GAMMA) Optimal kernel coefficient for the polynomial or radial kernel

Regression tree
Scalars
e(OPT_DEPTH) Optimal maximum depth of the tree

Remarks

To run this program you need to have both Stata 16 (or later versions) and Python
(from version 2.7 onwards) installed. You can find information about how to install
Python in your machine here: https://www.python.org/downloads.
Also, the Python Scikit-learn (and related dependencies) and the Stata Function Interface (sfi) APIs
must be uploaded before running the command. You can find information about how to install/use
them here (observe that the SFI API is already installed in your Stata 16 or later versions):
(1) Scikit-learn: https://scikit-learn.org/stable/install.html; (2) sfi: https://www.stata.com/python/api17.

Before displaying the Stata output, this program displays (in red) a panel of "Python warnings" concerning the current fit. These warnings can be useful sometimes, for example to check whether convergence of certain algorithms is achieved.

This program incorporates the pylearn, setup command carried out by Droste (2002) to check whether your computer has the prerequisite Python packages, trying to install them automatically if not.

Please, remember to have the most recent up-to-date version of this program installed.

Example

Example 1: Default regression tree with train/test predictions

Load intial dataset

sysuse auto, clear

Split dataset into train and test datasets

splitsample, generate(svar, replace) split(0.80 0.20)

Form the train dataset

preserve

keep if svar==1

save auto_train , replace

restore

Form the test dataset

preserve

keep if svar==2

save auto_test , replace

restore

Load train dataset

use auto_train, clear

Run tree regression

r_ml_stata_cv price mpg rep78 headroom , mlmodel("tree") data_test("auto_test") default prediction("pred") seed(10)

Example 2: Regression tree with train/test predictions, and cross-validation

Load intial dataset

sysuse auto, clear

Split dataset into train and test datasets

splitsample, generate(svar, replace) split(0.80 0.20)

Form the train dataset

preserve

keep if svar==1

save auto_train , replace

restore

Form the test dataset

preserve

keep if svar==2

save auto_test , replace

restore

Load train dataset

use auto_train, clear

Run tree regression

r_ml_stata_cv price mpg rep78 headroom , mlmodel("tree") data_test("auto_test") tree_depth(3) prediction("pred") cross_validation("CV") n_folds(5) seed(10)

Example 3: Regression tree with train/test predictions, cross-validation, and optimal tuning

Load intial dataset

sysuse auto, clear

Split dataset into train and test datasets

splitsample, generate(svar, replace) split(0.80 0.20)

Form the train dataset

preserve

keep if svar==1

save auto_train , replace

restore

Form the test dataset

preserve

keep if svar==2

save auto_test , replace

restore

Load train dataset

use auto_train, clear

Run tree regression

r_ml_stata_cv price mpg rep78 headroom , mlmodel("tree") data_test("auto_test") tree_depth(2 3 4 5 6 7 8) prediction("pred") cross_validation("CV") n_folds(5) seed(10)

Reference

Cerulli, G. 2021. Improving econometric prediction by machine learning, Applied Economics Letters, 28, 16, 1419-1425, 2021.

Droste, M. 2022. Stata-pylearn. Available at: https://github.com/mdroste/statapylearn.

Gareth, J., Witten, D., Hastie, D.T., Tibshirani, R. 2013. An Introduction to Statistical Learning : with Applications in R. New York, Springer.

Raschka, S., Mirjalili, V. 2019. Python Machine Learning. 3rd Edition, Packt Publishing.

Author

Giovanni Cerulli

IRCrES-CNR

Research Institute for Sustainable Economic Growth, National Research Council of Italy

E-mail: giovanni.cerulli@ircres.cnr.it

Also see

Online: python, c_ml_stata, srtree, srtree, subset