help c_ml_stata

Title

c_ml_stata- Implementing machine learning classification in Stata

Syntax

c_ml_stata
outcome
[varlist],
mlmodel(modeltype)
out_sample_x(filename)
out_sample_y(filename)
in_prediction(name)
out_prediction(name)
cross_validation(name)
seed(integer)
n_folds(integer)
[save_graph_cv(name)]

where:

outcome is a numerical discrete dependent variable representing
the different classes. It is recommended to recode this
variable so to take values [1,2,...,M] in a M-class setting.
For example, if outcome is binary taking values [0,1], please record it so to take values [1,2].
Missing values are not allowed.

varlist is a list of numerical variables representing the features. When a feature is categorical,
please generate the categorical dummies related to this feature. As the command does not do it by default,
it is user's responsibility to generate the appropriate dummies.
Missing values are not allowed.

Description

c_ml_stata is a command for implementing machine learning classification algorithms in Stata 16.
It uses the Stata/Python integration (sfi)
capability of Stata 16 and allows to implement the following classification
algorithms: tree, boosting, random forest, regularized multinomial, neural network, naive Bayes, nearest neighbor,
support vector machine, standard (unregularized) multinomial.
It provides hyper-parameters' optimal tuning via K-fold cross-validation using greed search.
For each observation (or instance), this command generates both predicted class probabilities
and predicted labels using the Bayes classification rule.
This command makes use of the Python Scikit-learn
API to carry out both cross-validation and prediction.

Options

mlmodel(modeltype)
specifies the machine learning algorithm to be estimated.

out_sample_x(filename)
requests to provide a dataset as filename containing the testing observations over which estimating predictions.
This dataset contains only predictors.

out_sample_y(filename)
requests to provide a dataset as filename containing the outcome of the testing observations over which estimating predictions. This dataset must contain only the outcome. When the outcome is unknown, this can be inserted as a variable with all missing values.

in_prediction(name)
requires to specify a name for the file that will contain in-sample predictions.

out_prediction(name)
requires to specify a name for the file that will contain out-sample predictions.
These predictions are those obtained from the option out_sample(filename).

cross_validation(name)
requires to specify a name for the dataset that will contain cross-validation results.

seed(integer) requests to specify a integer seed to assure replication
of same results.

n_folds(integer) requests to specify the number of folds to catty out K-fold cross-validation.

save_graph_cv(name) allows to obtain the cross-validation optimal tuning graph drawing both train and test accuracy.

modeltype_options

Model
tree Classification tree

randomforest Bagging and random forests

boost Boosting

regularizedmultinomial Regularized multinomial

nearestneighbor Nearest Neighbor

neuralnet Neural network

naivebayes Naive Bayes

svm Support vector machine

multinomial Standard multinomial

Returns

c_ml_stata returns into e-return scalars (if numeric) or macros (if string)
the "optimal hyper-parameters", the "optimal train accuracy", the "optimal test accuracy",
and the "standard error of the optimal test accuracy" obtained via cross-validation.
For each learner, the list and meaning of the tuned hyper-parameters are:

Boosting (option "boost"):

-> e(OPT_LEARNING_RATE): scalar. It is the coefficient that shrinks the contribution of each tree to the Boosting's prediction.

-> e(OPT_N_ESTIMATORS): scalar. It is the number of boosting iteration to perform.

Nearest-neighbor (option "nearestneighbor")

-> e(OPT_NN): scalar. It is the number of nearest neighbors to use.

-> e(OPT_WEIGHT): local macro. It returns the kernel weighting scheme, that can be:
(1) uniform: uniform weights, where all observations in each neighborhood are weighted equally;
or (2) distance, weighting observations by the inverse of their distance from the point of imputation.

Naive Bayes (option "naivebayes")

-> e(OPT_VAR_SMOOTHING): scalar. Portion of the largest variance of all predictors that is added to variances for calculation stability.

Neural network (option "neuralnet")

-> e(OPT_NEURONS_L_1): scalar. It is the number of neurons (or hidden units) in the first (hidden) layer.

-> e(OPT_NEURONS_L_2): scalar. It is the number of neurons (or hidden units) in the second (hidden) layer.

Random forests (option "randomforest")

-> e(OPT_MAX_DEPTH): scalar. It is the maximum depth of the tree.

-> e(OPT_MAX_FEATURES): scalar. It is the number of features (or predictors)
to consider when looking for the tree best split.

Regularized multinomial (option "regularizedmultinomial")

-> e(OPT_PENALIZATION): scalar. It is the inverse of regularization strength (positive coefficient). Like in the support vector machine, smaller values specify stronger regularization.

-> e(OPT_L1_RATIO): scalar. It is the elastic mixing parameter, varying between zero (Lasso regression) and one (Ridge regression). Intermediate values of this parameter weigh differently the Lasso and Ridge penalization terms.

Support vector machine (option "svm")

-> e(OPT_C): scalar. It is the SVM regularization parameter,
where the strength of the regularization is inversely proportional to C. It is strictly positive.

-> e(OPT_GAMMA): scalar. It is the kernel coefficient for the polynomial or radial kernel.

Regression tree (option "tree")

-> e(OPT_DEPTH): scalar. It is the maximum depth of the tree.

c_ml_stata provides model predictions both as predicted labels and as predicted probabilities.

Remarks

-> When running c_ml_stata, one has to prepare in advance three datasets:
(i) the main dataset to open and use in the current Stata session: it is the training dataset, employed also for carrying out K-fold cross-validation;
(ii) a dataset (located in the current working directory) to insert as filename in option out_sample_x(filename):
it is the testing dataset, only containing the predictors of testing observations;
(iii) a dataset (located in the current working directory) to insert as filename
in option out_sample_y(filename): it is the testing dataset,
only containing the outcome of testing observations.
The datasets defined in (i) and (ii) must not contain missing values.
Before running this command, please check whether these datasets contain missing values and delete observations containing them (listwise).
Of course, the datasets in (ii) and (iii) must have the same size and refer to the same out-of-sample observations.

-> The order in which the predictors appear in the testing dataset to insert in out_sample_x(filename) must be the same as the one of the training dataset.

-> If the variables appearing in the used Stata datasets contain value labels, it is recommended to eliminate them. Indeed, Python cannot read data with labeled values.

-> To run this program you need to have both Stata 16 (or later versions) and Python
(from version 2.7 onwards) installed. You can find information about how to install
Python in your machine here: https://www.python.org/downloads.
Also, the Python Scikit-learn (and related dependencies) and the Stata Function Interface (sfi) APIs
must be uploaded before running the command. You can find information about how to install/use
them here (observe that the SFI API is already installed in your Stata 16 or later versions):
(1) Scikit-learn: https://scikit-learn.org/stable/install.html; (2) sfi: https://www.stata.com/python/api17.

-> Please, remember to have the most recent up-to-date version of this program installed.

Example

*** RUNNING A CLASSIFICATION TREE ***

* Download these datasets, by putting them in the same directory:

* Dataset 1: "c_data" -> the complete dataset

* Dataset 2: "c_data_x" -> the test dataset containing "only" the features X

* Dataset 3: "c_data_y" -> the test dataset containing "only" the outcome y

* Load the complete dataset

. use "c_data" , clear

* Run the "c_ml_stata" command with model option "tree"

#delimit ;

. c_ml_stata y x1 x2 x3 x4 , mlmodel("tree") in_prediction("in_pred") cross_validation("CV") out_sample_x("c_data_x") out_sample_y("c_data_y") out_prediction("out_pred") seed(10) n_folds(10) save_graph_cv("graph_cv");

#delimit cr

* Look at the main results

. ereturn list

* Look at the cross-validation results

. use CV , clear

. list

* Look at the out-of-sample predictions

. use out_pred , clear

. list

Reference

Cerulli, G. 2021. Improving econometric prediction by machine learning, Applied Economics Letters, 28, 16, 1419-1425, 2021.

Gareth, J., Witten, D., Hastie, D.T., Tibshirani, R. 2013. An Introduction to Statistical Learning : with Applications in R. New York, Springer.

Raschka, S., Mirjalili, V. 2019. Python Machine Learning. 3rd Edition, Packt Publishing.

Author

Giovanni Cerulli

IRCrES-CNR

Research Institute for Sustainable Economic Growth, National Research Council of Italy

E-mail: giovanni.cerulli@ircres.cnr.it

Also see

Online: python, r_ml_stata, srtree, srtree, subset