help sctree

Title

sctree- Implementing classification trees via optimal pruning, bagging, random forests, and boosting methods

Syntax

sctree
outcome
[varlist]
[if] [in],
model(modeltype)
rversion(R_version)
[prune(integer)
cv_tree
prediction(new_data_filename)
in_samp_data(filename)
out_samp_data(filename)
ntree(integer)
mtry(integer)
inter_depth(integer)
shrinkage(number)
pdp(string)
seed(integer)]

Description

sctree is a Stata wrapper for the R functions "tree()", "randomForest()", and "gbm()".
It allows to implement the following classification tree models: (1) classification tree with optimal pruning,
(2) bagging, (3) random forests, and (4) boosting.

Options

model(modeltype) specifies the model
to be estimated.

rversion(R_version) specifies the R version intalled in the operating system. A typical value is: "3.6.0".

prune(integer) specifies the the size M of the optimal M-pruned tree.

cv_tree requests to use cross-validation to find the optimal tree size (i.e., optimal pruning).

prediction(new_data_filename) requests to generate the prediction of the outcome variable.
The user must specify the new_data_filename, i.e. the name of a dataset containing new observations
having the same variables' name of the original dataset, but with the dependent variable filled in with
all missing values.

in_samp_data(filename) allows to save into filename the in-sample predictions of the model.
Within filename, the user can find: "y_train"=the original outcome variable in the training dataset; "yhat_train"=the prediction of the outcome variable generated by the model in the training dataset; "Train_MSE"=the training dataset's mean square error.

out_samp_data(filename) allows to save into filename the out-of-sample predictions of the model. Within filename, the user can find: "y_test"=the original outcome variable in the test dataset; "yhat_test"=the prediction of the outcome variable generated by the model in the test dataset; "Test_MSE"=the test dataset's mean square error.

ntree(integer) specifies the number of bootstrapped trees to use in the bagging model.

mtry(integer) specifies the size of the subset of covariates to randomly choose when implementing the random forest model.

inter_depth(integer) specifies the complexity of the boosted ensemble. Often inter_depth = 1 works well,
in which case each tree is a stump, consisting of a single split and resulting in an additive model.
More generally the value of inter_depth is the interaction depth, and controls the interaction order of the boosted model, since a number of "inter_depth" splits can involve at most a number of "inter_depth" variables. This option matters only for
"boosting".

shrinkage(number) specifies the parameter controlling for the rate at which boosting learns.
Typical values are 0.01 or 0.001, and the right choice can depend on the problem. Very small values can require using a very
large value of the number of bootstrapped trees in order to achieve good performance. This option matters only for
"boosting".

pdp(covariate) produces A "partial dependence plot" for the specified covariate. This plot
illustrates the marginal effect of the selected variable on the response after integrating out the other variables. This option matters only for "boosting".

seed(integer) sets the seed number allowing for reproducing the same results.

modeltype_options

Model
tree Simple classification tree model

randomforests bagging and random forest models

boosting1 Boosting model with a binary outcome (i.e, y=0,1)

boosting2 Boosting model with a multinomial outcome (e.g., y=A,B,C)

Remarks

-> Model "boosting1" accepts only a binary outcome variable coded numerically (i.e, y=0,1).

-> Model "boosting2" accepts only a multinomial outcome variable coded as "literal string" (e.g., y=A,B,C).

-> For R to correctly read them, categorical (or factor) variables in Stata need to have "literal string values" and not "numerical string values" (for example, y=("A","B","C") is fine, but y=("1","2","3") is not fine as R will interpret these values as numbers in any case).

-> Please, before running this program, remember to have the most recent up-to-date version installed.

Example

* FITTING A CLASSIFICATION TREE

* LOAD THE DATASET AND SET PARAMETERS (i.e.: outcome, covariates, number of nodes)

. use carseats2 , clear

. global y "High"

. global xvars "CompPrice Income Advertising Population Price Age Education US Urban ShelveLoc"

. global M=6

* ESTIMATE A CLASSIFICATION TREE VIA VARIOUS MODELS

* 1. Entire classification tree + train-MSE

. sctree $y $xvars , model(tree) rversion("3.6.0") in_samp_data("IN") out_samp_data("OUT")

* 2. Tree of size M + train-MSE + test-MSE

. sctree $y $xvars , model(tree) rversion("3.6.0") prune($M) in_samp_data("IN") out_samp_data("OUT")

* 3. Cross-validation with optimal tree size

. sctree $y $xvars , model(tree) rversion("3.6.0") cv_tree in_samp_data("IN") out_samp_data("OUT")

* 4. Bagging & random-forests

. sctree $y $xvars , model(randomforests) rversion("3.6.0") mtry(5) ///

ntree(50) in_samp_data("IN") out_samp_data("OUT")

* 5. Boosting

* Recode the variable Y into [0;1] and eliminate Stata label for R to read it correctly

. encode High , gen(High2)

. recode High2 (1=0) (2=1)

. label drop _all

. global y "High2"

. sctree $y $xvars , model(boosting1) rversion("3.6.0") inter_depth(4) ntree(5000) ///

shrinkage(0.2) pdp("price") in_samp_data("IN") out_samp_data("OUT")

Reference

Gareth, J., Witten, Hastie, D.T., Tibshirani. 2013. An Introduction to Statistical Learning : with Applications in R. New York, Springer.

Author

Giovanni Cerulli

IRCrES-CNR

Research Institute for Sustainable Economic Growth, National Research Council of Italy

E-mail: giovanni.cerulli@ircres.cnr.it

Also see

Online: srtree, subset