Binary classification

Setup

. sysuse auto

Machine-learning methods like SVM are very easy to overfit. To compensate,
you must split data into training and testing sets, fit on the former,
and measure performance on the latter, so that performance measurements are
not artificially inflated by data they have already seen.

After splitting, the proportion of classes can become unbalanced. The
reliable way to handle this is a stratified split, which fixes the
proportions of each class in each partition of each class. The
quick-and-dirty way is a shuffle,

. set seed 9876

. generate u = runiform()

. sort u

before the actual train and test split:

. local split = floor(_N/2)

. local train = "1/`=`split'-1'"

. local test = "`split'/`=_N'"

Fit the classification model on the training set, with verbose enabled.
Training cannot handle missing data; here we omit it, but usually you should
impute.

. svmachines foreign price-gear_ratio if !missing(rep78) in `train', verbose

Predict on the test set. Unlike training, predict can handle missing
data it simply predicts missing.

. predict P in `test'

Compute error rate: the percentage of mispredictions is the mean of err.

. generate err = foreign != P in `test'

. summarize err in `test'

(click to run)

Multiclass classification

Setup

. use attitude_indicators

Shuffle

. set seed 4532

. generate u = runiform()

. sort u

Train and test split

. local split = floor(_N*3/4)

. local train = "1/`=`split'-1'"

. local test = "`split'/`=_N'"

In general, you need to do a grid search to find good tuning parameters. These
values of kernel(), gamma(), and coef0() just happened to be good enough.

. svmachines attitude q* in `train', kernel(poly) gamma(0.5) coef0(7)

. predict P in `test'

Compute error rate.

. generate err = attitude != P in `test'

. summarize err in `test'

An overly high percentage of SVs means overfitting

. display "Percentage that are support vectors: `=round(100*e(N_SV)/e(N),.3)'"

(click to run)

Class probability

Setup

. use attitude_indicators

Shuffle

. set seed 12998

. generate u = runiform()

. sort u

Train and test split

. local split = floor(_N*3/4)

. local train = "1/`=`split'-1'"

. local test = "`split'/`=_N'"

Model

. svmachines attitude q* in `train', kernel(poly) gamma(0.5) coef0(7) prob

. predict P in `test', prob

The value in column P matches the column P_<attitude> with the
highest probability.

. list attitude P* in `test'

Compute error rate.

. generate err = attitude != P in `test'

. summarize err in `test'

Beware: predict, probability is a different algorithm than
predict and can disagree about predictions. This disagreement will
become absurd if combined with poor tuning.

. predict P2 in `test'

. generate agree = P == P2 in `test'

. summarize agree in `test'

(click to run)

Regression

Setup

. webuse highschool

Shuffle

. set seed 793742

. generate u = runiform()

. sort u

Train and test split

. local split = floor(_N/2)

. local train = "1/`=`split'-1'"

. local test = "`split'/`=_N'"

Regression is invoked with type(svr) or type(nu_svr). Notice that
you can expand factors (categorical predictors) into sets of indicator
(Boolean and dummy) columns with standard i. syntax, and you can record
which observations were chosen as support vectors with sv().

. svmachines weight height i.race i.sex in `train', type(svr) sv(Is_SV)

Examine which observations were SVs. Ideally, a small number of SVs are enough.

. tab Is_SV in `train'

. predict P in `test'

Compute residuals.

. generate res = (weight - P) in `test'

. summarize res

(click to run)