*! version 0.0.1 30 April 2021

Title

aic_model_selection -- Forward Model Selection using AIC or BIC

Syntax

aic_model_selection
command_name
varlist
,
[
bic
]

Description

aic_model_selection performs a regression on a sequence of models adding one x-variable (in the order specified) at a time. For example, when specifying
aic_model_selection regress y x1 x2 x3 the following regression models are computed:

regress y x1

regress y x1 x2

regress y x1 x2 x3

For each model aic_model_selection outputs the AIC value.

Why is this useful? Often, it is not possible to conduct all possible subsets regression
because there are too many models to consider.
The traditional alternative has been forward/stepwise/backward model selection.
While the sequence of models generated this way is unproblematic,
the p-values are not valid because they do not take the selection procedure into account.
One solution is to compute AIC for the sequence of models generated, and to choose the model with the smallest AIC.
(see, for example, James et al. (2013, Section 6.1).

aic_model_selection facilitates this computation: First, run a forward selection ("stepwise , pe(.5)").
Next, use aic_model_selection to compute the AIC (or BIC) values of the same sequence of models.
Then, choose the model with lowest AIC (or BIC) value.

command_name is intended for all commands that are allowed for sw including regress and logistic.

Options

bic Display the BIC criterion instead of the AIC criterion. Note that bic is not capitalized.

Examples

Example: First, we run a forward selection with a generous p-value (here 0.5). The p-value is generous to avoid missing the model with the lowest AIC value.
We need a list of variables added in the order they were added during forward selection. The list can be created by looking at the output of sw.
Alternatively,
Stata saves the variables added in the right sequence in r(table) and one can copy and paste the sequence of variables.
Next, we run aic_model_selection with this ordered list of variables. We choose the model with the smallest AIC value. Here, the model with the smallest AIC includes variables up to mpg.

.sysuse auto

.sw, pe(0.5): logistic foreign mpg rep78 headroom trunk weight length turn displacement gear_ratio

.matrix list r(table)

.aic_model_selection logistic foreign weight gear_ratio rep78 mpg headroom turn

(click to run)

Authors

Matthias Schonlau <schonlau@uwaterloo.ca>

References

James, Witten, Hastie, Tibshirani. 2013. An introduction to statistical learning, Section 6.1