*! version 30aug2024

Partially-linear model - Detailed example with stacking regression using pystacked

Preparation: we load the data, define global macros and set the seed.

. use https://github.com/aahrens1/ddml/raw/master/data/sipp1991.dta, clear

. global Y net_tfa

. global D e401

. global X tw age inc fsize educ db marr twoearn pira hown

. set seed 42

We next initialize the ddml estimation and select the model.
partial refers to the partially linear model.
The model will be stored on a Mata object with the default name "m0"
unless otherwise specified using the mname(name) option.

We set the number of random folds to 2 so that
the model runs quickly. The default is kfolds(5). We recommend
considering at least 5-10 folds and even more if your sample size is small.

We recommend re-running the model multiple times on
different random folds; see options reps(integer).
Here we set the number of repetions to 2, again only so that the model runs quickly.

. ddml init partial, kfolds(2) reps(2)

Stacking regression is a simple and powerful method for
combining predictions from multiple learners.
Here we use pystacked with the partially linear model,
but it can be used with any model supported by ddml.

Note: the additional support provided by ddml for pystacked (see above)
is available only if, as in this example, pystacked is the only learner for each conditional expectation.
Mutliple learners are provided to pystacked, not directly to ddml.

Add supervised machine learners for estimating conditional expectations.
The first learner in the stacked ensemble is OLS.
We also use cross-validated lasso, ridge and two random forests with different settings,
which we save in the following macros:

. global rflow max_features(5) min_samples_leaf(1) max_samples(.7)

. global rfhigh max_features(5) min_samples_leaf(10) max_samples(.7)

. ddml E[Y|X]: pystacked $Y $X || method(ols) || method(lassocv) || method(ridgecv) || method(rf) opt($rflow) || method(rf) opt($rfhigh), type(reg)

. ddml E[D|X]: pystacked $D $X || method(ols) || method(lassocv) || method(ridgecv) || method(rf) opt($rflow) || method(rf) opt($rfhigh), type(reg)

Note: Options before ":" and after the first comma refer to ddml.
Options that come after the final comma refer to the estimation command.
Make sure to not confuse the two types of options.

Check if learners were correctly added:

. ddml desc, learners

Cross-fitting: The learners are iteratively fitted on the training data.
This step may take a while, depending on the number of learners, repetitions, folds, etc.
In addition to the standard stacking done by pystacked,
also request short-stacking to be done by ddml.
Whereas stacking relies on (out-of-sample) cross-validated predicted values
to obtain the relative weights for the base learners,
short-stacking uses the (out-of-sample) cross-fitted predicted values.

. ddml crossfit, shortstack

Finally, we estimate the coefficients of interest.

. ddml estimate, robust

Examine the standard (pystacked) stacking weights as well as the ddml short-stacking weights.

. ddml extract, show(stweights)

. ddml extract, show(ssweights)

Replicate the ddml estimate short-stacking results for resample 2 by hand,
using the estimated conditional expectations generated by ddml,
and compare using ddml estimate, replay:

. cap drop Yresid

. cap drop Dresid

. gen double Yresid = $Y - Y_net_tfa_ss_2

. gen double Dresid = $D - D_e401_ss_2

. regress Yresid Dresid, robust

. ddml estimate, mname(m0) spec(ss) rep(2) notable replay

Obtain the estimated coefficient using ridge - the 3rd pystacked learner -
as the only learner for the 2nd cross-fit estimation (resample 2),
using the estimated conditional expectations generated by ddml and pystacked.
This can be done using ddml estimate with the y(.) and d(.) options:
"L3" means the 3rd learner and "_2" means resample 2.
Then replicate by hand.

. ddml estimate, y(Y1_pystacked_L3_2) d(D1_pystacked_L3_2) robust

. cap drop Yresid

. cap drop Dresid

. gen double Yresid = $Y - Y1_pystacked_L3_2

. gen double Dresid = $D - D1_pystacked_L3_2

. regress Yresid Dresid, robust