*! Version 6, G. Cerulli, September 30, 2025

Title

opl_ma_fb -- Optimal Policy Learning for Multi-Action Treatment using First-Best Policy and Risk Preference

Syntax

opl_ma_fb depvar indepvars ,
policy_train(varname)
model(string)
name_opt_policy(name)
[match_name(name)
new_data(name)
policy_non_optimal_train(varname)
policy_non_optimal_new(varname)
save_preds_vars(name)
gr_action_train(name)
gr_reward_train(name)
gr_reward_new(name)]

Description

opl_ma_fb implements first-best Optimal Policy Learning (OPL) algorithm to
estimate the best treatment assignment given an outcome and a set of observed covariates
and treatment effects. It allows for different risk preferences in decision-making
(i.e., risk-neutral, risk-averse linear, risk-averse quadratic). This command uses linear regression for estimating nuisance conditional means.

Options

Required

policy_train(varname) specifies the treatment variable, which must contain consecutive integers starting from 0 (e.g., 0,1,2,...,M).

model(string) specifies the decision model:

risk_neutral: considers only expected reward (no variance or risk are accounted for).

risk_averse_linear: adjusts reward by a linear function of its variance.

risk_averse_quadratic: adjusts reward by a quadratic function of its variance.

name_opt_policy(name) specifies the name of the generated variable containing the estimated optimal policy.

Optional

match_name(name) specifies the name of the variable that stores whether the actual treatment matches the optimal one.

new_data(name) provides a second dataset to predict optimal actions for new units. This dataset contains the same features as the training dataset.

policy_non_optimal_train(varname) is an alternative (non-optimal) policy to compare against either training or optimal policy within training data.

policy_non_optimal_new(varname) is an alternative (non-optimal) policy to compare against optimal policy within new data.

save_preds_vars(name) saves conditional expectations and variances.

gr_action_train(name) saves a graph comparing actual vs. optimal action allocation in the training dataset.

gr_reward_train(name) saves a graph comparing actual vs. maximal expected reward in the training dataset.

gr_reward_new(name) saves a graph showing maximal expected reward for new policy observations.

Returns

Scalars

e(N_train) Number of observations in the training dataset

e(N_new) Number of observations in the new (unlabeled) dataset

e(N_train_opt_pol) Number of observations for computing the optimal policy in the training dataset

e(V_train) Value function in the training dataset

e(N_V_train) Number of observations for computing the value function in the training dataset

e(V_non_opt_train) Value function in the training dataset for the non-optimal policy

e(N_V_non_opt_train) Number of observations for computing the value function in the non-optimal training dataset

e(V_opt_train) Value function with optimal policy in the training dataset

e(N_V_opt_train) Number of observations for computing with optimal policy the value function in the training dataset

e(V_opt_new) Value function in the new dataset for the optimal policy

e(N_V_opt_new) Number of observations for computing the value function in the new dataset for the optimal policy

e(rate_opt_match) Rate of matches between the optimal and the current training policy

Generated variables

_index: indicator variable specifying the dataset source of each observation (0 = training data; 1 = new data).

_opt_policy: the estimated optimal policy rule, assigning to each unit the treatment that maximizes expected welfare.

_Y_hat_policy_train: the predicted outcome under the actual (observed) training policy, i.e. the historical assignment rule applied in the data.

_Y_hat_policy_train_non_optimal: the predicted outcome under a given non-optimal policy provided in the training set, used as a benchmark for comparison.

Y_hat_policy_optimal: the predicted outcome under the estimated optimal policy, i.e. the counterfactual outcome distribution if all units had followed the  rst-best policy.

_match_var: an indicator variable equal to 1 if the actual treatment coincides with the estimated optimal treatment, and 0 otherwise. It measures the rate of alignment between historical and optimal assignments.

Examples

Example: Basic usage with a risk-neutral model

Generate the initial dataset by simulation:

clear all

set obs 100

set seed 1010

generate A = floor(runiform()*3)

gen x1 = rnormal()

gen x2 = rnormal()

gen y = 100*runiform()

Split the dataset into training and testing (i.e., new data):

get_train_test , dataname(mydata) split(0.60 0.40) split_var(svar) rseed(101)

Run opl_ma_fb with risk-neutral preferences

opl_ma_fb y x1 x2 , policy_train(A) model(risk_neutral) name_opt_policy(opt_policy) new_data(mydata_test) match_name(match_var) gr_action_train(action_graph) gr_reward_train(reward_graph)

References

Athey, S., and Wager S. 2021. Policy Learning with Observational Data, Econometrica, 89, 1, 133–161.

Cerulli, G. 2021. Improving econometric prediction by machine learning, Applied Economics Letters, 28, 16, 1419-1425.

Cerulli, G. 2022. Optimal treatment assignment of a threshold-based policy: empirical protocol and related issues, Applied Economics Letters, 30, 8, 1010-1017.

Cerulli, G. 2023. Fundamentals of Supervised Machine Learning: With Applications in Python, R, and Stata, Springer, 2023.

Cerulli, G. 2024. Optimal policy learning with observational data in multi-action scenarios: Estimation, risk preference, and potential failures. arXiv preprint, arXiv:2403.20250. https://arxiv.org/abs/2403.20250.

Cerulli, G. 2025. Optimal policy learning using Stata. The Stata Journal, 25, 2, 309-343.

Gareth, J., Witten, D., Hastie, D.T., Tibshirani, R. 2013. An Introduction to Statistical Learning : with Applications in R. New York, Springer.

Kennedy, E. H. 2023. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, 17, 2, 3008-3049.

Kitagawa, T., and A. Tetenov. 2018. Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice, Econometrica, 86, 2, 591–616.

Kunzel, S. R., Sekhon, J. S., Bickel, P. J., Yu, B. (2019). Metalearners for estimating heterogeneous treatment effects using machine learning.
Proceedings of the National Academy of Sciences of the United States of America, 116, 10, 4156-4165.

Acknowledgment

The development of this software was supported by: FOSSR (Fostering Open Science in Social Science Research), a project funded by the European Union - NextGenerationEU under the NPRR Grant agreement n. MURIR0000008; PRIN Project RECIPE (Linking Research Evidence to Policy Impact and Learning: Increasing the Effectiveness of Rural Development Programmes Towards Green Deal Goals), MUR code: 20224ZHNXE.

Author

Giovanni Cerulli

IRCrES-CNR

Research Institute for Sustainable Economic Growth, National Research Council of Italy

E-mail: giovanni.cerulli@cnr.it

Also see

Online: make_cate, opl_tb, opl_lc, opl_lc_c, opl_dt, opl_dt_c