help opl_policy_eval

Title

opl_policy_eval - Evaluate a user-supplied binary treatment policy after opl_tb_cba

Syntax

opl_policy_eval policyvar

where policyvar is a numeric binary variable coded 1 for treated
observations and 0 for untreated observations.

Description

opl_policy_eval evaluates a user-supplied binary treatment policy after
opl_tb_cba. The command uses the treatment-effect variable, treatment
cost, cost-weight parameter, and estimation sample stored by
opl_tb_cba.

For a policy indicator D_i, the command computes treatment coverage,
total treatment benefit, total treatment cost, and weighted social surplus:

Q(D;lambda) = sum_i D_i tau_i - lambda sum_i D_i c_i.

Here, tau_i is the estimated individual treatment effect,
c_i is the individual treatment cost, and lambda is inherited from
the preceding opl_tb_cba estimation.

The evaluation sample is the estimation sample from opl_tb_cba,
excluding observations with missing values in the policy, treatment-effect,
or cost variables.

opl_policy_eval may be used to evaluate externally defined policies,
benchmark policies, or policy variables generated by opl_tb_cba, such
as the estimated optimal threshold policy and the first-best policy.

Requirements

opl_policy_eval must be run after opl_tb_cba. The preceding command
must store e(tauvar), e(costvar), e(lambda), and
e(sample).

policyvar must be numeric and must contain only 0 and 1 in the evaluation
sample.

Observations with missing values in policyvar, the treatment-effect
variable, or the cost variable are excluded from evaluation.

Policy evaluation measures

The command computes the following quantities:

N number of observations in the evaluation sample

N_treated number assigned to treatment

N_untreated number assigned to no treatment

coverage share of the evaluation sample assigned to treatment

total_benefit sum of estimated treatment effects among treated observations

total_cost sum of treatment costs among treated observations

Q weighted social surplus

ATET average estimated treatment effect among treated observations

avg_cost average treatment cost among treated observations

avg_surplus average weighted surplus among treated observations

Treatment coverage is

coverage = N_treated / N.

Weighted social surplus is

Q = total benefit - lambda x total cost.

Average surplus is defined as

average surplus = Q / N_treated.

If the policy assigns no observations to treatment, total benefit, total cost,
and total welfare are set to zero, while ATET, average cost, and average
surplus are returned as missing.

Stored results

opl_policy_eval is rclass and stores the following results:

r(N) number of observations in the evaluation sample

r(N_treated) number assigned to treatment

r(N_untreated) number assigned to no treatment

r(coverage) treatment coverage

r(total_benefit) total treatment benefit

r(total_cost) total treatment cost

r(Q) weighted social surplus

r(ATET) average treatment effect among treated observations

r(avg_cost) average treatment cost among treated observations

r(avg_surplus) average weighted surplus among treated observations

r(policy) evaluated policy variable

r(tauvar) treatment-effect variable inherited from opl_tb_cba

r(costvar) cost variable inherited from opl_tb_cba

Remarks

opl_policy_eval does not estimate a treatment policy. It evaluates an
existing binary policy using the treatment-effect and cost information stored
by the latest opl_tb_cba estimation.

The command evaluates the policy on the original estimation sample. It does
not permit an independent if or in restriction.

The reported ATET is the average estimated treatment effect among observations
assigned to treatment by the evaluated policy. It should therefore be
interpreted as a policy-specific average treatment effect among the treated.

Because lambda is inherited from opl_tb_cba, all policies
evaluated after the same estimation are compared using the same cost-benefit
criterion.

Examples

Estimate a threshold-based policy before evaluating alternative binary
policies:

sysuse data_opl_tb_cba_2, clear

opl_tb_cba tau, cost(cost) select(x1 x2) lambda(1) npoints(51) custompolicy(D) custom(.30 .30) generate(opl2) replace

Evaluate an externally defined policy:

opl_policy_eval D

Evaluate the optimal threshold policy generated by opl_tb_cba:

opl_policy_eval opl2_opt

Evaluate the first-best policy generated by opl_tb_cba:

opl_policy_eval opl2_fb

Inspect the returned results after any evaluation:

return list

Acknowledgment

The development of this software was supported by FOSSR (Fostering Open
Science in Social Science Research), a project funded by the European Union -
NextGenerationEU under the NPRR Grant agreement n. MURIR0000008.

Author

Giovanni Cerulli

IRCrES-CNR

Research Institute for Sustainable Economic Growth,
National Research Council of Italy

E-mail: giovanni.cerulli@cnr.it

Also see

Online: opl