Title

wald_mse -- Calculate the maximum mean square error (MSE) of a point-estimator of the mean, from a random
sample with missing data

Syntax

wald_mse
command_name
, options

Description

wald_mse calculates the maximum MSE of a point-estimator of the mean of a bounded outcome, from a random sample with missing data.
The MSE equals regret under square loss, so the maximum MSE is the maximum regret. In the case of bounded outcomes and no missing data,
Hodges and Lehmann (1950) derive the estimator with smallest maximum MSE; that is, the minimax-regret estimator.
With missing data, the minimax-regret estimator has no known analytical expression and numerical computation appears intractable.
wald_mse allows the user to compute the maximum MSE of any proposed estimator with a flexible specification of missing data. For an introduction
to these concepts, see Manski and Tabord-Meehan (2017).

Input

command_name specifies the name of the estimator to be used. It must be either a built-in estimator (for a list of the currently
supported estimators, see the Built-in Estimators section below), or any user-specified e-class command that returns to e(b) (for more information about using user-defined commands, see the section Tips for User-Defined Estimators below).

options specifies various features of the computation. For a list of required and discretionary options, see the Options section below.

Output

Returns the computed maximum MSE.

Options

Required Options

samp_size(#) specifies the size of the sample. This option must be specified.

dist(string) specifies type of data being considered. The two options supported are bernoulli for binary {0,1} outcomes and continuous for continuous outcomes bounded in the unit interval. The continuous distributions used in
the computations are beta distributions. This option must be specified.

Discretionary Options

In what follows, z = 1 if an outcome y is observable and z = 0 if it is missing. P(y|z = 1) and P(y|z = 0) are the population distributions
of observable and missing outcomes, while E(y|z = 1) and E(y|z = 0) are their respective means. P(z = 1) and P(z = 0) are the probabilities of response and nonresponse.

Options to specify the data generating process:

miss_l(#) specifies a lower bound on the nonresponse probability P(z = 0). For example, setting miss_l(0.5) specifies that P(z = 0) >= 0.5. The default option is 0.

miss_r(#) specifies an upper bound on the nonresponse probability P(z = 0). The default option is 1.

rdgp_l(#) specifies a lower bound on the mean E(y|z = 1) of the observable outcomes. For example, setting rdgp_l(0.5) specifies that E(y| z = 1) >= 0.5. The default option is 0.

rdgp_r(#) specifies an upper bound on the mean E(y|z = 1). The default option is 1.

mdgp_l(#) specifies a lower bound on the mean E(y|z = 0) of the missing outcomes. The default option is 0.

mdgp_r(#) specifies an upper bound on the mean E(y|z = 0). The default option is 1.

h_distance(#) specifies an upper bound on the Hellinger distance between the distributions P(y|z = 1) and P(y|z = 0). For example, setting h_distance(0) forces the distribution of missing data and
observable data to be identical, which is equivalent to making the assumption of missingness at random. The default option is 1, (equivalent to making no assumptions linking missing and observable data).

r_shape(#) specifies the shape of the distribution of the observable data, when dist is set to be continuous (recall when dist is set to continuous, the distributions
considered are beta distributions). If r_shape is set to 1, then only distributions whose modes are in the interior of the unit interval are considered.
If r_shape is set to 2, then only distributions whose densities are "u-shaped" are considered. If r_shape is set to 0, then both types are considered. The default option is 0.

m_shape(#) specifies the shape of the distribution of the missing data. See r_shape for details.

mon_select(#) specifies whether or not to make a "monotone selection" type assumption: when mon_select is set to 1, only DGPs where E(y|z=0) >= E(y|z=1) are considered. When mon_select is set to 2, only DGPs where E(y|z=1) >= E(y|z=0)
are considered. When mon_select is set to 0, both options are considered. The default option is 0.

Options to specify the accuracy of the computation:

mc_iter(#) specifies the number of Monte Carlo iterations used to compute the MSE of the estimator in repeated samples. The default option for built-in estimators is 3000, the default option for
user-defined commands is 300.

grid(#) specifies the number of grid-points used to generate the distributions. For example, if the outcome is binary and grid(6) is specified, the resulting grid of Bernoulli parameters for the distributions would be {0,0.2,0.4,0.6,0.8,1}.
The default option for binary outcomes is 25, the default option for continuous outcomes is 5.

Additional options:

user_def specifies whether or not the estimator being used is built-in or user-defined. Setting user_def specifies that the estimator to be used is user-defined.

true_beta specifies whether or not certain continuous distributions should be approximated by bernoulli distributions. As explained in the Manski and Tabord-Meehan (2017),
in general, specifying true_beta will slow down the program but may increase accuracy.

Built-in Estimators

mean is simply the sample mean.

midmean is an estimator that first estimates the identified interval under no assumptions on the missing data, and then selects the midpoint of the interval. See Dominitz and Manski (2016) for details.

MMRzero is the minimax-regret estimator of the mean with no missing data, as derived in Hodges and Lehmann (1950).

Examples

Evaluating maximum MSE of the sample mean, for continuous outcomes, with a sample size of 20, and where the amount of missingness considered is between 0.2 and 0.8.

. wald_mse mean, samp_size(20) dist("continuous") miss_l(0.2) miss_r(0.8)

Evaluating maximum MSE of the midpoint mean estimator, for binary outcomes, with a sample size of 50, where the amount of missingness considered is between 0 and 0.5, and the Hellinger distance between distributions is at most 0.6.

. wald_mse midmean, samp_size(50) dist("bernoulli") miss_l(0) miss_r(0.5) h_distance(0.6)

Tips for User-Defined Estimators

wald_mse allows the user to evaluate the maximum MSE of user-defined estimators of the mean, as long as they are e-class commands that return their result to e(b). User-defined estimators are in general much slower than built-in estimators, so by default
wald_mse will do the computations with less accuracy than with a built-in estimator. The option user_def must be specified when using user-defined estimators.
When evaluating a user-defined estimator, we recommend first running the computation with binary outcomes, since this will be much faster and frequently achieves maximum MSE.

Stored Results

wald_mse stores the following in r():

Scalars:
r(MSE) the computed maximum MSE

r(rmeanval) the value of E(y|z=1) where maximum MSE is achieved

r(mmeanval) the value of E(y|z=0) where maximum MSE is achieved

r(missval) the value of P(z=0) where maximum MSE is achieved

r(N) the specified sample size

r(missr) the specified upper bound on P(z = 0)

r(missl) the specified lower bound on P(z = 0)

r(mdgpr) the specified upper bound on E(y|z = 0)

r(mdgpl) the specified lower bound on E(y|z = 0)

r(rdgpr) the specified upper bound on E(y|z = 1)

r(rdgpl) the specified lower bound on E(y|z = 1)

r(hd) the specified bound on Hellinger distance between P(y|z = 0) and P(y|z = 1)

r(mshape) the specified shape option for P(y|z = 0), if applicable

r(rshape) the specified shape option for P(y|z = 1), if applicable

Macros:
r(est) the estimator used

r(cmd) the name of this command: wald_mse

r(d) the specified distribution type

References

- Dominitz, J. and Manski, C. (2016). More Data or Better Data? A Statistical Decision Problem. Manuscript.

- Hodges, E. and Lehmann, E. (1950). Some Problems in Minimax Point Estimation. Annals of Mathematical Statistics.

- Manski, C. and Tabord-Meehan, M. (2017). wald_mse: Evaluating the Maximum MSE of Mean Estimates with Missing Data. Mansuscript.

Authors

Chuck Manski, Northwestern University, cfmanski@northwestern.edu

Max Tabord-Meehan, Northwestern University, mtabordmeehan@u.northwestern.edu