*! version1.0.0 2021/04/22

Title

minmse -- create balanced groups for treatment in experiments with one or several treatment arms based on (possibly continuous and multiple) variables specified by the user according to the MinMSE Treatment Assignment Method (Schneider & Schlather, 2017)

Syntax

minmse
[varlist],
generate(newvar)
[options]

where varlist contains the covariate input (missing values are automatically replaced by a variable's mean), and newvar is the name of the variable to be created containing treatment group number(s).

Any of the following options might be specified:

Main
treatments(#) number of treatment groups that you want to have additional to the control group; default is treatments(1).

iterations(#) number of iterations that you want the algorithm to perform; default is iterations(50).

assignment(varname) takes a numerical vector of partial treatment assignment as argument, and assigns the missing units (where varname == .) to a treatment group while minimizing the objective function.

Options for Controlling the Optimization Procedure (Simulated Annealing)
change(#) how many units should exchange treatment in each iteration; default is change(3).

cooling(#) specify the cooling scheme for the simulated annealing algorithm to use, see details below; default is cooling(1).

t0(#) specify the starting temperature for the minimization algorithm, see details below; default is t0(10).

tmax(#) specify the number of function evaluations at each temperature for the minimization algorithm; default is tmax(10).

Plot
plot(#) supress drawing a plot showing the value of the objective function for the last iterations by setting plot(0); default is plot(1), which shows a plot.

*! option brief description of option

Description

minmse performs treatment assignment for (field) experiments based on pre-treatment information, i.e., it assigns treatment group number(s) to observations based on covariates specified by the user. It is suited for experiments with one or several treatment groups. The created treatment groups are balanced with respect to the specified covariates according to the minMSE method proposed by Schneider and Schlather (2017); the package implements their treatment assignment method (based on work by Kasy, 2016). Pre-treatment information to be considered can be continuous and multivariate (i.e., several variables). Optimization is performed using the stochastic simulated annealing algorithm (Kirkpatrick, Gelatt, and Vecchi, 1983).

Options

Main

*!

*!optionnameoption description

*!

continued option description, if necessary

*!

*!optionnamesecond option description

treatments(#) Specifies the number of treatment groups desired (in addition to the control group); minimum and default value is treatments(1).

iterations(#) specifies the number of iterations the algorithm performs; the default value is iterations(50).
With small samples and few covariates, depending on the desired method of inference, a relatively small value is recommended, see Remarks for details.
Depending on the number of units and the number of covariates to consider for group assignment, a high value could result in a long run-time.

assignment(varname)
Takes a numerical vector of partial treatment assignment as argument, and assigns the missing units (where varname == .) to a treatment group while minimizing the objective function.
Non-missing values are copied to the new vector, i.e., treatment group assignment of these observations is unaffected.

This option is helpful for, e.g., sequential treatment assignment over several days, or to cope with attrition in an experiment implemented over several days.
For example, before implementing treatment on the second day of an experiment, one might want to readjust balance for attrition on the first day.
Thus, one would create a vector with the number of treatment of the units where treatment was already implemented, and set the value to missing for the remaining ones.
(The subjects that left the experiment can be deleted from the dataset used for treatment assignment.)
Given a vector as just described, the command assigns treatment to the the missing units (where varname == .).
The goal is to minimize the objective function, where group assignment for units that already received treatment is held constant.

Controlling the Optimization Procedure (Simulated Annealing)

The following options control the minimization process performed by the simulated annealing algorithm.
The plot option might be useful for adjusting these parameters.
Default values correspond to the default values applied in the R implementation of simulated annealing (in the command optim), and, especially the default value for t0 is likely too small for many applications, see below.

change(#) is a control parameter of the simulated annealing algorithm minimizing the objective function.
It specifies how many units should exchange treatment in each iteration.
In case of big datasets (with more than 100 units), one might consider increasing the default value.

cooling(#) specifies the cooling scheme to be used for the simulated annealing algorithm.

cooling(1), which is default, sets the temperature to

t_0 / ln(floor((k-1)/t_max) * t_max + exp(1)),

whereas cooling(2) sets the temperature to the faster decreasing sequence

t_0 / (floor((k-1) / t_max) * t_max + 1).

In practice, cooling schemes are mostly of one of these forms.
Users may want to change the cooling scheme if the plot indicates a too slow deacrease of objective values.
For a theoretical discussion of cooling schemes see Belisle (1992, p. 890).
The default cooling scheme (scheme 1) is the same as for the R implementation of optim.

t0(#) sets the starting temperature for the minimization algorithm, see Belisle (1992) for theoretical convergence considerations.
In practice, a lower starting temperature t_0 decreases the acceptance rate of a worse solution more rapidly.

Specifying a negative number allows values proportional to the objective function, which we recommend.
That is: Specifying t0(-5) sets the starting temperature to 1/5 of the objective function for the starting point.
Thus, for the first t_max iterations of the algorithm, the difference of the old and the proposed solution is scaled by 1/5.

When changing the default value, it should be considered that also worse solutions have to be accepted in order for the algorithm to escape a local minimum.
Thus, it should be chosen high enough.
Default value (t0(10)) is the same as for the R implementation of optim, and likely too small for many applications.

tmax(#) specifies the number of function evaluations at each temperature:
For instance, tmax(10) makes the algorithm evaluate 10 treatment assignments that are found based on the current solution, before the temperature is decreased and thus the probability of accepting a solution that is worse than the current one is decreased. Default value is the same as for the R implementation of optim.

Plot

plot(#) can be used to suppress drawing a plot showing the value of the objective function for the last iterations by setting plot(0).
The default setting is plot(1), which shows a plot.
While the convergence plot is a helpful tool for setting the control parameters of the simulated annealing algorithm and for detecting convergence, it might be less interesting when
generating a big number of alternative treatment assignments, e.g., for performing Fisher's exact test for inference when analyzing the experiment.

Remarks

Note that the analysis of the data once the experiment has been conducted can be based on a different statistic than the one for which the MSE is to be minimized by the minmse command. Additional variables to the ones used for treatment assignment can also be included in the analysis.

The number of participants does not need to be even and no difficulties aries for uneven numbers of participants in a certain treatment group.

In case treatment is not implemented at the same time for all units and attrition is happening, the assignment option can be used for re-balancing the treatment groups, given the units and their treatment group for which treatment has already been implemented.

The command uses Stata's fast Mata language for computation, and provides a convenient single command line Stata interface for the user.

The minimization is performed by a combination of random draws for starting points and a variant of simulated annealing with default values as implemented in the R command optim; see details above.

Should the user wish to perform randomization inference after having conducted the experiment using, e.g., Fisher's exact test, they should make sure that the algorithm does not converge. Otherwise, one cannot compute the test statistic of interest in for different (hypothetical) treatment assignments, as applying the procedure various times always would lead to the same treatment assignment. Convergence can be checked for example by running the command several times with the same settings: If the program always yields the same treatment assignment, one might want to decrease the number of iterations, as the algorithm might have converged.

Note that the R implementation of this package, available via CRAN (https://cran.r-project.org/package=minMSE), has several built-in features to facilitate picking appropriate optimization parameters as well as preparing randomization inference, by, e.g., plotting a convergence graph for a desired number of treatment assignment vectors, or by specifying the number of alternative treatment assignment vectors to be used for randomization inference, and by specifying the percentage of equal treatment assignment vectors this number of vectors, which determines the highest, theoretically achievable significance level using this approach.

Examples: Basic Treatment Assignment

Setup

. webuse auto

Create a treatment and a control group, with group numbers to be stored in variable Treatment, where trunk size and weight should be balanced. The number of iterations is the default: 50

. minmse trunk weight, gen(Treatment)

Inspect the result

. tabstat weight trunk, by(Treatment) stat(mean sd p(25) p(50) p(75) n) long col(stat)

Examples: Improve Basic Treatment Assignment

Setup

. webuse auto

Create a treatment and a control group, with group numbers to be stored in variable Treatment_Improved, where trunk size and weight should be balanced. Set the number of iterations to 500, use the second cooling scheme, set the starting temperature to 1/2 of the initial value of the objective function, and decrease the temperature every $20$ iterations.

. minmse weight trunk, gen(Treatment_Improved) iterations(500) t0(-2) tmax(20) cooling(2)

Inspect the result

. tabstat weight trunk, by(Treatment_Improved) stat(mean sd min max n) long col(stat)

Examples: Sequential Treatment Assignment/Correcting for Attrition

Setup

. webuse auto

Perform initial treatment assignment as in the example above for half of the sample

. minmse weight trunk, gen(treatmentDayOne) iterations(500) t0(-2) tmax(20) cooling(2)

. gen u = runiform()

. sort u, stable

. drop u

. replace treatmentDayOne = . if _n > _N/2

To partially rebalance your treatment assigment, where the overall balance of trunk and weight should be achieved (across units that already have been assigned and potentially have been treated and those still to be treated) type

. minmse weight trunk, gen(Treatment) assignment(treatmentDayOne) iterations(500) t0(-2) cooling(2)

Author

Sebastian O. Schneider

Max Planck Institute for Research on Collective Goods, Bonn (Germany)

sschneider@coll.mpg.de

http://sebastianoschneider.com

References

Belisle, C. J. P. 1992. "Convergence theorems for a class of simulated annealing algorithms on Rd." Journal of Applied Probability 29 (4): 885-895.

Bertsimas, D., M. Johnson, and N. Kallus. 2015. "The power of optimization over randomization in designing experiments involving small samples." Operations Research 63 (4): 868-876.

Kasy, M. 2016. "Why experimenters might not always want to randomize, and what they could do instead." Political Analysis 24 (3): 324-338.

Kirkpatrick, S., C. D. Gelatt, and M. P. Vecchi. 1983. "Optimization by simulated annealing." Science 220 4598): 671-680.

Schneider, S. O., and M. Schlather. 2017. "A new approach to treatment assignment for one and multiple treatment groups." Courant Research Centre: Poverty, Equity and Growth-Discussion Papers 228.