*! version 1.0 March 2014

Title

mipllest -- Parallel estimation for multiple imputed datasets (Requires George Vega Yon's -parallel-)(Version 1.0)

Syntax

Obtain MI estimates similar to Stata's -mi estimate-

mipllest [, options] : estimation_command ...

To debug by viewing logs of estimations done in different clusters...

mipllest debug [, debug_options]

Main
imputations(numlist) specify which imputations to use

stub(name) specify the stub of the estimation results

keep keep auxiliary files generated by mipllest

replace allow existing estimates in memory to be overwritten

pllopts(parallel_options) Options to be passed to parallel

You must mi set your data before using mipllest;
see [MI] mi set.

debug_options

pll_id(pll_id) specify pll_id

pll_dir(pll_dir) specify pll_dir

Description

mipllest allows parallel estimation of
estimation_command by making use of
George Vega Yon's -parallel-,
which starts multiple instances of Stata to perform parallel processing
and therefore does not require Stata MP. mipllest saves estimation results using
estimates store. Results from estimation are saved with names
stub1, stub2, etc., where stub is either a user-supplied name or
_mipllest_. Estimates from different imputed datasets can be combined together using
mirubin.

Before using mipllest users are advised to familiarize themselves with the behaviour of
parallel.

Options

options

imputations(numlist) specifies which imputations to use. The
default is to use all of them.

stub(name) specifies the stub of names of the estimation results from the
different imputed datasets. If stub(name) is not specified, it defaults to
_mipllest_. After running mipllest, type "est dir stub*"
to see the estimated results.

keep specifies that auxiliary files created by mipllest be kept.
mipllest causes each of the clusters to run the estimation in parallel
and save the results as temporary _mipllest_#.ster files using estimates save,
where # are the numbers
given in imputations(numlist) or else the values of _mi_m. keep requests that
these files be kept, as well as the _mipllest_script.do file which is run by the
parallel clusters.

replace specifies that the estimation results stub1, stub2, etc. can be over-written.

pllopts(parallel_options) passes additional options to parallel. A
particularly useful command to pass is keep, which requests that the auxiliary
files produced by parallel be kept also.

debug_options

Usually, mipllest debug can only be used after we have run mipllest with
the pllotps(keep) option. This option requests that the log files created by
parallel be kept. The log files show the log of the estimation commands performed
in the different clusters, and as of 26/3/2014, have the form
__pllpll_id_do#.log, where pll_id is a unique ID given
to a particular parallel task and # is the cluster number. mipllest debug then brings up
these log files in the viewer to assist debugging.

pll_id(pll_id) specifies the pll_id which identifies the log files created by
parallel that are to be displayed. If pll_id(pll_id) is not specified, then
mipllest debug searches for r(pll_id) from the previous parallel
command. Sometimes, r(pll_id) may not exist, or is overwritten by subsequent commands.
This is especially the case where parallel ended with a break or an error.
However, in these instances the log files are often kept, even if the user did not specify the
pllopts(keep) option. Here, users can supply r(pll_id)
manually to identify them.

pll_dir(pll_dir) specifies the directory where the log files created by
parallel are kept. If pll_id(pll_id) is not specified, then
mipllest debug searches for r(pll_dir) from the previous parallel
command, and failing that, assumes the current directory.

Known issues

. Datasets with temporary variables, especially "__000000" and "__000001" can
somestimes (but not always) cause estimation to fail in the clusters.

. In theory, users can supply their own estimation commands instead of those
listed. However, this should be done in extreme caution
as during testing it has been found that some commands such as preserve, or
mi xeq can cause interference between clusters when running parallel. Likewise,
the use of tempfile should be avoided.

Example

parallel setclusters 2

webuse mheart1s20

mi describe

mipllest: logit attack smokes age bmi hsgrad female

est dir _mipllest_*

Stored results

mipllest does not create additional r() or e() results. However,
the r() macros stored when using parallel are retained. The macros
r(pll_id) is particularly useful. See the options for pll_id(pll_id) above.

Author

Timothy Mak, University of Hong Kong
tshmak@hku.hk
March 2014