help mcmcsum-------------------------------------------------------------------------------

Title

mcmcsum-runmlwinpostestimation - MCMC summary statistics and plots

mcmcsumin parameter mode

mcmcsumparameter_list[if] [in] [,optionsgetchains]

mcmcsumin variable mode

mcmcsumvarlist[if] [in] [,options]variables

optionsDescription ------------------------------------------------------------------------- Parameter mode only optionsgetchainssave the MCMC parameter chains from the currentrunmlwinestimation results as variables in the current data setVariable mode only options

variablesreads the data from the variables currently in memory instead of the current estimation resultsMain

sqrtbase MCMC summary statistics on square rooted MCMC chainseformbase MCMC summary statistics on exponentiated MCMC chainsmodereport MCMC chain modes rather than meansmedianreport MCMC chain medians rather than meanszratioreport classical z-ratios and p-valueslevel(#)set credible level; default is level(95)width(#)parameter/variable width; default iswidth(12)detaildisplay additional MCMC statisticsGraphics

trajectoriestrajectory plot for each MCMC chaindensitieskernel density plot for each MCMC chainfivewaytrajectory, kernal density, ACF, PACF, and MCSE plots for a single chosen MCMC chainAdvanced options

thinning(#)specifies that thinning every # iterations was used when storing the MCMC chains -------------------------------------------------------------------------

mcmcsumis a postestimation command forrunmlwin.mcmcsumcalculates and displays a variety of MCMC summary statistics and plots. These statistics and plots can be based on either the currentrunmlwinestimation results or the variables currently in memory.

+-----------------------------+ ----+ Parameter mode only options +--------------------------------------

getchainssave the MCMC parameter chains from the currentrunmlwinestimation results as variables in the current data set. Note this will overwrite the current data set.

+----------------------------+ ----+ Variable mode only options +---------------------------------------

variablesreads the data from the variables currently in memory instead of the current estimation results. The latter is useful if the MCMC chains have been saved to disk.

+------+ ----+ Main +-------------------------------------------------------------

sqrtbase MCMC summary statistics on square rooted MCMC chains. This is useful for MCMC variance parameter chains.

eformbase MCMC summary statistics on exponentiated MCMC chains. This is useful for parameters fitted on the log-odds and log scales (i.e. multilevel logit and poisson models).

modereports the modes of the MCMC chains rather than the means.

medianreports the medians of the MCMC chains rather than the means.

zratioreports classical z-ratios and p-values (i.e. under the assumption that the chains are normally distributed)

level(#)set credible level; default islevel(95).

width(#)parameter/variable width; default iswidth(12).

detaildisplay additional MCMC statistics including various percentiles, the Raftery Lewis statistics and the Brooks Draper statistic.

+----------+ ----+ Graphics +---------------------------------------------------------

trajectoriesdisplay a trajectory plot for each MCMC chain.

densitiesdisplays a kernel density plot for each MCMC chain.

fivewaydisplays a five-way plot containing the MCMC trajectory plot, kernel density plot, ACF plot, PACF plot, and MCSE plot for the chosen MCMC chain. The trajectory and kernel density plots are as described above. Note that only one MCMC chain can be specified when using this option.

+------------------+ ----+ Advanced options +-------------------------------------------------

thinning(#)specifies that thinning every # iterations was used when storing the MCMC chains.

Remarks are presented under the following headings:

Remarks on referencing specific parameters when using parameters mode Remarks on referencing specific parameters when using variables mode Remarks on MCMC summary statistics Remarks on MCMC plots

Remarks on referencing specific parameters when using parameters modeYou can find the names assigned to parameters by

runmlwinusing thematlist e(b)command. For example, if your model contains the parameter FP1:cons, you would refer to this as [FP1]cons. Similarly, the parameter RP2:var(cons) would be referred to as [RP2]var(cons). See theExamplessection for an example.

Remarks on referencing specific parameters when using variables modeAn alternative to referencing the parameters in the current

runmlwinestimation results is to use thegetchainsoption to save these parameter chains as variables in the current data set. Note this will overwrite the current data set. For example, if your model contains the parameter FP1:cons, this would be saved as the variable FP1_cons in your current data set. Similarly, the parameter RP2:var(cons) would be saved as RP2_var_cons_. See theExamplessection for an example.

Remarks on MCMC summary statistics

mcmcsumcalculates and displays a variety of MCMC summary statistics.(1) The chain mean (posterior mean). This gives the parameter point estimate.

(2) The MCSE of this mean. The MCSE will decrease as the chain length is increased.

(3) The chain standard deviation. This gives the parameter standard error.

(4) The chain mode.

(5) The proportion of chain values of the opposite sign to the chain mean.

(6) The proportion of chain values of the opposite sign to the chain mode.

(7) The proportion of chain values of the opposite sign to the chain median.

(8) The 0.5th, 2.5th, 5th, 25th, 50th, 75th, 95th, 97.5th, and 99.5th quantiles. The 2.5th and 97.5th quantiles give the 95% Bayesian credible interval. This is equivalent to a 95% confidence interval in maximum likelihood. Unlike maximum likelihood, this is not based on a normal sampling distribution assumption.

(9) The thinned chain length.

(10) The Effective Sample Size (ESS) gives an estimate of the equivalent number of independent iterations that the chain represents. The ESS will typically be less than the number of actual iterations because the chain is positively autocorrelated (it is a Markov chain).

(11) Brooks-Draper (mean): This statistic is based on the mean of the distribution. It is used to estimate the length of chain required to produce a mean estimate to 2 significant figures with a given accuracy.

(12) Raftery-Lewis (quantile): This statistic is based on the 2.5th and 97.5th quantiles of the posterior distribution (i.e. the 95% credible interval). It is used to estimate the length of chain required to estimate the boundaries of the 95% credible interval to a given accuracy.

We recommend users seeking further information to consult the comprehensive MLwiN MCMC manual by Browne (2012).

mcmcsumcalculates and displays a variety of MCMC plots.Trajectory plots can be thought of as "time series" plots of each chain. The chain values are plotted against the iteration number. Healthy chains are those that resemble white noise.

Kernel density plots are smoothed histograms of the chains. They plot the posterior distributions, the fundamental things of interest. Note that posterior distributions for variance parameters will typically be right skewed.

The ACF plot shows the autocorrelation between iteration t and t - k. A Markov chain should have a power relationship in the lags i.e. if ACF(1) = rho then ACF(2) = rho^2 etc. This is known as an AR(1) process. The less correlated the chain the better.

The PACF plot shows the autocorrelation between iteration t and t - k, having accounted for t - 1,...,t - (k - 1). It is used to identify the extent to which the chain departs from an ACF(1). That is, it is used to identify the extent of the lag in the chain. Look for the point on the plot where the partial autocorrelations for all higher lags are essentially zero.

The MCSE is an indication of how much error is in the mean estimate due to the fact that MCMC is used. As the number of iterations increases the MCSE tends to 0. The MCSE is used to calculate how long to run the chain to achieve a mean estimate with a particular desired MCSE.

We recommend users seeking further information to consult the comprehensive MLwiN MCMC manual by Browne (2012).

The following examples will only work on your computer if you have installed

runmlwin.Two-level random-intercept model, analogous to xtreg --------------------------------------------------------------------------- Setup

. use http://www.bristol.ac.uk/cmm/media/runmlwin/tutorial, clearFit model using IGLS

. runmlwin normexam cons standlrt, level2(school: cons)level1(student: cons) nopauseFit model using MCMC

. runmlwin normexam cons standlrt, level2(school: cons)level1(student: cons) mcmc(on) initsprevious nopauseCalculate and display MCMC summary statistics for all model parameters

. mcmcsumCalculate and display additional MCMC summary statistics for all model parameters

. mcmcsum, detailTrajectory plots for all model parameters

. mcmcsum, trajectoriesKernel density plots for all model parameters

. mcmcsum, densitiesFiveway plot for the level 2 variance parameter ([RP2]var(cons))

. mcmcsum [RP2]var(cons), fivewaySave the MCMC parameter chains from the current

runmlwinmodel as variables in the current data set. mcmcsum, getchainsCompute the intraclass correlation (a non-linear combination of model parameters)

. gen icc = RP2_var_cons_/(RP2_var_cons_ + RP1_var_cons_)Calculate and display a variety of MCMC summary statistics for the derived ICC parameter

. mcmcsum icc, variablesFiveway plot for the ICC parameter

. mcmcsum icc, fiveway variables

mcmcsumsaves the following inr()when noplotis specified:Scalars

r(thinnedchain)length of chain after thinningr(mean)mean of parameter chainr(mode)mode of parameter chainr(sd)standard deviation of parameter chainr(ess)effective sample sizer(meanmcse)mean Monte-Carlo standard errorr(bd)Brook-Draper diagnostic statisticr(rlub)Raftery-Lewis upper boundr(rllb)Raftery-Lewis lower boundr(p99_5)99.5% quantile of the chainr(p95)95% quantile of the chainr(p75)75% quantile of the chainr(p50)50% quantile (median) of the chainr(p25)25% quantile of the chainr(p5)5% quantile of the chainr(p2_5)2.5% quantile of the chainr(p0_5)0.5% quantile of the chain

About the Centre for Multilevel ModellingThe MLwiN software is developed at the Centre for Multilevel Modelling. The Centre was established in 1986, and has been supported largely by project grants from the UK Economic and Social Research Council. The Centre has been based at the University of Bristol since 2005.

The Centre’s website:

http://www.bristol.ac.uk/cmm

contains much of interest, including new developments, and details of courses and workshops. This website also contains the latest information about the MLwiN software, including upgrade information, maintenance downloads, and documentation.

The Centre also runs a free online multilevel modelling course:

http://www.bristol.ac.uk/cmm/learning/course.html

which contains modules starting from an introduction to quantitative research progressing to multilevel modelling of continuous and categorical data. Modules include a description of concepts and models and instructions of how to carry out analyses in MLwiN, Stata and R. There is a also a user forum, videos and interactive quiz questions for learners’ self-assessment.

Citation ofrunmlwinand MLwiN

runmlwin(andmcmcsum) is not an official Stata command. It is a free contribution to the research community, like a paper. Please cite it as such:Leckie, G. and Charlton, C. 2011.

runmlwin: Stata module for fitting multilevel models in the MLwiN software package. Centre for Multilevel Modelling, University of Bristol.Similarly, please also cite the MLwiN software:

Rasbash, J., Charlton, C., Browne, W.J., Healy, M. and Cameron, B. 2009. MLwiN Version 2.1. Centre for Multilevel Modelling, University of Bristol.

For models fitted using MCMC estimation, we ask that you additionally cite:

Browne, W.J. 2012. MCMC Estimation in MLwiN, v2.26. Centre for Multilevel Modelling, University of Bristol.

Please use the

runmlwinuser forum to post any questions you have aboutmcmcsum(orrunmlwin). We will try to answer your questions as quickly as possible, but where you know the answer to another user's question please also reply to them!http://www.cmm.bristol.ac.uk/forum/

Chris Charlton Centre for Multilevel Modelling University of Bristol c.charlton@bristol.ac.uk

George Leckie Centre for Multilevel Modelling University of Bristol

The code to calculate the MCMC summary statistics was adapted from that written by Bill Browne for the MCMC engine in the MLwiN software (Browne, 2012). We are very grateful to colleagues at the Centre for Multilevel Modelling and the University of Bristol for their useful comments.

The development of this command was funded under the LEMMA project, a node of the UK Economic and Social Research Council's National Centre for Research Methods (grant number RES-576-25-0003).

mcmcsumcomes with no warranty. Where users are usingmcmcsumafter fitting a model byrunmlwin, we recommend that users check their results with those obtained through operating MLwiN by its graphical user interface.

Browne, W.J. 2009. MCMC Estimation in MLwiN, v2.13. Centre for Multilevel Modelling, University of Bristol. http://www.bristol.ac.uk/cmm/software/mlwin/download/manuals.html

Also seeOnline:

runmlwin,savewsz,reffadjust,winbugs