{smcl}
{* February 25th 2022}{...}
{hline}
{cmd:mstfreq} {hline 2} Effect Size calculation for Multisite Randomised Trials
{hline}
{marker syntax}{...}
{title:Syntax}
{cmd:mstfreq} {varlist} {ifin}{cmd:,} {opt int:ervention(interv_var)} {opt ran:dom(clust_var)} [{it:options}]
{synoptset 20 tabbed}{...}
{synopthdr: main}
{synoptline}
{synopt :{opt int:ervention()}}requires a factor variable identifying the intervention (arms) of the trial.{p_end}
{synopt :{opt ran:dom()}}requires a factor variable identifying the clusters (Schools) of the trial.{p_end}
{synoptline}
{synoptset 20 tabbed}{...}
{synopthdr}
{synoptline}
{syntab:Model}
{synopt :{opt ml}}fits model via maximum likelihood; default is RMLE.{p_end}
{synopt :{opt seed(#)}}seed number; default is 1020252.{p_end}
{synopt :{opt np:erm(#)}}number of permutations; default is NULL. {p_end}
{synopt :{opt nb:oot(#)}}number of bootstraps; default is NULL. {p_end}
{synopt :{opt case(# [#])}}level of case bootstrap resampling with option to use both levels; default is 1. {p_end}
{synopt :{opt res:idual}}residual bootstrap; default is case bootstrap. {p_end}
{synopt :{opt perc:entile}}percentile confidence interval for bootstrap. {p_end}
{synopt :{opt basic}}basic confidence interval for bootstrap; default is percentile. {p_end}
{synopt :{opt *}}additional maximization options such as {cmd:technique()}, {cmd:difficult}, {cmd:iterate()}. {p_end}
{syntab:Reporting}
{synopt :{opt noi:sily}}displays the calculation of conditional models.{p_end}
{synopt :{opt nodot}}suppresses display of dots; default is one dot character every 10 replications.{p_end}
{synopt :{opt paste}}attaches bootstrapped/permutated effect sizes on the existing dataset.{p_end}
{synoptline}
{phang}
{it:varlist} and {it:interv_var} may contain factor-variable operators; see {help fvvarlist}.{p_end}
{phang}
{marker description}{...}
{title:Description}
{pstd}
{cmd:mstfreq} Performs analysis of multisite randomised trials using a multilevel model under a frequentist setting.
This analysis produces {cmd:Effect Size} (ES) estimates for both conditional and unconditional model specifications. It also allows for sensitivity analysis options such as permutations
and bootstraps.
{marker options}{...}
{title:Options}
{dlgtab:Model}
{phang}
{opt ml} Calculates model using maximum likelihood estimation; default is Restricted Maximum Likelihood.
{phang}
{opt seed(#)} Sets seed number for permutations/bootstraps.
{phang}
{opt nperm(#)} Specifies number of permutations required to generate permutated p-values; see Stored results.
If specified with {cmd:paste}, a list of generated variables attaches to the user's dataset containing the permutated effect sizes.
{phang}
{opt nboot(#)} Specifies number of bootstraps required to generate the bootstrap confidence intervals.
If specified with {cmd:paste}, a list of generated variables attaches to the user's dataset containing the bootstrapped effect sizes.
{phang2}
{cmd:Note:} When running bootstraps or permutations, use {cmd:iterate()} to reduce the maximum number of log-likelihood iterations in order to induce quicker failure in the event of non-convergence; Stata default is {cmd:iterate(16000)}.
To identify non-convergence you can also specify the option {cmd:noisily}. Supplementary permutations/bootstraps are automatically deployed when # number of permutated/bootstrapped models have failed to converge.
{phang}
{opt case(# [#])} Specifies case bootstrap with a specific resampling structure. The numlist may take the value(s) of integers 1 and/or 2 indicating which level should be resampled with replacement from.
If {cmd:case(1,2)} is selected, then resampling will first take place at school level (2) followed by resampling at the pupil level (1); default is {cmd:case(1)}. {p_end}
{phang}
{opt res:idual} Specifies type of bootstrap as the non-parametric residual bootstrap; default is case bootstrap. {p_end}
{phang}
{opt perc:entile} Specifies use of the percentile bootstrap confidence interval. {p_end}
{phang}
{opt basic} Specifies use of the basic (Hall's) bootstrap confidence interval; default is percentile. {p_end}
{phang}
{cmd:*} Additional maximization options are allowed including {cmd:technique()}, {cmd:difficult} see {helpb maximize:[R] Maximize}. {p_end}
{dlgtab:Reporting}
{phang}
{opt noisily} Displays the permutated/bootstrapped conditional models' regression results as they occur.
{phang}
{opt nodot} Suppresses display of dots indicating progress of permutations/bootstraps. Default is one dot character displayed for every block of 10 replications. {p_end}
{phang}
{opt paste} Attaches bootstrapped or permutated effect sizes to existing dataset if nperm (PermC/Unc_I#_W/T) or nboot (BootC/Unc_I#_W/T) has been specified,
where I# denotes number of interventions, C/Unc denotes Conditional and Unconditional estimates and W/T within and total effect sizes. Existing variables generated from previous use are replaced. {p_end}
{marker Examples}{...}
{title:Examples}
{hline}
{pstd}Setup:{p_end}
{phang2}{cmd:. use mstData.dta, clear}{p_end}
{pstd}Simple model:{p_end}
{phang2}{cmd:. mstfreq Posttest Prettest, int(Intervention) ran(School)}{p_end}
{pstd}Model using residual bootstraps including factor parameters and additional maximization options with base level change:{p_end}
{phang2}{cmd:. mstfreq Posttest Prettest i.School, int(ib2.Intervention) ran(School) nboot(3000) res iterate(100)}{p_end}
{pstd}Model using permutations with three-arm intervention variable, additional maximization options and attaching permutated effect sizes to existing dataset:{p_end}
{phang2}{cmd:. mstfreq Posttest Prettest, int(Intervention2) ran(School) nperm(3000) iterate(100) paste}{p_end}
{marker results}{...}
{title:Stored results}
{pstd}
{cmd:mstfreq} stores the following in {cmd:r()}:
{synoptset 20 tabbed}{...}
{p2col 5 20 24 2: Matrices}{p_end}
{synopt:{cmd:r(CondES#)}}conditional Hedgesâ€™ g effect size and its 95% confidence intervals for # number of arms in {it:interv_var}. If nboot is specified, CIs are replaced with bootstrapped CIs.{p_end}
{synopt:{cmd:r(UncondES#)}}unconditional effect size for # number of arms in {it:interv_var}, obtained based on variances from the unconditional model (model with only the intercept as a fixed effect).{p_end}
{synopt:{cmd:r(Beta)}}estimates and confidence intervals for variables specified in the model.{p_end}
{synopt:{cmd:r(Cov)}}variance decomposition into within cluster variance (Pupils) and Total variance. It also contains intra-cluster correlation (ICC).{p_end}
{synopt:{cmd:r(schCov)}}variance decomposition into between cluster variance-covariance matrix (School by Intervention).{p_end}
{synopt:{cmd:r(UschCov)}}variance decomposition for the Unconditional model into between cluster variance (School).{p_end}
{synopt:{cmd:r(SchEffects)}}estimated deviation of each school from the intercept and intervention slope.{p_end}
{synopt:{cmd:r(CondPv)}}conditional two-sided within and total effect size permutation p-values (available if nperm(#) has been selected).{p_end}
{synopt:{cmd:r(UncondPv)}}unconditional two-sided within and total effect size permutation p-values (available if nperm(#) has been selected).{p_end}