mlmr2 -- a postestimation command that computes r-squared measures for models estimated by mixed

Syntax

mlmr2

Description

mlmr2 produces r-squared measures for models estimated by mixed. Using the Rights and Sterba (2019; 2021; 2023b) framework for decomposing the total model-implied outcome variance from a linear mixed model into its sources, mlmr2 computes measures of the proportion of explained variance attributable to each (or combinations) of those sources (e.g., level-1 portion of predictors via fixed slopes). mlmr2 can be used as a postestimation command with any set of estimation results from mixed except for results from cross-classified models and/or models with more than 5 levels. mlmr2 also assesses whether the model's fixed and random effects may be subject to conflation bias. If mlmr2 detects any predictors at risk of having conflated fixed or random effects, the output will contain warning messages detailing which predictors are at risk. These warning messages are to aid the user in avoiding conflation bias in the r-squared measures (Rights, 2023; Rights & Sterba, 2023a).

Examples

Setup

. webuse nlswork

Random-intercept and random-slope (coefficient) model, correlated random effects

. mixed ln_w grade age c.age#c.age ttl_exp tenure c.tenure#c.tenure || id: tenure, cov(unstruct)

Using mlmr2 to compute r-squared measures for the estimated model

. mlmr2

Note that every predictor except grade contains both level-1 and level-2 variation, and none of those predictors have their contextual effects accounted for (i.e., their corresponding cluster mean variables are not included as predictors), so mlmr2 produces warning messages about these predictors because they are all at risk of having their fixed and random effects biased by conflation.

Setup

. webuse pig

Two-level model

. mixed weight week || id:

Using mlmr2 to compute r-squared measures for the estimated model

. mlmr2

Note that week has no variation across the id clusters, so mlmr2 produces no warning message because there is no risk of week's fixed effect being biased by conflation.

Setup

. webuse productivity

Three-level nested random interactions model with ANOVA DF

. mixed gsp private emp hwy water other unemp || region:water || state:other, dfmethod(anova)

Using mlmr2 to compute r-squared measures for the estimated model

. mlmr2

Note that every predictor in this model contains level-1, level-2, and level-3 variation, and no contextual effects have been accounted for, so mlmr2 produces warning messages about the predictors because they are all at risk of having their fixed and random effects biased by conflation.

Setup

. webuse ovary

Linear mixed model with MA 2 errors

. mixed follicles sin1 cos1 || mare: sin1, residuals(ma 2, t(time))

Using mlmr2 to compute r-squared measures for the estimated model

. mlmr2

Note that there is a warning message for cos1 but not for sin1. This is because the level-2 variance of sin1 is so small that it's basically zero (it's less than 10^-10), meaning that there is basically no risk of its fixed or random effect being biased by conflation. Conversely, the level-2 variance of cos1 is technically nonzero (it's about 0.00003) and its level-2 cluster means have not been included in the model as a predictor, so there is technically a risk of its fixed effect being biased by conflation. However, considering the level-1 variance of cos1 (about 0.51) is many magnitudes larger than its level-2 variance, there is probably very little risk of conflation bias, and this warning message can be ignored.

Setup

. webuse childweight

Linear mixed model with heteroskedastic error variances

. mixed weight age || id:age, residuals(independent, by(girl))

Using mlmr2 to compute r-squared measures for the estimated model

. mlmr2

Stored Results

mlmr2 stores the following in r():

Scalars

r(f1) estimate of outcome variance attributable to the level-1 portion of predictors via fixed slopes

r(f2) estimate of outcome variance attributable to the level-2 portion of predictors via fixed slopes

r(f3) estimate of outcome variance attributable to the level-3 portion of predictors via fixed slopes

r(f4) estimate of outcome variance attributable to the level-4 portion of predictors via fixed slopes

r(f5) estimate of outcome variance attributable to the level-5 portion of predictors via fixed slopes

r(f) estimate of outcome variance attributable to all predictors via fixed slopes

r(v12) estimate of outcome variance attributable to the level-1 portion of predictors via level-2 random slope (co)variation

r(v22) estimate of outcome variance attributable to the level-2 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(v32) estimate of outcome variance attributable to the level-3 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(v42) estimate of outcome variance attributable to the level-4 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(v52) estimate of outcome variance attributable to the level-5 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(v2) estimate of outcome variance attributable to all predictors via level-2 random slope (co)variation

r(v13) estimate of outcome variance attributable to the level-1 portion of predictors via level-3 random slope (co)variation

r(v23) estimate of outcome variance attributable to the level-2 portion of predictors via level-3 random slope (co)variation

r(v33) estimate of outcome variance attributable to the level-3 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(v43) estimate of outcome variance attributable to the level-4 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(v53) estimate of outcome variance attributable to the level-5 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(v3) estimate of outcome variance attributable to all predictors via level-3 random slope (co)variation

r(v14) estimate of outcome variance attributable to the level-1 portion of predictors via level-4 random slope (co)variation

r(v24) estimate of outcome variance attributable to the level-2 portion of predictors via level-4 random slope (co)variation

r(v34) estimate of outcome variance attributable to the level-3 portion of predictors via level-4 random slope (co)variation

r(v44) estimate of outcome variance attributable to the level-4 portion of predictors via level-4 random slope (co)variation (when cwc is not specified)

r(v54) estimate of outcome variance attributable to the level-5 portion of predictors via level-4 random slope (co)variation (when cwc is not specified)

r(v4) estimate of outcome variance attributable to all predictors via level-4 random slope (co)variation

r(v15) estimate of outcome variance attributable to the level-1 portion of predictors via level-5 random slope (co)variation

r(v25) estimate of outcome variance attributable to the level-2 portion of predictors via level-5 random slope (co)variation

r(v35) estimate of outcome variance attributable to the level-3 portion of predictors via level-5 random slope (co)variation

r(v45) estimate of outcome variance attributable to the level-4 portion of predictors via level-5 random slope (co)variation

r(v55) estimate of outcome variance attributable to the level-5 portion of predictors via level-5 random slope (co)variation (when cwc is not specified)

r(v5) estimate of outcome variance attributable to all predictors via level-5 random slope (co)variation

r(m2) estimate of outcome variance attributable to level-2 outcome means via level-2 random intercept variation

r(m3) estimate of outcome variance attributable to level-3 outcome means via level-3 random intercept variation

r(m4) estimate of outcome variance attributable to level-4 outcome means via level-4 random intercept variation

r(m5) estimate of outcome variance attributable to level-5 outcome means via level-5 random intercept variation

r(s2) estimate of outcome variance attributable to level-1 residuals (i.e., amount of unexplained variance)

r(L1_MI_Var) estimate of level-1 model-implied outcome variance

r(L2_MI_Var) estimate of level-2 model-implied outcome variance

r(L3_MI_Var) estimate of level-3 model-implied outcome variance

r(L4_MI_Var) estimate of level-4 model-implied outcome variance

r(L5_MI_Var) estimate of level-5 model-implied outcome variance

r(Total_MI_Var) estimate of total model-implied outcome variance

r(R2_f1_L1) proportion of level-1 outcome variance explained by the level-1 portion of predictors via fixed slopes

r(R2_v12_L1) proportion of level-1 outcome variance explained by the level-1 portion of predictors via level-2 random slope (co)variation

r(R2_v13_L1) proportion of level-1 outcome variance explained by the level-1 portion of predictors via level-3 random slope (co)variation

r(R2_v14_L1) proportion of level-1 outcome variance explained by the level-1 portion of predictors via level-4 random slope (co)variation

r(R2_v15_L1) proportion of level-1 outcome variance explained by the level-1 portion of predictors via level-5 random slope (co)variation

r(Resid_L1) proportion of level-1 outcome variance explained by the level-1 residuals (i.e., proportion of unexplained level-1 variance)

r(R2_f2_L2) proportion of level-2 outcome variance explained by the level-2 portion of predictors via fixed slopes

r(R2_v22_L2) proportion of level-2 outcome variance explained by the level-2 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v23_L2) proportion of level-2 outcome variance explained by the level-2 portion of predictors via level-3 random slope (co)variation

r(R2_v24_L2) proportion of level-2 outcome variance explained by the level-2 portion of predictors via level-4 random slope (co)variation

r(R2_v25_L2) proportion of level-2 outcome variance explained by the level-2 portion of predictors via level-5 random slope (co)variation

r(R2_m2_L2) proportion of level-2 outcome variance explained by level-2 outcome means via level-2 random intercept variation

r(R2_f3_L3) proportion of level-3 outcome variance explained by the level-3 portion of predictors via fixed slopes

r(R2_v32_L3) proportion of level-3 outcome variance explained by the level-3 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v33_L3) proportion of level-3 outcome variance explained by the level-3 portion of predictors via level-3 random slope (co)variation

r(R2_v34_L3) proportion of level-3 outcome variance explained by the level-3 portion of predictors via level-4 random slope (co)variation

r(R2_v35_L3) proportion of level-3 outcome variance explained by the level-3 portion of predictors via level-5 random slope (co)variation

r(R2_m3_L3) proportion of level-3 outcome variance explained by level-3 outcome means via level-3 random intercept variation

r(R2_f4_L4) proportion of level-4 outcome variance explained by the level-4 portion of predictors via fixed slopes

r(R2_v42_L4) proportion of level-4 outcome variance explained by the level-4 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v43_L4) proportion of level-4 outcome variance explained by the level-4 portion of predictors via level-3 random slope (co)variation

r(R2_v44_L4) proportion of level-4 outcome variance explained by the level-4 portion of predictors via level-4 random slope (co)variation

r(R2_v45_L4) proportion of level-4 outcome variance explained by the level-4 portion of predictors via level-5 random slope (co)variation

r(R2_m4_L4) proportion of level-4 outcome variance explained by level-4 outcome means via level-4 random intercept variation

r(R2_f5_L5) proportion of level-5 outcome variance explained by the level-5 portion of predictors via fixed slopes

r(R2_v52_L5) proportion of level-5 outcome variance explained by the level-5 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v53_L5) proportion of level-5 outcome variance explained by the level-5 portion of predictors via level-3 random slope (co)variation

r(R2_v54_L5) proportion of level-5 outcome variance explained by the level-5 portion of predictors via level-4 random slope (co)variation

r(R2_v55_L5) proportion of level-5 outcome variance explained by the level-5 portion of predictors via level-5 random slope (co)variation

r(R2_m5_L5) proportion of level-5 outcome variance explained by level-5 outcome means via level-5 random intercept variation

r(R2_f1_Total) proportion of total outcome variance explained by the level-1 portion of predictors via fixed slopes

r(R2_f2_Total) proportion of total outcome variance explained by the level-2 portion of predictors via fixed slopes

r(R2_f3_Total) proportion of total outcome variance explained by the level-3 portion of predictors via fixed slopes

r(R2_f4_Total) proportion of total outcome variance explained by the level-4 portion of predictors via fixed slopes

r(R2_f5_Total) proportion of total outcome variance explained by the level-5 portion of predictors via fixed slopes

r(R2_f_Total) proportion of total outcome variance explained by all predictors via fixed slopes

r(R2_v12_Total) proportion of total outcome variance explained by the level-1 portion of predictors via level-2 random slope (co)variation

r(R2_v22_Total) proportion of total outcome variance explained by the level-2 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v32_Total) proportion of total outcome variance explained by the level-3 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v42_Total) proportion of total outcome variance explained by the level-4 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v52_Total) proportion of total outcome variance explained by the level-5 portion of predictors via level-2 random slope (co)variation (when cwc is not specified)

r(R2_v2_Total) proportion of total outcome variance explained by all predictors via level-2 random slope (co)variation

r(R2_v13_Total) proportion of total outcome variance explained by the level-1 portion of predictors via level-3 random slope (co)variation

r(R2_v23_Total) proportion of total outcome variance explained by the level-2 portion of predictors via level-3 random slope (co)variation

r(R2_v33_Total) proportion of total outcome variance explained by the level-3 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(R2_v43_Total) proportion of total outcome variance explained by the level-4 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(R2_v53_Total) proportion of total outcome variance explained by the level-5 portion of predictors via level-3 random slope (co)variation (when cwc is not specified)

r(R2_v3_Total) proportion of total outcome variance explained by all predictors via level-3 random slope (co)variation

r(R2_v14_Total) proportion of total outcome variance explained by the level-1 portion of predictors via level-4 random slope (co)variation

r(R2_v24_Total) proportion of total outcome variance explained by the level-2 portion of predictors via level-4 random slope (co)variation

r(R2_v34_Total) proportion of total outcome variance explained by the level-3 portion of predictors via level-4 random slope (co)variation

r(R2_v44_Total) proportion of total outcome variance explained by the level-4 portion of predictors via level-4 random slope (co)variation (when cwc is not specified)

r(R2_v54_Total) proportion of total outcome variance explained by the level-5 portion of predictors via level-4 random slope (co)variation (when cwc is not specified)

r(R2_v4_Total) proportion of total outcome variance explained by all predictors via level-4 random slope (co)variation

r(R2_v15_Total) proportion of total outcome variance explained by the level-1 portion of predictors via level-5 random slope (co)variation

r(R2_v25_Total) proportion of total outcome variance explained by the level-2 portion of predictors via level-5 random slope (co)variation

r(R2_v35_Total) proportion of total outcome variance explained by the level-3 portion of predictors via level-5 random slope (co)variation

r(R2_v45_Total) proportion of total outcome variance explained by the level-4 portion of predictors via level-5 random slope (co)variation

r(R2_v55_Total) proportion of total outcome variance explained by the level-5 portion of predictors via level-5 random slope (co)variation (when cwc is not specified)

r(R2_v5_Total) proportion of total outcome variance explained by all predictors via level-5 random slope (co)variation

r(R2_v_Total) proportion of total outcome variance explained by all predictors via random slope (co)variation

r(R2_m2_Total) proportion of total outcome variance explained by level-2 outcome means via level-2 random intercept variation

r(R2_m3_Total) proportion of total outcome variance explained by level-3 outcome means via level-3 random intercept variation

r(R2_m4_Total) proportion of total outcome variance explained by level-4 outcome means via level-4 random intercept variation

r(R2_m5_Total) proportion of total outcome variance explained by level-5 outcome means via level-5 random intercept variation

r(R2_m_Total) proportion of total outcome variance explained by all outcome means via random intercept variation

r(R2_fv_Total) proportion of total outcome variance explained by all predictors via fixed slopes and all random slope (co)variation

r(R2_fvm_Total) proportion of total outcome variance explained by the whole model

r(Resid_Total) proportion of total outcome variance explained by level-1 residuals (i.e., proportion of unexplained variance)

r(R2_L1_Total) proportion of total outcome variance explained by level-1 sources

r(R2_L2_Total) proportion of total outcome variance explained by level-2 sources

r(R2_L3_Total) proportion of total outcome variance explained by level-3 sources

r(R2_L4_Total) proportion of total outcome variance explained by level-4 sources

r(R2_L5_Total) proportion of total outcome variance explained by level-5 sources

Matrices

r(R2) matrix containing each of the r-squared measures

Note that results regarding levels above the highest level in the estimated model will not be stored (e.g., r(R2_f5_Total) will not be stored if the estimated model has fewer than 5 levels).

Author

Anthony J. Gambino, University of Connecticut, anthony.gambino@uconn.edu

References

Rights, J. D. 2023. Aberrant distortion of variance components in multilevel models under conflation of level-specific effects. Psychological Methods, 28: 1154–1177.

Rights, J. D., and S. K. Sterba. 2019. Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological Methods, 24: 309–338.

Rights, J. D., and S. K. Sterba. 2021. Effect size measures for longitudinal growth analyses: Extending a framework of multilevel model r-squareds to accommodate heteroscedasticity, autocorrelation, nonlinearity, and alternative centering strategies. New Directions for Child and Adolescent Development, 2021: 65–110.

Rights, J. D., and S. K. Sterba. 2023a. On the common but problematic specification of conflated random slopes in multilevel models. Multivariate Behavioral Research: 1–28.

Rights, J. D., and S. K. Sterba. 2023b. R-squared measures for multilevel models with three or more levels. Multivariate Behavioral Research, 58: 340–367.