{smcl}
{* 4 January 2018}{…}
{hline}
help for {hi:caterpillar} 
{hline}

{title:Title}

{pstd}{cmd:caterpillar} - Given a set of estimates and standard errors, generate confidence intervals, Bonferroni-corrected confidence intervals, and null distribution.

{title:Syntax}

{pstd}{cmd:caterpillar} {it:est} {it:se} {it:id} {ifin}, [saving(dataset) by(id) {it: options}]

{synoptset 25 tabbed}{...}
{marker opt}{synopthdr:options}
{synoptline}
{synopt :{opt center}} If specified, centers estimates around their precision-weighted mean and creates new variable, {it:contrast}. 
{p_end}

{synopt :{opt graph}} If specified, creates graph of estimates (centered if {it:center} is also used), 95% pointwise confidence intervals, Bonferroni-corrected confidence intervals, and null distribution. Cannot be used with -by-.  
{p_end}

{title:Description}

{pstd}{cmd:caterpillar} takes a set of estimates {it:est} and standard errors {it:se}, along with a unique identifier {it:id} for each estimate. 
The estimates may represent the effects of different programs (as in von Hippel & Bellows, 2017) or the results of different studies (as in a meta-analysis).

{pstd}{cmd:caterpillar} outputs a "caterpillar plot" containing point estimates (sorted in ascending order), along with 95% pointwise confidence intervals, Bonferroni-corrected 95% confidence intervals, and an estimate of the null distribution. 
The null distribution represents what the distribution of estimates would look like under homogeneity -- i.e., if there were no differences between the effects, and the estimates differed only because of random estimation error.

{pstd}{cmd:caterpillar} prints summary statistics on the plot, including Cochran's Q test (with degrees of freedom and p value), a method-of-moments estimate of the heterogeneity standard deviation (tau), and the Higgins-Thompson estimate of the reliability (rho). All calculations described in von Hippel and Bellows (2018). 

{pstd}The program generates five variables, {it:CI_lo}, {it:CI_hi}, {it:CI_lo_bon}, {it:CI_hi_bon}, and {it:null_quantile}, that contain the confidence intervals and quantiles of the null distribution. 

{pstd}{cmd:caterpillar} optionally saves Cochran's Q test, degrees of freedom for Cochran's Q, p value for Cochran's Q, the heterogeneity standard deviation, and the reliability into a separate dataset. 


{title:Remarks} 

{pstd}{cmd:caterpillar} requires that {cmd:_gwtmean} be installed from SSC. 

{pstd}{cmd:caterpillar} requires a unique identifier for estimates and will not complete if a unique identifier is not given. If the -by- option is used, {cmd:caterpillar} requires that the identifier is unique within each group. 

{title:Examples}

{pstd}This example uses the dataset {it:caterpillar_replication.dta} to replicate some results from von Hippel & Bellows (2018). 
To replicate von Hippel & Bellows more completely, run the code in {it:vonHippelBellows2018.do}. 

Use install {it:caterpillar_replication.dta} and {it:vonHippelBellows2018.do} in the working directory, run {}cmd:ssc install caterpillar, all}.

. // Graph CIs, Bonferroni-corrected CIs, and null distribution 
. // for all New York City teacher preparation programs (TPPs) in math
. use caterpillar_replication, clear
. caterpillar est se tpp if state=="NYC" & subject=="Math" ///
. 	& size=="All", graph center

. // Re-graph without the outlier. 
. // The Q test becomes non-significant, and 
. // the heterogeneity and reliability estimates go to 0.
. use caterpillar_replication, clear
. caterpillar est se tpp if state=="NYC" & subject=="Math" ///
. 	& size=="All" & est<.4, graph center

. // Generate the same statistics (except for graph) separately for different states and test subjects (math, reading, etc.), 
. //  as well as different data subsets and modeling decisions described in von Hippel & Bellows (2018).
. use caterpillar_replication, clear
. caterpillar est se tpp, by(state subject size schlfe experienced se_inflation) ///
. 	center saving(tpp_estimates, replace)

{title:Saved Results}

{pstd}Scalars: {p_end}

{pstd}{cmd:r(Q)}{space 6}Cochran's Q statistic {p_end}
{pstd}{cmd:r(df)}{space 5}Degrees of freedom for Cochran's Q statistic {p_end}
{pstd}{cmd:r(p)}{space 6}P-value for Cochran's Q statistic {p_end}
{pstd}{cmd:r(tau)}{space 4}Heterogeneity standard deviation {p_end}
{pstd}{cmd:r(rho)}{space 4}Reliability {p_end}

{pstd}Matrices: {p_end}

{pstd}If -by- is used, instead of scalars, statistics are saved into vectors. Also, a local macro {cmd:r(levels)} holds the -by- levels for reference. 

{pstd}{cmd:r(Q)}{space 6} Vector of Cochran's Q statistics, by group {p_end}
{pstd}{cmd:r(df)}{space 5} Vector of degrees of freedom for Cochran's Q statistic, by group  {p_end}
{pstd}{cmd:r(p)}{space 6} Vector of p-values for Cochran's Q statistic, by group {p_end}
{pstd}{cmd:r(tau)}{space 4} Vector of heterogeneity standard deviation, by group  {p_end}
{pstd}{cmd:r(rho)}{space 4} Vector of reliabilities, by group {p_end}

{title:Authors} 

{pstd}Laura Bellows, Duke University, USA{break}
laura.bellows@duke.edu

{pstd}Paul von Hippel, University of Texas at Austin, USA{break}
paulvonhippel.utaustin@gmail.com


{title:References}

{pstd}von Hippel, Paul T., and Laura Bellows (2018). "How Much Does Teacher Quality Vary Across Teacher Preparation Programs? Reanalyses from 6 States." Economics of Education Review, in press. 
Also available as SSRN Working Paper, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2990498.

{pstd}Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186

{title:Also see} 

{pstd}von Hippel, Paul T., Laura Bellows, Cynthia Osborne, Jane Lincove, and Nicholas Mills (2016). "Teacher Quality Differences Between Teacher Preparation Programs: How Big? How Reliable? Which Programs Are Different?" {it}Economics of Education Review 53, {sf}31-45. doi:10.1016/j.econedurev.2016.05.002