------------------------------------------------------------------------------- help forparmhetandparmiv(Roger Newson) -------------------------------------------------------------------------------

Heterogeneity tests and inverse-variance weights inparmestresultssets

parmhetest_varnamese_varname[dof_varname] [if] [in] [,basic_optsresultsset_optshettest_opts]

parmivest_varnamese_varname[dof_varname] [if] [in] [,basic_optshettest_opts]

Description

parmhetandparmivare designed for use withparmestresultssets, which have one observation per estimated parameter and data on parameter estimates.parmhetinputs variables containing parameter estimates, standard errors and (optionally) degrees of freedom, and outputs an output dataset (or resultsset) with one observation, or one observation per by-group, and data on heterogeneity test statistics on the input parameters in the dataset or by-group. The output dataset can be listed to the Stata log, saved to a disk file, or written to memory, overwriting the existing dataset. Optionally,parmhetmay also add variables to the existing dataset, containing inverse-variance weights and/or semi-weights and/or semi-weight-based standard errors. These additional variables may then be input to themetaparmmodule of theparmestpackage, to produce summary meta-analysed parameter estimates, using the fixed-effect method or the DerSimonian-Laird randomly-variable-effect method.parmivis a routine intended for use by programmers, and inputs variables containing estimates, standard errors and (optionally) degrees of freedom, and outputs additional variables in the existing dataset, containing inverse-variance weights and/or semi-weights and/or semi-weight-based standard errors and/or heterogeneity test statistics for the dataset or by-group.

Options

parmhetandparmivhave a large number of options, which fall into the following 3 groups:Option group Description --------------------------------------------------------------------------

basic_optsBasic options forparmhetandparmivresultsset_optsResultsset options forparmhethettest_optsHeterogeneity-test options forparmhetandparmiv--------------------------------------------------------------------------

Basic options forparmhetandparmiv

optionsDescription -------------------------------------------------------------------------by(varlist)Variables specifying by-groupseformEstimates and confidence limits exponentiatedfloatNumeric output variables of typefloator lessdfcombine(combination_rule)Rule for combining degrees of freedomivweight(newvarname)Name of generated inverse-variance weight variablesweight(newvarname)Name of generated semi-weight variablesstderr(newvarname)Name of generated semi-weight-based standard error variable -------------------------------------------------------------------------where

combination_ruleis

welch|constantSee

parmhet_basic_optsfor details of these options.

Resultsset options forparmhet

optionsDescription -------------------------------------------------------------------------list(list_spec)List output dataset to Stata log and/or Results windowsaving(filename[,replace])Save output dataset to a disk filenorestoreWrite output dataset to memoryfastWrite output dataset to memory without precautionsflist(global_macro_name)Append output filename to a global macroidnum(#)Value of numeric dataset ID variablenidnum(newvarname)Name of numeric dataset ID variableidstr(string)Value of string dataset ID variablenidstr(newvarname)Name of string dataset ID variablesumvar(varlist)Variables to be summed in the output datasetformat(formatting_list)Display formats for variables in the output datasetkeep(varlist)Variables to be kept in the output dataset -------------------------------------------------------------------------where

formatting_listis a list of form

varlist_1format_1...varlist_nformat_nand

list_specis a specification of the form[

varlist] [if] [in] [ , [list_options] ]and

list_optionsis a list of options used by thelistcommand.See

parmhet_resultsset_optsfor details of these options.

Heterogeneity-test options forparmhetandparmiv

optionsDescription -------------------------------------------------------------------------chi2het(newvarname)Heterogeneity chi-squared statistic variabledfhet(newvarname)Heterogeneity degrees of freedom variablei2het(newvarname)HeterogeneityI-squared statistic variabletau2het(newvarname)Heterogeneity tau-squared statistic variablefhet(newvarname)HeterogeneityF-statistic variableresdfhet(newvarname)Heterogeneity residual degrees of freedom variablephet(newvarname)HeterogeneityP-value variable -------------------------------------------------------------------------See

parmhet_hettest_optsfor details of these options.

Output dataset created byparmhetThis output dataset (or resultsset) is described in detail in

parmhet_resultsset.

Methods and formulas

parmhetandparmivuse formulas introduced by Cochran (1954) and Welch (1951), and later extended by DerSimonian and Laird (1986) and by Higgins and Thompson (2002). In general, it is assumed that, in the input dataset (or in a by-group of the input dataset), there areNobservations, corresponding to parameterstheta_1, ..., theta_N, with corresponding standard errorsse_1, ..., se_N, and corresponding degrees of freedondf_1, ..., df_Nif a degrees of freedom variable is specified. The parameterstheta_jand the standard errorsse_jare given by the input variables specified byest_varnameandse_varname, unless theeformoption is specified, in which case thetheta_jare the logs of the values ofest_varname, and these_jare the standard errors of these logs. We assume that thetheta_jwere sampled independently, or at least in an uncorrelated way, fromNrespective sub-populations.The inverse-variance weights, output in the

ivweight()variable, are defined by the formula

ivweight_j = (se_j)^(-2)for each

jfrom 1 toN. The inverse-variance-weighted mean parameter value is

Theta = Sum ( iwweight_j * theta_j ) / Sum (ivweight_j)where

Sum()is the sum forjfrom 1 toN.Thetacan be used as an estimate for a common sub-population mean parameter value, assuming that there is no heterogeneity between sub-populations. This estimate, with a standard error, confidence limits and aP-value, can be calculated by inputting the estimates, standard errors and (optionally) degrees of freedom to themetaparmoption of theparmestpackage, with aweights equal to theivweight()variable.The heterogeneity chi-squared statistic is defined by

Q = Sum ( ivweight_j * (theta_j - Theta)^2 )Under the null hypothesis of no heterogeneity,

Qis sampled from a distribution approximating to a chi-squared distribution withN-1degrees of freedom. The I-squared statistic of Higgins and Thompson (2002) is equal to

I^2 = 100*max( 0 , (Q-N+1)/(N-1) )and is equal to zero if the heterogeneity chi-squared statistic is no more than its mean of

N-1under the null hypothesis of no heterogeneity, and otherwise measures the percent relative excess of heterogeneity, compared to this null mean. And the tau-squared statistic of Higgins and Thompson (2002) is given by

tau^2 = max( 0 , (Q-N+1)/( Sum(ivweight_j) -(Sum((ivweight_j)^2))/(Sum(ivweight_j)) ) )and estimates the sampling variance of the true sub-population values of the

theta_jin the meta-population from which their respective sub-populations were sampled. It is expressed in squaredtheta_junits.The sampling variance of each

theta_j, in the two-stage sampling process where thejth sub-population is sampled from the meta-population and thetheta_jis sampled from thejth subpopulation, can be estimated by(se_j)^2 + tau^2. It follows that the two-stage sampling standard error oftheta_j, output in thesstderr()variable, is equal to

sstderr_j = sqrt((se_j)^2 + tau^2)and the two-stage inverse-variance weights (or semi-weights), output in the

sweight()variable, is equal to

sweight_j = ((se_j)^2 + tau^2)^{-1}These two-stage sampling standard errors and two-stage inverse-variance weights can then be input to the

metaparmmodule of theparmestpackage, to produce an estimate, a standard error, confidence limits, and aP-value for the common meta-population mean of thetheta_j. Thesstderr()variable generated byparmhetorparmivis input tometaparmas thestderr()option, and thesweight()variable generated byparmhetorparmivis input tometaparmas the aweights. This computation implements the randomly-variable-effect method of DerSimonian and Laird (1986).An alternative to fixed-effect meta-analysis and randomly-variable-effect meta-analysis is nonrandomly-variable-effect meta-analysis. This can be implemented using

metaparmby setting the aweights equal to the respective sample numbers used in estimating thetheta_j. The nonrandomly-variable-effect method does not assume that the sub-population means of thetheta_jare equal (as the fixed-effect method does), and does not assume that we can estimate the meta-population variance (as the randomly-variable-effect method does). However, it does assume that the sample-size-weighted average of the sub-population effects is a useful parameter to know, even thought the sub-population effects may vary between sub-populations. Whichever of the three methods we use, our colleagues will usually expect a heterogeneity test to be done.The

F-test statistic, computed if a degrees of freedom variable is supplied toparmhetorparmiv, is defined by a method specified by thedfcombine()option. If the user specifiesdfcombine(welch)(the default), then theF-test statistic, and its denominator degrees of freedom, are defined using the formulas of Welch (1951), popularized by Cochran (1954). If the user specifiesdfcombine(constant), thenparmhetandparmivcheck that the input degrees of freedom variable is constant, or constant within by-groups if aby()option is specified, and then defines theF-statistic asQ/(N-1), withN-1numerator degrees of freedom, and the denominator degrees of freedom given by the constant value of the input degrees of freedom variable. The optiondfcombine(constant)is useful if the input parameters are uncorrelated parameters of a single regression model, and their standard errors are computed using an equal-variance formula, and the degrees of freedom is pooled for all parameters in the model.

Remarks

parmhetandparmivare designed for use with theparmestpackage, and particularly with themetaparmmodule of that package. More information about the production and use ofparmestresultssets can be found in Newson (2010), Newson (2008), Newson (2006), Newson (2004), Newson (2003) and Newson (2002).parmhetcan be used withparmest,metaparm,eclplotandlisttabto form a comprehensive system for meta-analyses and/or interaction analyses, enabling the user to produce plots and/or tables of results.Other programs are available in the Stata community for carrying out meta-analyses, notably

metan, written by Michael J. Bradburn, Jonathan J. Deeks, Douglas G. Altman, Ross J. Harris, Roger M. Harbord, and Jonathan A. C. Sterne. (Seemetaand[R] metafor details.) The combination ofparmhet,parmest,metaparm,eclplotandlisttabis arguably even more flexible thanmetan, at the price of having to know some Stata programming.Other programs for heterogeneity tests on

parmestresultssets are theestparmandestparmtestmodules of theestparmpackage. These allow the user to carry out Wald heterogeneity tests, comparing each other parameter estimate with the first parameter estimate. The Cochran and Welch heterogeneity tests, by contrast, weight the parameters inversely by variance, and do not otherwise treat the first parameter differently from the others. It is therefore expected thatparmhetwill supersedeestparmfor most purposes.The

parmest,eclplot,listtab,estparmandmetanpackages can be downloaded from SSC.

Examples: Sequence 1The first example sequence uses the dataset

parmhet_example1, which is distributed with theparmhetpackage as an ancillary file. This dataset contains data from a meta-analysis of Glasziouet al.(1993), distributed in Bland (2000), and involving 5 clinical trials, comparing death rates in subjects treated with vitamin A and untreated control subjects. The dataset contains 1 observation per treatment group per trial, and data on total numbers of subjects, and numbers of deaths, in that treatment group in that trial, and also data on the quantity (in international units or IU) and frequency (per unit time) of the vitamin A dosage applied to the treated group in that trial. The set-up uses theparmbymodule of theparmestpackage, downloadable from SSC, to replace this dataset with a new dataset (or resultsset) in memory, with 1 observation per trial, and data on odds ratios of death with respect to vitamin A treatment, with their confidence limits andP-values, and also the total number of subjects in the trial (in variableN).Set-up

. use "http://www.imperial.ac.uk/nhli/r.newson/stata10/parmhet_example1 > .dta", clear . describe . parmby "blogit deaths number vitamina, or", by(studyseq doseiu dosefreq) eform norestore escal(N) rename(es_1 N) . keep if parm=="vitamina" . describe . list studyseq doseiu dosefreq parm N estimate stderr min* max* p

Simple examples

. parmhet estimate stderr, eform list(,)

. parmhet estimate stderr, eform by(doseiu) list(, subvarname abbr(32))

. parmhet estimate stderr, eform by(doseiu) saving(myhet1.dta, replace)

Note that, if we do separate heterogeneity tests for different vitamin A doses using the option

by(doseiu), then dosage groups with only 1 study have zero heterogeneity chi-squared statistics, zero heterogeneity degrees of freedom, zeroI-squared and tau-squared statistics, and missing heterogeneityP-values. Note, also, that the suboptionssubvarname abbr(32)of thelist()option cause the heterogeneity test statistics to be listed with informative headings. These are set by thevarnamecharacteristic, set byparmhetand used by thelistcommand.The following example uses the

norestoreoption. This replaces the resultssetset in memory, containing 1 observation per study, with a new resultsset, containing 1 observation per vitamin A dosage group, and data on heterogeneity test results:. preserve . parmhet estimate stderr, eform by(doseiu) norestore . describe . list . restore

The following example uses

parmhettogether with themetaparmmodule of theparmestpackage.parmhettests for heterogeneity, and also adds to the existing dataset a new variableivwt, containing inverse-variance weights for each study. These are then used as aweights bymetaparmto perform a fixed-effect meta-analysis, estimating a common odds ratio for all studies, and assuming that such a common odds ratio exists.. parmhet estimate stderr, eform list(,) ivweight(ivwt) . metaparm [aweight=ivwt], eform sumvar(N) list(,)

The following example uses

parmhetwithmetaparmto perform a DerSimonian-Laird randomly-variable-effects meta-analysis (DerSimonian and Laird, 1986). This time,parmhetadds variablessemiwtandswse, containing semi-weights and semi-weight-based standard errors, respectively. These are passed tometaparmas aweights and standard errors, respectively.. parmhet estimate stderr, eform list(,) sweight(semiwt) sstderr(swse) . metaparm [aweight=semiwt], eform sumvar(N) stderr(swse) list(,)

The following example performs a nonrandomly-variable-effects analysis, weighted by the study size variable

N.. parmhet estimate stderr, eform sumvar(N) list(,) . metaparm [aweight=N], eform sumvar(N) list(,)

The following example carries out a nonrandomly-variable-effects meta-analysis, saving the resultssets from

parmhetandmetaparmin disk-file datasetsmyhetandmymeta, respectively. These resultssets are appended to the existing resultsset to produce an extended resultsset, which is listed to show the study odds ratios, the weighted geometric mean (GM) odds ratio, andP-values for the odds ratios and for the heterogeneity test.. preserve . gene idstr="Single study" . order idstr . parmhet estimate stderr, eform sumvar(N) idstr(Heterogeneity) phet(p) keep(idstr N p) saving(myhet.dta, replace) . metaparm [aweight=N], eform sumvar(N) idstr("Weighted GM") saving(mymeta.dta, replace) . append using mymeta . append using myhet . describe . list idstr studyseq doseiu dosefreq N estimate min* max* p, sepby(idstr) . restore

Examples: Sequence 2The following examples also use the dataset

parmhet_example1. However, this time, theparmbymodule ofparmestis used to produce a dataset of log odds ratios. These are then tested for heterogeneity, usingparmhetwithout theeformoption.Set-up

. use "http://www.imperial.ac.uk/nhli/r.newson/stata10/parmhet_example1 > .dta", clear . describe . parmby "blogit deaths number vitamina", by(studyseq doseiu dosefreq) norestore escal(N) rename(es_1 N) . keep if parm=="vitamina" . describe . list studyseq doseiu dosefreq parm N estimate stderr min* max* p

Simple examples

. parmhet estimate stderr, list(,)

. parmhet estimate stderr, by(doseiu) list(,)

Examples: Sequence 3The following example uses the

autodata, distributed with official Stata, and adds a new variablecontinent, containing the continent of origin of a car model. Theparmbymodule ofparmestis then used to create a resultsset with 1 parameter per continent and data on mean weights of cars (in US pounds), with numbers of car models, estimates, standard errors, degrees of freedom, confidence limits, andP-values. This resultsset is then input toparmhetto perform a Welch unequal-variance heterogeneity test (Welch, 1951), which shows that mean car weights vary between continents.Set-up

. sysuse auto, clear . gene firm=word(make,1) . lab var firm "Firm" . lab def continent 1 "America" 2 "Asia" 3 "Europe" . gene continent=1 if !foreign . replace continent=2 if inlist(firm,"Datsun","Honda","Mazda","Subaru","Toyota") . replace continent=3 if inlist(firm,"Audi","BMW","VW","Fiat","Peugeot","Renault","Volvo") . lab val continent continent . lab var continent "Continent of origin" . describe . tab firm continent, missing . parmby "mean weight", by(continent) norestore . describe . list, abbr(32)

Examples

. parmhet estimate stderr dof, list(,)

. parmhet estimate stderr dof, list(, subvarname abbr(32))

AuthorRoger Newson, National Heart and Lung Institute, Imperial College London, UK. Email: r.newson@imperial.ac.uk

ReferencesBland, M. 2000.

An introduction to medical statistics. 3rd ed.Oxford: Oxford University Press.Cochran, W. G. 1954. The combination of estimates from different experiments.

Biometrics10(1): 101-129.DerSimonian, R. and Laird, N. 1986. Meta-analysis in clinical trials.

Controlled Clinical Trials7(3): 177-188.Glasziou, P.P. and Mackerras, D. E. M. 1993. Vitamin A supplementation in infectious disease: a meta–analysis.

British Medical Journal306(6874): 366-370.Higgins, J. P. T. and Thompson, S. G. 2002. Quantifying heterogeneity in a meta-analysis.

Statistics in Medicine21(11): 1539-1558.Newson, R. B. 2010. Post-

parmestperipherals:fvregen,invcise, andqqvalue. Presented at the 16th United Kingdom Stata Users' Group Meeting, London, 9-10 September, 2010. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.Newson, R. B. 2008.

parmestand extensions. Presented at the 14th United Kingdom Stata Users' Group Meeting, London, 8-9 September, 2008. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.Newson, R. 2006. Resultssets, resultsspreadsheets, and resultsplots in Stata. Presented at the 4th German Stata Users' Group Meeting, Mannheim, 31 March, 2006. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.

Newson, R. 2004. From datasets to resultssets in Stata. Presented at the 10th United Kingdom Stata Users' Group Meeting, London, 29-30 June, 2004. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.

Newson, R. 2003. Confidence intervals and

p-values for delivery to the end user.The Stata Journal3(3): 245-269. Pre-publication draft downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.Newson, R. 2002. Creating plots and tables of estimation results using

parmestand friends. Presented at the 8th United Kingdom Stata Users' Group Meeting, 20-21 May, 2002. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.Welch, B. L. 1951. On the comparison of several mean values: an alternative approach.

Biometrika36(3/4): 330-336.

Also seeManual:

[R] meta,[R] testOn-line: help forparmhet_basic_opts,parmhet_resultsset_opts,parmhet_hettest_opts,parmhet_resultssethelp fortest,listhelp forparmest,parmby,parmcip,metaparm,eclplot,listtab,estparm,metanif installed