help for parmhet and parmiv (Roger Newson)

Heterogeneity tests and inverse-variance weights in parmest resultssets

parmhet est_varname se_varname [ dof_varname ]
[if] [in] [ ,
basic_opts
resultsset_opts
hettest_opts
]

parmiv est_varname se_varname [ dof_varname ]
[if] [in] [ , basic_opts hettest_opts ]

Description

parmhet and parmiv are designed for use with parmest resultssets,
which have one observation per estimated parameter and data on parameter estimates.
parmhet inputs variables containing parameter estimates, standard errors and (optionally) degrees of freedom,
and outputs an output dataset (or resultsset) with one observation,
or one observation per by-group,
and data on heterogeneity test statistics on the input parameters
in the dataset or by-group.
The output dataset can be listed to the Stata log, saved to a disk file,
or written to memory, overwriting the existing dataset.
Optionally, parmhet may also add variables to the existing dataset,
containing inverse-variance weights and/or semi-weights and/or semi-weight-based standard errors.
These additional variables may then be input to the metaparm module of the parmest package,
to produce summary meta-analysed parameter estimates,
using the fixed-effect method or the DerSimonian-Laird randomly-variable-effect method.
parmiv is a routine intended for use by programmers,
and inputs variables containing estimates, standard errors and (optionally) degrees of freedom,
and outputs additional variables in the existing dataset,
containing inverse-variance weights and/or semi-weights and/or semi-weight-based standard errors
and/or heterogeneity test statistics for the dataset or by-group.

Options

parmhet and parmiv have a large number of options, which fall into the following 3 groups:

Option group Description

basic_opts Basic options for parmhet and parmiv

resultsset_opts Resultsset options for parmhet

hettest_opts Heterogeneity-test options for parmhet and parmiv

Basic options for parmhet and parmiv

by(varlist) Variables specifying by-groups

eform Estimates and confidence limits exponentiated

float Numeric output variables of type float or less

dfcombine(combination_rule) Rule for combining degrees of freedom

ivweight(newvarname) Name of generated inverse-variance weight variable

sweight(newvarname) Name of generated semi-weight variable

sstderr(newvarname) Name of generated semi-weight-based standard error variable

where combination_rule is

welch | constant

See parmhet_basic_opts for details of these options.

Resultsset options for parmhet

list(list_spec) List output dataset to Stata log and/or Results window

saving(filename[,replace]) Save output dataset to a disk file

norestore Write output dataset to memory

fast Write output dataset to memory without precautions

flist(global_macro_name) Append output filename to a global macro

idnum(#) Value of numeric dataset ID variable

nidnum(newvarname) Name of numeric dataset ID variable

idstr(string) Value of string dataset ID variable

nidstr(newvarname) Name of string dataset ID variable

sumvar(varlist) Variables to be summed in the output dataset

format(formatting_list) Display formats for variables in the output dataset

keep(varlist) Variables to be kept in the output dataset

where formatting_list is a list of form

varlist_1 format_1 ... varlist_n format_n

and list_spec is a specification of the form

[varlist] [if] [in] [ , [list_options ] ]

and list_options is a list of options used by the list command.

See parmhet_resultsset_opts for details of these options.

Heterogeneity-test options for parmhet and parmiv

chi2het(newvarname) Heterogeneity chi-squared statistic variable

dfhet(newvarname) Heterogeneity degrees of freedom variable

i2het(newvarname) Heterogeneity I-squared statistic variable

tau2het(newvarname) Heterogeneity tau-squared statistic variable

fhet(newvarname) Heterogeneity F-statistic variable

resdfhet(newvarname) Heterogeneity residual degrees of freedom variable

phet(newvarname) Heterogeneity P-value variable

See parmhet_hettest_opts for details of these options.

Output dataset created by parmhet

This output dataset (or resultsset) is described in detail in parmhet_resultsset.

Methods and formulas

parmhet and parmiv use formulas introduced by Cochran (1954) and Welch (1951),
and later extended by DerSimonian and Laird (1986)
and by Higgins and Thompson (2002).
In general, it is assumed that, in the input dataset (or in a by-group of the input dataset),
there are N observations, corresponding to parameters theta_1, ..., theta_N,
with corresponding standard errors se_1, ..., se_N,
and corresponding degrees of freedon df_1, ..., df_N
if a degrees of freedom variable is specified.
The parameters theta_j and the standard errors se_j
are given by the input variables specified by est_varname and se_varname,
unless the eform option is specified,
in which case the theta_j are the logs of the values of est_varname,
and the se_j are the standard errors of these logs.
We assume that the theta_j were sampled independently,
or at least in an uncorrelated way,
from N respective sub-populations.

The inverse-variance weights, output in the ivweight() variable,
are defined by the formula

ivweight_j = (se_j)^(-2)

for each j from 1 to N. The inverse-variance-weighted mean parameter value is

Theta = Sum ( iwweight_j * theta_j ) / Sum (ivweight_j)

where Sum() is the sum for j from 1 to N.
Theta can be used as an estimate for a common sub-population mean parameter value,
assuming that there is no heterogeneity between sub-populations.
This estimate, with a standard error, confidence limits and a P-value,
can be calculated by inputting the estimates, standard errors and (optionally) degrees of freedom
to the metaparm option of the parmest package,
with aweights equal to the ivweight() variable.

The heterogeneity chi-squared statistic is defined by

Q = Sum ( ivweight_j * (theta_j - Theta)^2 )

Under the null hypothesis of no heterogeneity,
Q is sampled from a distribution approximating to a chi-squared distribution
with N-1 degrees of freedom.
The I-squared statistic of Higgins and Thompson (2002) is equal to

I^2 = 100*max( 0 , (Q-N+1)/Q )

and is equal to zero if the heterogeneity chi-squared statistic
is no more than its mean of N-1 under the null hypothesis of no heterogeneity,
and otherwise measures the percent relative excess of heterogeneity,
compared to this null mean.
And the tau-squared statistic of Higgins and Thompson (2002)
is given by

tau^2 = max( 0 , (Q-N+1)/( Sum(ivweight_j) - (Sum((ivweight_j)^2))/(Sum(ivweight_j)) ) )

and estimates the sampling variance of the true sub-population values of the theta_j
in the meta-population from which their respective sub-populations were sampled.
It is expressed in squared theta_j units.

The sampling variance of each theta_j,
in the two-stage sampling process where the jth sub-population is sampled from the meta-population
and the theta_j is sampled from the jth subpopulation,
can be estimated by (se_j)^2 + tau^2.
It follows that the two-stage sampling standard error of theta_j,
output in the sstderr() variable,
is equal to

sstderr_j = sqrt((se_j)^2 + tau^2)

and the two-stage inverse-variance weights (or semi-weights),
output in the sweight() variable,
is equal to

sweight_j = ((se_j)^2 + tau^2)^{-1}

These two-stage sampling standard errors and two-stage inverse-variance weights can then be input
to the metaparm module of the parmest package,
to produce an estimate, a standard error, confidence limits, and a P-value
for the common meta-population mean of the theta_j.
The sstderr() variable generated by parmhet or parmiv
is input to metaparm as the stderr() option,
and the sweight() variable generated by parmhet or parmiv
is input to metaparm as the aweights.
This computation implements the randomly-variable-effect method
of DerSimonian and Laird (1986).

An alternative to fixed-effect meta-analysis and randomly-variable-effect meta-analysis
is nonrandomly-variable-effect meta-analysis.
This can be implemented using metaparm
by setting the aweights equal to the respective sample numbers
used in estimating the theta_j.
The nonrandomly-variable-effect method
does not assume that the sub-population means of the theta_j are equal
(as the fixed-effect method does),
and does not assume that we can estimate the meta-population variance
(as the randomly-variable-effect method does).
However, it does assume that the sample-size-weighted average of the sub-population effects
is a useful parameter to know,
even thought the sub-population effects may vary between sub-populations.
Whichever of the three methods we use,
our colleagues will usually expect a heterogeneity test to be done.

The F-test statistic,
computed if a degrees of freedom variable is supplied to parmhet or parmiv,
is defined by a method specified by the dfcombine() option.
If the user specifies dfcombine(welch) (the default),
then the F-test statistic, and its denominator degrees of freedom,
are defined using the formulas of Welch (1951),
popularized by Cochran (1954).
If the user specifies dfcombine(constant),
then parmhet and parmiv check that the input degrees of freedom variable is constant,
or constant within by-groups if a by() option is specified,
and then defines the F-statistic as Q/(N-1),
with N-1 numerator degrees of freedom,
and the denominator degrees of freedom given by the constant value
of the input degrees of freedom variable.
The option dfcombine(constant) is useful
if the input parameters are uncorrelated parameters of a single regression model,
and their standard errors are computed using an equal-variance formula,
and the degrees of freedom is pooled for all parameters in the model.

Remarks

parmhet and parmiv are designed for use with the parmest package,
and particularly with the metaparm module of that package.
More information about the production and use of parmest resultssets
can be found in Newson (2010), Newson (2008), Newson (2006), Newson (2004), Newson (2003) and Newson (2002).
parmhet can be used with parmest, metaparm, eclplot and listtab
to form a comprehensive system for meta-analyses and/or interaction analyses,
enabling the user to produce plots and/or tables of results.

Other programs are available in the Stata community for carrying out meta-analyses,
notably metan,
written by Michael J. Bradburn, Jonathan J. Deeks, Douglas G. Altman,
Ross J. Harris, Roger M. Harbord, and Jonathan A. C. Sterne.
(See meta and [R] meta for details.)
The combination of parmhet, parmest, metaparm, eclplot and listtab
is arguably even more flexible than metan,
at the price of having to know some Stata programming.

Other programs for heterogeneity tests on parmest resultssets
are the estparm and estparmtest modules of the estparm package.
These allow the user to carry out Wald heterogeneity tests,
comparing each other parameter estimate with the first parameter estimate.
The Cochran and Welch heterogeneity tests, by contrast,
weight the parameters inversely by variance,
and do not otherwise treat the first parameter differently from the others.
It is therefore expected that parmhet will supersede estparm for most purposes.

The parmest, eclplot, listtab, estparm and metan packages
can be downloaded from SSC.

Examples: Sequence 1

The first example sequence uses the dataset parmhet_example1,
which is distributed with the parmhet package as an ancillary file.
This dataset contains data from a meta-analysis of Glasziou et al. (1993),
distributed in Bland (2000),
and involving 5 clinical trials,
comparing death rates in subjects treated with vitamin A and untreated control subjects.
The dataset contains 1 observation per treatment group per trial,
and data on total numbers of subjects, and numbers of deaths,
in that treatment group in that trial,
and also data on the quantity (in international units or IU)
and frequency (per unit time)
of the vitamin A dosage applied to the treated group in that trial.
The set-up uses the parmby module of the parmest package,
downloadable from SSC,
to replace this dataset with a new dataset (or resultsset) in memory,
with 1 observation per trial,
and data on odds ratios of death with respect to vitamin A treatment,
with their confidence limits and P-values,
and also the total number of subjects in the trial (in variable N).

Set-up

. use "http://www.imperial.ac.uk/nhli/r.newson/stata10/parmhet_example1.dta", clear

. describe

. parmby "blogit deaths number vitamina, or", by(studyseq doseiu dosefreq) eform norestore escal(N) rename(es_1 N)

. keep if parm=="vitamina"

. describe

. list studyseq doseiu dosefreq parm N estimate stderr min* max* p

Simple examples

. parmhet estimate stderr, eform list(,)

. parmhet estimate stderr, eform by(doseiu) list(, subvarname abbr(32))

. parmhet estimate stderr, eform by(doseiu) saving(myhet1.dta, replace)

Note that,
if we do separate heterogeneity tests for different vitamin A doses using the option by(doseiu),
then dosage groups with only 1 study have zero heterogeneity chi-squared statistics,
zero heterogeneity degrees of freedom,
zero I-squared and tau-squared statistics,
and missing heterogeneity P-values.
Note, also, that the suboptions subvarname abbr(32) of the list() option
cause the heterogeneity test statistics to be listed with informative headings.
These are set by the varname characteristic,
set by parmhet and used by the list command.

The following example uses the norestore option.
This replaces the resultssetset in memory, containing 1 observation per study,
with a new resultsset, containing 1 observation per vitamin A dosage group,
and data on heterogeneity test results:

. preserve

. parmhet estimate stderr, eform by(doseiu) norestore

. describe

. list

. restore

The following example uses parmhet
together with the metaparm module of the parmest package.
parmhet tests for heterogeneity,
and also adds to the existing dataset a new variable ivwt,
containing inverse-variance weights for each study.
These are then used as aweights by metaparm
to perform a fixed-effect meta-analysis,
estimating a common odds ratio for all studies,
and assuming that such a common odds ratio exists.

. parmhet estimate stderr, eform list(,) ivweight(ivwt)

. metaparm [aweight=ivwt], eform sumvar(N) list(,)

The following example uses parmhet with metaparm
to perform a DerSimonian-Laird randomly-variable-effects meta-analysis
(DerSimonian and Laird, 1986).
This time, parmhet adds variables semiwt and swse,
containing semi-weights and semi-weight-based standard errors, respectively.
These are passed to metaparm as aweights and standard errors,
respectively.

. parmhet estimate stderr, eform list(,) sweight(semiwt) sstderr(swse)

. metaparm [aweight=semiwt], eform sumvar(N) stderr(swse) list(,)

The following example performs a nonrandomly-variable-effects analysis,
weighted by the study size variable N.

. parmhet estimate stderr, eform sumvar(N) list(,)

. metaparm [aweight=N], eform sumvar(N) list(,)

The following example carries out a nonrandomly-variable-effects meta-analysis,
saving the resultssets from parmhet and metaparm
in disk-file datasets myhet and mymeta, respectively.
These resultssets are appended to the existing resultsset to produce an extended resultsset,
which is listed to show the study odds ratios, the weighted geometric mean (GM) odds ratio,
and P-values for the odds ratios and for the heterogeneity test.

. preserve

. gene idstr="Single study"

. order idstr

. parmhet estimate stderr, eform sumvar(N) idstr(Heterogeneity) phet(p) keep(idstr N p) saving(myhet.dta, replace)

. metaparm [aweight=N], eform sumvar(N) idstr("Weighted GM") saving(mymeta.dta, replace)

. append using mymeta

. append using myhet

. describe

. list idstr studyseq doseiu dosefreq N estimate min* max* p, sepby(idstr)

. restore

Examples: Sequence 2

The following examples also use the dataset parmhet_example1.
However, this time, the parmby module of parmest is used
to produce a dataset of log odds ratios.
These are then tested for heterogeneity,
using parmhet without the eform option.

Set-up

. use "http://www.imperial.ac.uk/nhli/r.newson/stata10/parmhet_example1.dta", clear

. describe

. parmby "blogit deaths number vitamina", by(studyseq doseiu dosefreq) norestore escal(N) rename(es_1 N)

. keep if parm=="vitamina"

. describe

. list studyseq doseiu dosefreq parm N estimate stderr min* max* p

Simple examples

. parmhet estimate stderr, list(,)

. parmhet estimate stderr, by(doseiu) list(,)

Examples: Sequence 3

The following example uses the auto data, distributed with official Stata,
and adds a new variable continent,
containing the continent of origin of a car model.
The parmby module of parmest is then used
to create a resultsset with 1 parameter per continent
and data on mean weights of cars (in US pounds),
with numbers of car models, estimates, standard errors, degrees of freedom, confidence limits,
and P-values.
This resultsset is then input to parmhet
to perform a Welch unequal-variance heterogeneity test (Welch, 1951),
which shows that mean car weights vary between continents.

Set-up

. sysuse auto, clear

. gene firm=word(make,1)

. lab var firm "Firm"

. lab def continent 1 "America" 2 "Asia" 3 "Europe"

. gene continent=1 if !foreign

. replace continent=2 if inlist(firm,"Datsun","Honda","Mazda","Subaru","Toyota")

. replace continent=3 if inlist(firm,"Audi","BMW","VW","Fiat","Peugeot","Renault","Volvo")

. lab val continent continent

. lab var continent "Continent of origin"

. describe

. tab firm continent, missing

. parmby "mean weight", by(continent) norestore

. describe

. list, abbr(32)

Examples

. parmhet estimate stderr dof, list(,)

. parmhet estimate stderr dof, list(, subvarname abbr(32))

Author

Roger Newson, National Heart and Lung Institute, Imperial College London, UK.
Email: r.newson@imperial.ac.uk

References

Bland, M. 2000. An introduction to medical statistics. 3rd ed.
Oxford: Oxford University Press.

Cochran, W. G. 1954. The combination of estimates from different experiments.
Biometrics 10(1): 101-129.

DerSimonian, R. and Laird, N. 1986. Meta-analysis in clinical trials.
Controlled Clinical Trials 7(3): 177-188.

Glasziou, P.P. and Mackerras, D. E. M. 1993.
Vitamin A supplementation in infectious disease: a meta–analysis.
British Medical Journal 306(6874): 366-370.

Higgins, J. P. T. and Thompson, S. G. 2002. Quantifying heterogeneity in a meta-analysis.
Statistics in Medicine 21(11): 1539-1558.

Newson, R. B. 2010. Post-parmest peripherals: fvregen, invcise, and qqvalue.
Presented at the 16th United Kingdom Stata Users' Group Meeting, London, 9-10 September, 2010.

Newson, R. B. 2008. parmest and extensions.
Presented at the 14th United Kingdom Stata Users' Group Meeting, London, 8-9 September, 2008.

Newson, R. 2006. Resultssets, resultsspreadsheets, and resultsplots in Stata.
Presented at the 4th German Stata Users' Group Meeting, Mannheim, 31 March, 2006.

Newson, R. 2004. From datasets to resultssets in Stata.
Presented at the 10th United Kingdom Stata Users' Group Meeting, London, 29-30 June, 2004.

Newson, R. 2003. Confidence intervals and p-values for delivery to the end user.
The Stata Journal 3(3): 245-269.
Download from The Stata Journal website.

Newson, R. 2002. Creating plots and tables of estimation results using parmest and friends.
Presented at the 8th United Kingdom Stata Users' Group Meeting, 20-21 May, 2002.

Welch, B. L. 1951. On the comparison of several mean values: an alternative approach.
Biometrika 36(3/4): 330-336.

Also see

Manual: [R] meta, [R] test

On-line: help for parmhet_basic_opts,
parmhet_resultsset_opts,
parmhet_hettest_opts,
parmhet_resultsset

help for test, list

help for parmest, parmby, parmcip, metaparm, eclplot, listtab,
estparm, metan if installed