help for metaparm and parmcip (Roger Newson)

Meta-analysis and calculating confidence intervals using parmest-format resultssets

parmcip [if] [in] [ , parmcip_opts ]

metaparm [weight] [if] [in] [ ,
metaparm_outdest_opts
metaparm_content_opts
parmcip_opts
]

aweights and iweights are allowed with metaparm.
See help for weights.

by varlist: can be used with parmcip and metaparm.
(See help for by.)
However, if by varlist: is used with metaparm,
then a by() option must also be present,
and must start with the variables in the by varlist: prefix.

Description

metaparm and parmcip are resultsset-processing programs
designed for use with parmest and parmby resultssets,
which have one observation per estimated parameter and data on parameter estimates.
(See Newson (2010), Newson (2008), Newson (2006), Newson (2004), Newson (2003) and Newson (2002).)
metaparm inputs a dataset in memory with one observation per parameter and variables containing
estimates, standard errors and (optionally) degrees of freedom, and creates an output dataset with one observation,
or one observation per by-group, and data on estimates, standard errors,
z- or t-statistics, P-values and confidence limits
for a linear combination (weighted sum) of the input parameters,
assuming that the input parameters are estimated independently (or at least are uncorrelated) for different sub-populations.
The output dataset can be listed to the Stata log, saved to a new data frame, saved to a disk file,
or written to the current data frame, overwriting the existing dataset.
metaparm is useful for meta-analyses (where aweights are used),
and can also be used with iweights for calculating confidence intervals
and P-values for differences or ratios between parameters for different subpopulations.
parmcip inputs a dataset in memory with one observation per parameter
and variables containing parameter estimates, standard errors and (optionally) degrees of freedom,
and adds new variables to the dataset, containing the confidence limits, z- or t-statistics,
P-values, and (optionally) stars for P-values.
parmcip is useful if the user requires confidence intervals for a Normalizing
and/or variance-stabilizing transformation of the original parameters.
metaparm works by calling parmcip, and therefore should not be downloaded without parmcip.

Output datasets created by metaparm and parmcip

These output datasets (or resultssets) are described in detail in metaparm_resultssets.

Options for metaparm and parmcip

metaparm and parmcip have a large number of options, which fall into the following 3 groups:

Option group Description

metaparm_outdest_opts Output-destination options for metaparm

metaparm_content_opts Output-content options for metaparm

parmcip_opts Options for metaparm and parmcip

Output-destination options for metaparm

list(list_spec) List output dataset to Stata log and/or Results window

frame(framename[,replace change]) Save output dataset to a data frame

saving(filename[,replace]) Save output dataset to a disk file

norestore Write output dataset to memory

fast Write output dataset to memory without precautions

flist(global_macro_name) Append output filename to a global macro

where list_spec is a specification of the form

[varlist] [if] [in] [ , [list_options ] ]

and list_options is a list of options used by the list command.

See metaparm_outdest_opts for details of these options.

Output-content options for metaparm

by(varlist) Variables specifying by-groups

sumvar(varlist) Variables to be summed in output dataset

dfcombine(combination_rule) Rule for combining degrees of freedom

idnum(#) Value of numeric dataset ID variable

nidnum(newvarname) Name of numeric dataset ID variable

idstr(string) Value of string dataset ID variable

nidstr(newvarname) Name of string dataset ID variable

format(formatting_list) Display formats for variables in the output dataset

where combination_rule is

satterthwaite | welch | constant

and formatting_list is a list of form

varlist_1 format_1 ... varlist_n format_n

See metaparm_content_opts for details of these options.

Options for metaparm and parmcip

notdist Use Normal or t-distribution

eform Estimates and confidence limits exponentiated

float Numeric output variables of type float or less

nozerop Reset zero P-values to smallest positive double-precision number

nullvalue(#) Value of estimated parameters under null hypotheses

fast Calculate confidence limits without precautions

plower Create variable containing lower one-tailed P-values

pupper Create variable containing upper one-tailed P-values

estimate(varname) Name of input estimate variable

stderr(varname) Name of input standard error variable

dof(varname) Name of input degrees of freedom variable

zstat(newvarname) Name of output z-statistic variable

tstat(newvarname) Name of output t-statistic variable

pvalue(newvarname) Name of output P-value variable

plovalue(newvarname) Name of output lower one-sided P-value variable

pupvalue(newvarname) Name of output upper one-sided P-value variable

stars(numlist) List of P-value thresholds for stars

nstars(newvarname) Name of output stars variable for P-value

nstarslower(newvarname) Name of output stars variable for lower one-sided P-value

nstarsupper(newvarname) Name of output stars variable for upper one-sided P-value

level(numlist) Confidence level(s) for calculating confidence limits

clnumber(numbering_rule) Numbering rule for naming confidence limit variables

minprefix(prefix) Prefix for lower confidence limits

maxprefix(prefix) Prefix for upper confidence limits

mcompare(method) Multiple-comparison method

mcomci(method) Multiple-comparison method for confidence limits only

replace Replace variables with same names as output variables

where numbering_rule is

level | rank

and method is

noadjust | bonferroni | sidak

See parmcip_opts for details of these options.

Methods and formulas

metaparm generates an output dataset with one observation, or one observation per by-group,
and data on estimates, standard errors and degrees of freedom
for linear combinations (weighted sums) of parameters,
and then uses parmcip to derive the t- or z-statistics, P-values and confidence limits.
The exact method used to calculate the estimates, standard errors and degrees of freedom
depends on whether the tdist option is used,
and on whether aweights or iweights are specified.
In general, it is assumed that, in the input dataset (or in a by-group of the input dataset),
there are N observations, corresponding to parameters theta_1, ..., theta_N,
with corresponding standard errors se_1, ..., se_N,
corresponding coefficients a_1, ..., a_N,
and corresponding degrees of freedon df_1, ..., df_N if the tdist option is used.
We wish to calculate an estimate
for the linear combination

Theta = Sum ( a_j * theta_j )

and a corresponding standard error

SE = sqrt( Sum (a_j * se_j)^2 )

and degrees of freedom (if tdist is specified) calculated
from the df_j and the se_j.
The calculation of the degrees of freedom uses
the Satterthwaite formula (Satterthwaite, 1946) if dfcombine(satterthwaite) is specified,
or the Welch formula (Welch, 1947) if dfcombine(welch) is specified.
If dfcombine(constant) is specified.
then metaparm checks that the input degrees of freedom df_j are all equal,
and then sets the output degrees of freedom to the input degrees of freedom.
The definition of the a_j depends on whether
iweights or aweights are specified. If iweights are specified,
then the a_j are given by the result of the weight expression,
which may be positive, zero or negative.
If aweights are specified, then the result of the weight expression
must be non-negative, and is divided by its total within the input dataset or by-group to give the a_j.
If no weights are specified, then aweights are assumed, and are set to 1,
so that the a_j are equal to 1/N, and their sum in the dataset or by-group is equal to 1.
aweights are typically specified if the user wishes to carry out a meta-analysis,
whereas iweights are typically specified if the user wishes to estimate a difference
between two parameters theta_1 and theta_2 in each by-group.

To perform meta-analyses,
metaparm should usually be used together with the parmhet package,
which can also be downloaded from SSC,
and which calculates heterogeneity test statistics,
and also inverse-variance weights for input to metaparm.

Remarks

More information about parmest and parmby resultssets
can be found in Newson (2010), Newson (2008), Newson (2006), Newson (2004), Newson (2003) and Newson (2002).

Other programs are available in the Stata community for carrying out meta-analyses,
including metan,
written by Stephen Sharp and Jonathan Sterne and downloadable from SSC.
(See [R] meta for details.)
metaparm is complementary to these, and is designed specifically for use with parmest resultssets,
and with the SSC package parmhet.
Note, however, that the input resultsset does not have to be produced by parmest.
It need only contain an estimate variable, a standard error variable, and a degrees of freedom variable
(if tdist is specified).
Linear combinations of parameters for the same model can be estimated
using the official Stata command lincom, or by the lincomest package,
downloadable from SSC.
metaparm is complementary to these, and is designed for use when parameters are estimated
by sampling independently from distinct subpopulations.

parmcip may be used for defining additional confidence limits in a parmest resultsset,
with different confidence levels from those originally calculated by parmby
or parmest. However, parmcip is also useful for calculating confidence limits for
transformed parameters, using standard errors calculated using the
delta method.

The confidence limits calculated by metaparm or parmcip can be plotted
using the eclplot command, which is downloadable from SSC,
and which can be used to calculate Cochrane forest plots for meta-analyses.

Examples

. metaparm, list(,)

. metaparm [awei=studynum], sumvar(studynum) list(,)

. parmcip

. parmcip, replace

The following example uses the auto data. A variable mod4 is defined, equal to 0, 1, 2 or 3 for approximately equal numbers of cars.
We then use parmby to fit a regression model, comparing mileage in non-American and American cars, for cars with each value of mod4,
and to store the parameters in a parmby resultsset in memory, which is listed.
Finally, we carry out a meta-analysis on the differences between the non-American and American cars in the 4 mod4 groups,
weighting the differences by numbers of cars to produce a weighted mean difference, which is listed,
together with its 95% confidence limits and P-value.

. sysuse auto, clear

. gene byte mod4=mod(_n,4)

. parmby "regress mpg foreign", by(mod4) norestore escal(N) rename(es_1 N) format(estimate min* max* %8.2f p %-8.2g)

. bysort mod4 (N parmseq): list

. metaparm [awei=N] if parm=="foreign", sumvar(N) list(,)

The following example uses the auto data. A variable odd is defined, equal to 0 for even-numbered cars
and 1 for odd-numbered cars in the order in the dataset. We then demonstrate that metaparm (with iweights)
produces the same results as ttest with the unequal option when comparing the weights of American and non-American cars
within the odd-numbered and even-numbered categories. metaparm has the advantage that it can store the confidence intervals in a resultsset
on disk or memory, which can later be plotted using eclplot, although this is not done here.

. sysuse auto, clear

. gene byte odd=mod(_n,2)

. bysort odd: ttest weight, by(foreign) unequal

. parmby "regress weight", by(odd foreign) norestore format(estimate min* max* %8.2f p %-8.2g)

. list

. metaparm [iwei=!foreign-foreign], by(odd) norestore

. list odd estimate min* max* p dof

The following example uses the auto data to demonstrate the use of metaparm,
with the option dfcombine(constant), to carry out an equal-variance t-test
between weights of US and non-US cars.
This is done using a parmby output dataset,
in which the command factext (downloadable from SSC) has been used
to reconstruct the variable foreign.
Again, the use of metaparm allows the user to save confidence intervals to be plotted and/or tabulated,
although this is not done here.

. sysuse auto, clear

. ttest weight, by(foreign)

. parmby "xi, noomit: regress weight i.foreign, noconst", norestore label format(estimate min* max* %8.2f p %-8.2g)

. factext

. list

. metaparm [iwei=!foreign-foreign], dfcombine(constant) list(,)

The following example uses the svyset and svy: tabulate commands,
and the SSC package xcontract, in the auto data
to demonstrate the use of parmcip for calculating transformed confidence intervals.
It uses xcontract to calculate counts and proportions and save them to a file on disk,
uses svy: tabulate
to calculate confidence intervals for the proportions of cars with each combination of
the values of the variables rep78 and foreign,
and uses parmest
to save the estimates and standard errors of these proportions in a resultsset in memory.
We then replace the estimates and standard errors of the proportions
with the estimates and standard errors of the corresponding logged odds,
drop the existing t-statistics, P-values and confidence limits,
use parmcip to calculate t-statistics, P-values and confidence limits for the logged odds,
and then calculate new estimates and confidence limits for the proportions by
back-transforming the estimates and confidence limits for the logged odds.
These confidence limits are listed,
and are identical to those displayed by svy: tabulate,
but have the advantage that they may then be plotted using eclplot,
or listed using formats chosen by the user.
(This is an example of the
delta method.
See [SVY] svy: tabulate twoway for details of the formulas.)

. sysuse auto, clear

. tempfile tf1

. xcontract rep78 foreign, zero saving(`"`tf1'"', replace)

. gene byte pwt=1

. version 8: svyset [pweight=pwt]

. svy: tabulate rep78 foreign, ci

. parmest, norestore

. list

. replace stderr=stderr/(estimate*(1-estimate))

. replace estimate=log(estimate/(1-estimate))

. drop t p min* max*

. parmcip

. gene estimate_2=exp(estimate)/(1+exp(estimate))

. gene min95_2=exp(min95)/(1+exp(min95))

. gene max95_2=exp(max95)/(1+exp(max95))

.merge using `"`tf1'"'

. sort rep78 foreign

. list rep78 foreign _freq estimate_2 min95_2 max95_2, sepby(rep78)

Acknowledgements

I would like to thank Jill Starkes and Lee Sieswerda of Thunder Bay District Health Unit for drawing my attention
to the example of the delta method used to calculate the confidence intervals
displayed by svy: tabulate.

Author

Roger Newson, King's College London, UK.
Email: roger.newson@kcl.ac.uk

References

Newson, R. B. 2010. Post-parmest peripherals: fvregen, invcise, and qqvalue.
Presented at the 16th United Kingdom Stata Users' Group Meeting, London, 9-10 September, 2010.

Newson, R. B. 2008. parmest and extensions.
Presented at the 14th United Kingdom Stata Users' Group Meeting, London, 8-9 September, 2008.

Newson, R. 2006. Resultssets, resultsspreadsheets, and resultsplots in Stata.
Presented at the 4th German Stata Users' Group Meeting, Mannheim, 31 March, 2006.

Newson, R. 2004. From datasets to resultssets in Stata.
Presented at the 10th United Kingdom Stata Users' Group Meeting, London, 29-30 June, 2004.

Newson, R. 2003. Confidence intervals and p-values for delivery to the end user.
The Stata Journal 3(3): 245-269.
Download from
the Stata Journal website.

Newson, R. 2002. Creating plots and tables of estimation results using parmest and friends.
Presented at the 8th United Kingdom Stata Users' Group Meeting, 20-21 May, 2002.

Satterthwaite, F. E. 1946. An approximate distribution of estimates of variance components.
Biometrics Bulletin 2(6): 110-114.

Welch, B. L. 1947. The generalization of `Student's' problem when several different population variances are involved.
Biometrika 34(1/2): 28-35.

Also see

Manual: ,

[I] estimation commands,

[D] append, [R] lincom, [R] meta, [SVY] svyset,
[SVY] svy: tabulate oneway, [SVY] svy: tabulate twoway.

Help: [U] 20 Estimation and postestimation commands,

[I] estimation commands,

[D] append, [R] lincom, [R] meta, [SVY] svyset,
[SVY] svy: tabulate oneway, [SVY] svy: tabulate twoway

parmest, parmby,
metaparm_outdest_opts, metaparm_content_opts,
parmcip_opts, metaparm_resultssets, parmest_resultssets

meta, eclplot, factext, lincomest, parmhet, qqvalue, xcontract if installed