Meta-analysis and calculating confidence intervals using parmest-format results > sets
parmcip [if] [in] [ , parmcip_opts ]
metaparm [weight] [if] [in] [ , metaparm_outdest_opts metaparm_content_opts parmcip_opts ]
aweights and iweights are allowed with metaparm. See help for weights.
Description
metaparm and parmcip are resultsset-processing programs designed for use with parmest and parmby resultssets, which have one observation per estimated parameter and data on parameter estimates. (See Newson (2006), Newson (2004), Newson (2003) and Newson (2002).) metaparm inputs a dataset in memory with one observation per parameter and variables containing estimates, standard errors and (optionally) degrees of freedom, and creates an output dataset with one observation, or one observation per by-group, and data on estimates, standard errors, z- or t-statistics, P-values and confidence limits for a linear combination (weighted sum) of the input parameters, assuming that the input parameters are estimated independently (or at least are uncorrelated) for different sub-populations. The output dataset can be listed to the Stata log, saved to a disk file, or written to memory, overwriting the existing dataset. metaparm is useful for meta-analyses (where aweights are used), and can also be used with iweights for calculating confidence intervals and P-values for differences or ratios between parameters for different subpopulations. parmcip inputs a dataset in memory with one observation per parameter and variables containing parameter estimates, standard errors and (optionally) degrees of freedom, and adds new variables to the dataset, containing the confidence limits, z- or t-statistics, P-values, and (optionally) stars for P-values. parmcip is useful if the user requires confidence intervals for a Normalizing and/or variance-stabilizing transformation of the original parameters. metaparm works by calling parmcip, and therefore should not be downloaded without parmcip.
Options
metaparm and parmcip have a large number of options, which fall into the following 3 groups:
Option group Description -------------------------------------------------------------------------- metaparm_outdest_opts Output-destination options for metaparm metaparm_content_opts Output-content options for metaparm parmcip_opts Options for metaparm and parmcip --------------------------------------------------------------------------
Output-destination options for metaparm
options Description ------------------------------------------------------------------------- list(list_spec) List output dataset to Stata log and/or Results window saving(filename[,replace]) Save output dataset to a disk file norestore Write output dataset to memory fast Write output dataset to memory without precautions flist(global_macro_name) Append output filename to a global macro -------------------------------------------------------------------------
where list_spec is a specification of the form
[varlist] [if] [in] [ , [list_options ] ]
and list_options is a list of options used by the list command.
See metaparm_outdest_opts for details of these options.
Output-content options for metaparm
options Description ------------------------------------------------------------------------- by(varlist) Variables specifying by-groups sumvar(varlist) Variables to be summed in output dataset dfcombine(combination_rule) Rule for combining degrees of freedom idnum(#) Value of numeric dataset ID variable nidnum(newvarname) Name of numeric dataset ID variable idstr(string) Value of string dataset ID variable nidstr(newvarname) Name of string dataset ID variable format(formatting_list) Display formats for variables in the output dataset -------------------------------------------------------------------------
where combination_rule is
satterthwaite | constant
and formatting_list is a list of form
varlist_1 format_1 ... varlist_n format_n
See metaparm_content_opts for details of these options.
Options for metaparm and parmcip
options Description ------------------------------------------------------------------------- notdist Use Normal or t-distribution eform Estimates and confidence limits exponentiated float Numeric output variables of type float or less fast Calculate confidence limits without precautions estimate(varname) Name of input estimate variable stderr(varname) Name of input standard error variable dof(varname) Name of input degrees of freedom variable zstat(newvarname) Name of output z-statistic variable tstat(newvarname) Name of output t-statistic variable pvalue(newvarname) Name of output P-value variable stars(numlist) List of P-value thresholds for stars nstars(newvarname) Name of output stars variable level(numlist) Confidence level(s) for calculating confidence limits clnumber(numbering_rule) Numbering rule for naming confidence limit variables minprefix(prefix) Prefix for lower confidence limits maxprefix(prefix) Prefix for upper confidence limits replace Replace variables with same names as output variables -------------------------------------------------------------------------
where numbering_rule is
level | rank
See parmcip_opts for details of these options.
Output datasets created by metaparm and parmcip
These output datasets (or resultssets) are described in detail in metaparm_resultssets.
Methods and formulas
metaparm generates an output dataset with one observation, or one observation per by-group, and data on estimates, standard errors and degrees of freedom for linear combinations (weighted sums) of parameters, and then uses parmcip to derive the t- or z-statistics, P-values and confidence limits. The exact method used to calculate the estimates, standard errors and degrees of freedom depends on whether the tdist option is used, and on whether aweights or iweights are specified. In general, it is assumed that, in the input dataset (or in a by-group of the input dataset), there are N observations, corresponding to parameters theta_1, ..., theta_N, with corresponding standard errors se_1, ..., se_N, corresponding coefficients a_1, ..., a_N, and corresponding degrees of freedon df_1, ..., df_N if the tdist option is used. We wish to calculate an estimate for the linear combination
Theta = Sum ( a_j * theta_j )
and a corresponding standard error
SE = sqrt( Sum (a_j * se_j)^2 )
and degrees of freedom (if tdist is specified) calculated from the df_j and the se_j by the Satterthwaite formula (Satterthwaite, 1946), unless the user specifies the option dfcombine(constant), in which case metaparm checks that the input degrees of freedom df_j are all equal, and then sets the output degrees of freedom to the input degrees of freedom. The definition of the a_j depends on whether iweights or aweights are specified. If iweights are specified, then the a_j are given by the result of the weight expression, which may be positive, zero or negative. If aweights are specified, then the result of the weight expression must be non-negative, and is divided by its total within the input dataset or by-group to give the a_j. If no weights are specified, then aweights are assumed, and are set to 1, so that the a_j are equal to 1/N, and their sum in the dataset or by-group is equal to 1. aweights are typically specified if the user wishes to carry out a meta-analysis, whereas iweights are typically specified if the user wishes to estimate a difference between two parameters theta_1 and theta_2 in each by-group.
Remarks
More information about parmest and parmby resultssets can be found in Newson (2006), Newson (2004), Newson (2003) and Newson (2002). Other programs are available in the Stata community for carrying out meta-analyses, including meta, written by Stephen Sharp and Jonathan Sterne. (See [R] meta for details.) metaparm is complementary to these, and is designed specifically for use with parmest resultssets. Note, however, that the input resultsset does not have to be produced by parmest. It need only contain an estimate variable, a standard error variable, and a degrees of freedom variable (if tdist is specified). Linear combinations of parameters for the same model can be estimated using the official Stata command lincom, or by the lincomest package, downloadable from SSC. metaparm is complementary to these, and is designed for use when parameters are estimated by sampling independently from distinct subpopulations.
parmcip may be used for defining additional confidence limits in a parmest resultsset, with different confidence levels from those originally calculated by parmby or parmest. However, parmcip is also useful for calculating confidence limits for transformed parameters, using standard errors calculated using the delta method.
The confidence limits calculated by metaparm or parmcip can be plotted using the eclplot command, which is downloadable from SSC, and which can be used to calculate Cochrane forest plots for meta-analyses.
Examples
. metaparm, list(,)
. metaparm [awei=studynum], sumvar(studynum) list(,)
. parmcip
. parmcip, replace
The following example uses the auto data. A variable mod4 is defined, equal to 0, 1, 2 or 3 for approximately equal numbers of cars. We then use parmby to fit a regression model, comparing mileage in non-American and American cars, for cars with each value of mod4, and to store the parameters in a parmby resultsset in memory, which is listed. Finally, we carry out a meta-analysis on the differences between the non-American and American cars in the 4 mod4 groups, weighting the differences by numbers of cars to produce a weighted mean difference, which is listed, together with its 95% confidence limits and P-value.
. sysuse auto, clear . gene byte mod4=mod(_n,4) . parmby "regress mpg foreign", by(mod4) norestore escal(N) rename(es_1 N) format(estimate min* max* %8.2f p %-8.2g) . bysort mod4 (N parmseq): list . metaparm [awei=N] if parm=="foreign", sumvar(N) list(,)
The following example uses the auto data. A variable odd is defined, equal to 0 for even-numbered cars and 1 for odd-numbered cars in the order in the dataset. We then demonstrate that metaparm (with iweights) produces the same results as ttest with the unequal option when comparing the weights of American and non-American cars within the odd-numbered and even-numbered categories. metaparm has the advantage that it can store the confidence intervals in a resultsset on disk or memory, which can later be plotted using eclplot, although this is not done here.
. sysuse auto, clear . gene byte odd=mod(_n,2) . bysort odd: ttest weight, by(foreign) unequal . parmby "regress weight", by(odd foreign) norestore format(estimate min* max* %8.2f p %-8.2g) . list . metaparm [iwei=!foreign-foreign], by(odd) norestore . list odd estimate min* max* p dof
The following example uses the auto data to demonstrate the use of metaparm, with the option dfcombine(constant), to carry out an equal-variance t-test between weights of US and non-US cars. This is done using a parmby output dataset, in which the command factext (downloadable from SSC) has been used to reconstruct the variable foreign. Again, the use of metaparm allows the user to save confidence intervals to be plotted and/or tabulated, although this is not done here.
. sysuse auto, clear . ttest weight, by(foreign) . parmby "xi, noomit: regress weight i.foreign, noconst", norestore label format(estimate min* max* %8.2f p %-8.2g) . factext . list . metaparm [iwei=!foreign-foreign], dfcombine(constant) list(,)
The following example uses the svyset and svy: tabulate commands, and the SSC package xcontract, in the auto data to demonstrate the use of parmcip for calculating transformed confidence intervals. It uses xcontract to calculate counts and proportions and save them to a file on disk, uses svy: tabulate to calculate confidence intervals for the proportions of cars with each combination of the values of the variables rep78 and foreign, and uses parmest to save the estimates and standard errors of these proportions in a resultsset in memory. We then replace the estimates and standard errors of the proportions with the estimates and standard errors of the corresponding logged odds, drop the existing t-statistics, P-values and confidence limits, use parmcip to calculate t-statistics, P-values and confidence limits for the logged odds, and then calculate new estimates and confidence limits for the proportions by back-transforming the estimates and confidence limits for the logged odds. These confidence limits are listed, and are identical to those displayed by svy: tabulate, but have the advantage that they may then be plotted using eclplot, or listed using formats chosen by the user. (This is an example of the delta method. See [SVY] svy: tabulate twoway for details of the formulas.)
. sysuse auto, clear . tempfile tf1 . xcontract rep78 foreign, zero saving(`tf1', replace) . gene byte pwt=1 . version 8: svyset [pweight=pwt] . svy: tabulate rep78 foreign, ci . parmest, norestore . list . replace stderr=stderr/(estimate*(1-estimate)) . replace estimate=log(estimate/(1-estimate)) . drop t p min* max* . parmcip . gene estimate_2=exp(estimate)/(1+exp(estimate)) . gene min95_2=exp(min95)/(1+exp(min95)) . gene max95_2=exp(max95)/(1+exp(max95)) . merge using `tf1' . sort rep78 foreign . list rep78 foreign _freq estimate_2 min95_2 max95_2, sepby(rep78)
Acknowledgements
I would like to thank Jill Starkes and Lee Sieswerda of Thunder Bay District Health Unit for drawing my attention to the example of the delta method used to calculate the confidence intervals displayed by svy: tabulate.
Author
Roger Newson, Imperial College London, UK. Email: r.newson@imperial.ac.uk
References
Newson, R. 2006. Resultssets, resultsspreadsheets, and resultsplots in Stata. Presented at the 4th German Stata Users' Group Meeting, Mannheim, 31 March, 2006. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.
Newson, R. 2004. From datasets to resultssets in Stata. Presented at the 10th United Kingdom Stata Users' Group Meeting, London, 29 June, 2004. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.
Newson, R. 2003. Confidence intervals and p-values for delivery to the end user. The Stata Journal 3(3): 245-269. Pre-publication draft downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.
Newson, R. 2002. Creating plots and tables of estimation results using parmest and friends. Presented at the 8th UK Stata Users' Group Meeting, 20-21 May, 2002. Also downloadable from Roger Newson's website at http://www.imperial.ac.uk/nhli/r.newson.
Satterthwaite, F. E. 1946. An approximate distribution of estimates of variance components. Biometrics Bulletin 2(6): 110-114.
Also see
Manual: [U] 20 Estimation and postestimation commands, [D] append, [R] lincom, [R] meta, [SVY] svyset, [SVY] svy: tabulate oneway, [SVY] svy: tabulate twoway. On-line: help for append, lincom, svyset, svy tabulate oneway, svy tabulate twoway help for parmest, parmby, metaparm_outdest_opts, metaparm_content_opts, parmcip_opts, metaparm_resultssets, parmest_resultssets help for meta, dsconcat, factext, lincomest, xcontract if installed