help for rcentile(Roger Newson)

Robust confidence intervals for percentiles allowing for clusters and weights

rcentile depvar [weight] [if] [in]
[ , centile(numlist) level(#)
cluster(varname)
cfweight(expression)
tdist transf(transformation_name)
fast
]

where transformation_name is one of

iden | z | asin

fweights, iweights, and pweights are allowed; see
weight.

bootstrap, by, jackknife, and statsby
are allowed; see prefix.

Description

rcentile calculates robust confidence intervals for percentiles,
allowing for clustered sampling and/or weighting if necessary.
rcentile requires the SSC packages somersd, scsomersd and expgen in order to work.

Options for use with rcentile

centile(numlist) specifies a list of percentiles
to be reported and defaults to centile(50) (median only) if not
specified. Specifying centile(25 50 75) will produce the 25th, 50th, and
75th percentiles.

level(#) specifies the confidence level, as a percentage, for
confidence intervals; see level.

cluster(varname) specifies the variable that defines sampling clusters.
If cluster() is defined,
then the confidence intervals are calculated
assuming that the data are a sample of clusters from a population of clusters,
rather than a sample of observations from a population of observations.

cfweight(expression) specifies an expression giving the
cluster frequency weights. These cluster frequency weights must have the same
value for all observations in a cluster. If cfweight() and
cluster() are both specified,
then each cluster in the dataset is
assumed to represent a number of identical clusters
equal to the cluster frequency weight for that cluster.
If cfweight() is specified and cluster() is unspecified,
then each observation in the dataset is treated as a cluster,
and assumed to represent a number of identical one-observation clusters equal to the cluster frequency weight.
For more details on the interpretation of weights,
see Interpretation of weights in the help for somersd.

tdist specifies that the standardized Somers' D estimates are
assumed to be sampled from a t-distribution with n-1 degrees of freedom,
where n is the number of clusters,
or the number of observations if cluster() is not specified.
If tdist is not specified,
then the standardized Somers' D estimates are assumed to be sampled from a standard Normal distribution.
Simulation study data suggest that the tdist option should be used.

transf(transformation_name) specifies
that the estimates of mean signs are to be transformed,
defining a standard error for the transformed population value,
from which the confidence limits for the percentile differences are calculated.
z (the default) specifies Fisher's z (the hyperbolic arctangent),
asin specifies Daniels' arcsine,
and iden specifies identity or untransformed.

fast is a programmer's option.
It specifies that rcentile will not do any work to restore the original dataset
if the user presses Break.

Methods and formulas

rcentile is described in Newson (2014).
It estimates confidence intervals for percentiles,
using standard errors for mean signs to derive them.
The 100qth percentile of a variable Y can be defined as a solution, in theta,
to the equation

E[sign( Y - theta )] = 1-2q

where sign(.) denotes the sign function (-1 for negative arguments, +1 for positive arguments
and 0 for zero arguments).
At each value of theta, the expression E[sign( Y - theta )] is known as a mean sign.
More precisely, the 100qth percentile of Y can be defined
as the mean of the left and right inverse folded reverse ridits (IFRRs) of q.
The left IFRR of q is defined as

sup{theta: E[sign( Y - theta )] > 1-2q }

and the right IFRR of q is defined as

inf{theta: E[sign( Y - theta )] < 1-2q }

where the supremum and infimum of an empty set of numbers are set to minus and plus infinity,
respectively.
The mean of a finite number and an infinite number is set to the finite number.

Note that this method,
when applied to the sample distribution of Y,
produces percentiles defined in the same way
as those produced by collapse.
A slightly different definition of percentiles is used by centile.
However, if the variable Y really is continuous,
then the 2 percentile definitions are consistent estimators of the same parameter.

rcentile produces confidence limits for percentiles using the delta-jackknife standard error of a transformation

g(E[sign( Y - theta )])

where g(.) is a function, specified by the transf() option,
which may be the hyperbolic arctangent (Fisher's z),
the arcsine (Daniels' arcsine),
or the identity.
This standard error is used to define lower and upper confidence limits (by end point transformation)
for the mean sign at the chosen percentile theta.
These limits, in turn, are used to define lower and upper confidence limits for the percentile,
equal to the sample left IFRR of the upper confidence limit for the mean sign
and to the sample right IFRR of the lower confidence limit for the mean sign,
respectively.

Note that this method uses standard errors for the mean sign,
or for a transformwd mean sign,
instead of using standard errors for the percentile itself.
This is probably a good idea,
because standard errors are usually estimated using mean squared influence functions,
and the central limit theorem works a lot faster
for the influence function of a mean sign
than for the influence function of a percentile.
See Hampel (1974) and Hampel et al (1986)
for more about influence functions,
for percentiles and other statistics.

rcentile works by calling the sccendif module of the scsomersd paclage,
which in turn calls the expgen package and the cendif module of the somersd package,
which in turn calls the somersd module of the somersd package.
This method treats a percentile as a special case of a percentile slope (see Newson, 2006a),
and treats a mean sign as a special case of Somers' D (see Newson, 2006b).
For a comprehensive review of Kendall's tau-a, Somers' D, and median differences,
see Newson (2002).
The packages scsomersd, somersd and expgen
can be downloaded from SSC.

Examples

Set-up:

. sysuse auto, clear

. gene firm=word(make,1)

. label var firm "Firm"

. tab firm, m

Simple examples:

. rcentile length, tdist

. rcentile length, tdist transf(iden)

. rcentile length, tdist centile(0(25)100)

. rcentile length, tdist centile(0(25)100) cluster(firm)

. rcentile length [pweight=price], tdist centile(0(25)100)

. rcentile length [pweight=price], tdist centile(0(25)100) cluster(firm)

. by foreign: rcentile length [pweight=price], tdist centile(0(25)100) cluster(firm)

Note that some of these examples will produce "infinite" confidence limits,
equal to the creturn results c(mindouble) for lower limits
and c(maxdouble) for upper limits.
(See help for creturn.)
This is because the correct confidence intervals for percentiles may be infinite,
especially for percentiles 0 and 100.
See Newson, 2006a.

Saved results

rcentile saves the following in r():

Scalars

r(N) number of observations

r(N_clust) number of clusters

r(df_r) residual degrees of freedom (if tdist present)

r(level) confidence level

Macros

r(depvar) name of Y-variable

r(clustvar) name of cluster variable

r(cfweight) cfweight() expression

r(tdist) tdist if specified

r(wtype) weight type

r(wexp) weight expression

r(centiles) list of percents for percentiles

r(transf) transformation specified by transf()

r(tranlab) transformation label in output

Matrices

r(cimat) confidence intervals for percentiles

Author

Roger Newson, Imperial College London, UK.

Email: r.newson@imperial.ac.uk

References

Hampel, F. R., E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. 1986.
Robust Statistics: The Approach Based on Influence Functions.
New York: Wiley.

Hampel, F. R. 1974.
The influence curve and its role in robust estimation.
Journal of the American Statistical Association 69: 383–393.

Newson, R. B. 2014.
Easy-to-use packages for estimating rank and spline parameters.
Presented at the 20th UK Stata User Meeting, 11–12 September, 2014.

Newson, R. 2006a.
Confidence intervals for rank statistics: Percentile slopes, differences, and ratios.
Stata Journal 6: 497-520.
Download from the Stata Journal website.

Newson, R. 2006b.
Confidence intervals for rank statistics: Somers' D and extensions.
Stata Journal 6: 309-334.
Download from the Stata Journal website.

Newson R. 2002.
Parameters behind "nonparametric" statistics: Kendall's tau, Somers' D and median differences.
Stata Journal 2: 45-64.
Download from the Stata Journal website.

Also see

Manual: [D] collapse, [R] centile

Online: collapse, centile

somersd, cendif, scsomersd, sccendif, expgen (if installed)