help for fractileplot

Smoothing with respect to distribution function predictors

fractileplot
yvar xvarlist
[if] [in]
[,
a(#)
bwidth(#)
degree(#)
kernel(kernel_name)

cycles(#)
log

combine(combine_options)
draw(numlist)
nograph
nopts
omit(numlist)
scatter(scatter_options)
line_options

generate(stub)
predict(newvar)
replace]

Description

fractileplot computes smooths of yvar on all predictors in
xvarlist simultaneously; that is, each smooth is adjusted for the others.
Each predictor x_j is treated on the scale of its distribution function,
F(x_j), estimated for a sample of n as (rank of x_j - a) / (n - 2a + 1). a
defaults to 0.5. Smoothing is with lpoly.
Fitted values may be saved in new variables with names
beginning with stub, as specified in the generate() option.

By default, for each xvar in xvarlist adjusted
values of yvar and the smooth for F(xvar) are plotted against
F(xvar). See Remarks for more details.

Options

Smoothing

a(#) specifies a in the formula for F. The default
is a = 0.5, giving (i - 0.5) / n. Other choices include a = 0, giving i / (n +
1), and a = 1/3, giving (i - 1/3) / (n + 1/3). More discussion on this
is available at http://www.stata.com/support/faqs/stat/pcrank.html.

bwidth(#) specifies the halfwidth of the kernel.
The default is 0.2.
Note that each distribution function has values in (0, 1). The choice
of default has no more sanction than that and some experience that it
works quite well.

degree(#) specifies the degree of the polynomial to be
used in the smoothing. 0 is the default, meaning local mean smoothing.

kernel() specifies the kernel. biweight is the default.

Fitting

cycles(#) sets the number of cycles. The default is
cycles(3).

log displays the squared correlation coefficient between the
overall fitted values and yvar at each cycle for monitoring convergence.
This option is provided mainly for pedagogic interest.

Graphics

combine(combine_options) specifies any of the
options allowed by the graph combine command. Useful examples are
combine(ycommon) and combine(saving(graphname)).

draw(numlist) specifies that smooths for a subset of
the variables in xvarlist be plotted. The elements of numlist are
indexes determined by the order of the variables in xvarlist. For
example, fractileplot y x1 x2 x3, draw(2 3) would plot smooths only for
F(x2) and F(x3). By default results for all variables in
varlist are plotted. draw() takes precedence over omit() in
the sense that results for variables included (by index) in numlist are
plotted, even if they are excluded by omit(). See also omit().

nograph suppresses the graph.

omit(numlist) specifies that smooths for a subset of
the variables in xvarlist not be plotted. The elements of numlist are
indexes determined by the order of the variables in varlist. For example,
fractileplot y x1 x2 x3, omit(3) would plot smooths only for F(x1) and
F(x2). By default results for no variables in varlist are omitted.
draw() takes precedence over omit(). See also draw().

nopts suppresses the points in the plots. Only the lines
representing the smooths are drawn.

scatter(scatter_options) specifies any of the options
allowed by the scatter command. These should be specified to control
the rendering of the data points. The default includes msymbol(oh), or
msymbol(p) with over 299 observations.

line_options are any of the options allowed with line.
These should be specified to control the rendering of the smoothed lines or the
overall graph.

Results

generate(stub) specifies that fitted values for each
member of xvarlist be saved in new variables with names beginning with
stub.

predict(newvar) specifies that the predicted values
be saved in new variable newvar.

replace allows variables specified by any of the
generate() and predict() options to be replaced if they already
exist.

Remarks

The intent of this command is exploratory. It may help in signalling
dependence structure, or its lack, given an outcome and a bundle of (potential) predictors.

Smoothing with respect to distribution functions has various
elementary attractions. An F scale provides a common scale for variables with
different level and spread and even different units. Subject to the occurrence
of ties, values are equally spaced on the F scale and so in good condition for
smoothing. This can be especially useful when predictors are highly skewed. F
is invariant under strictly increasing transformations, so that for example
F(log x) is identical to F(x) so long as x > 0. This can be useful when it is
not clear whether predictors should be transformed.

At users' discretion, the predictors may include binary, ordinal or
even nominal predictors. For example, a binary predictor will produce two
clusters of points spaced 0.5 apart on the corresponding plot.

Sen (2005) gives a useful account of kernel smoothing of
responses with respect to distribution functions of predictors. The canonical
reference is Mahalanobis (1960), which introduced the term "fractile graphical
analysis". Mahalanobis plotted means of one variable for bins defined by
selected fractiles of the other variable. Binning and averaging now appear
arbitrary and awkward, and some kind of kernel-based smoothing is more
appealing. The approach in fractileplot is based on methodology for
generalised additive models (Hastie and Tibshirani 1990).

The idea of plotting the values of one or more variables versus
the rank of another can be found in Wallace (1889), Lock (1906) and
Fisher (1925). See Edwards (2013) for commentary.

For one independent invention, see Van Kerm (2006) on income mobility
profiles.

Terminology is a little problematic here. Terms such as "fractile graph" (Sen
2005) and "fractile plot" (Nordhaus 2006) persist in recent literature for
modern versions of Mahalanobis' plots, even though neither ordinate nor
abscissa in the resulting graphs is a fractile. The term "fractile" was
introduced to the English literature by Hald (1952) with the sense of
"quantile", but it has never supplanted "quantile" and is sometimes misunderstood
to mean fraction or cumulative probability or plotting position (e.g. Nordhaus
2006). Hald used "fractile diagram" as a name for plots of observed quantiles against reference
(e.g. normal) distributions. In Stata
terms, many of his examples are equivalent to qnorm with axes reversed.
This usage also continues in recent literature (e.g. Blæsild and
Granfeldt 2003).

An R-square (squared correlation coefficient) is provided as a
goodness of fit indicator. However, this R-square can typically be increased simply
by just smoothing less, which is often likely to be unhelpful. As the resulting
predictions come closer to interpolating the data, R-square will approach 1,
but scientific usefulness and the possibility of insight will usually diminish.

Note that you do not need the machinery here to do this for just
one predictor. The following is a basic recipe:

. gen touse = (y < .) & (x < .)

. egen abscissa = rank(x) if touse

. count if touse

. replace abscissa = (abscissa - 0.5) / r(N)

. lpoly y abscissa, xti("fraction of data")

Suppose that there are p >= 1 predictors. fractileplot estimates the
smooths f_1,...,f_p by using a backfitting algorithm and a local
polynomial smoother S[y|F(x_j)] for each predictor, as follows:

1. Initialize: alpha = mean(y), f_1,...,f_p
estimated by multiple linear regression.

2. Cycle: j = 1,...,p, 1,...,p, ...

f_j = S[y - alpha - sum_ f_i|F(x_j)]

3. Continue for cycles() rounds.

No convergence criterion is applied. In practice, three cycles are
usually more than sufficient to get results adequate for exploratory work.

The smooths are adjusted so that the mean of each equals the mean of yvar.

The points in the plots provided by fractileplot
depict y - sum_ f_i|F(x_j), i.e., the partial residuals plus alpha.

Examples

. sysuse auto, clear

. fractileplot mpg weight displ length

. fractileplot mpg weight displ length, bwidth(0.3)

. fractileplot mpg weight displ length, degree(1) bwidth(0.4)

. fractileplot mpg weight displ length, generate(S) nograph

. fractileplot mpg weight displ length, omit(2) combine(saving(graph1))

For comparison, bivariate smooths may be compared like this:

. foreach v in weight displ length {

. fractileplot mpg `v', combine(saving(fl_`v'))

. }

. graph combine "fl_weight" "fl_displ" "fl_length"

Author

Nicholas J. Cox

Durham University

n.j.cox@durham.ac.uk

Acknowledgements

The main features of the implementation here depend on the work of
Patrick Royston, as reported by Royston and Cox (2005). Thanks to Philippe
Van Kerm for telling me about his work.

References

Blæsild, P. and Granfeldt, J. 2003.
Statistics with Applications in Geology and Biology.
Boca Raton, FL: Chapman & Hall/CRC.

Edwards, A.W.F. 2013. Robert Heath Lock and his textbook of genetics, 1906.
Genetics 194: 529537.
https://doi.org/10.1534/genetics.113.151266

Fisher, R.A. 1925.
Statistical Methods for Research Workers.
Edinburgh: Oliver and Boyd.

Hald, A. 1952.
Statistical Theory with Engineering Applications.
New York: John Wiley.

Hastie, T. and Tibshirani, R. 1990.
Generalized Additive Models.
London: Chapman and Hall.

Lock, R.H. 1906.
Recent Progress in the Study of Variation, Heredity, and Evolution.
London: John Murray.

Mahalanobis, P.C. 1960. A method of fractile graphical analysis.
Econometrica 28: 325-351. Reprinted 1961.
Sankhya Series A 23: 41-64.

Nordhaus, W.D. 2006. Geography and macroeconomics: new data and new
findings. Proceedings, National Academy of Sciences 103(10): 3510-3517.

Royston, P. and Cox, N.J. 2005.
A multivariable scatterplot smoother.
Stata Journal 5(3): 405-412.

Sen, B. 2005. Estimation and comparison of fractile graphs using kernel
smoothing techniques. Sankhya 67: 305-334.
https://www.jstor.org/stable/pdf/25053435.pdf

Van Kerm, P. 2006. Comparisons of income mobility profiles. IRISS
Working Paper 2006-03, CEPS/INSTEAD.
http://ideas.repec.org/p/irs/iriswp/2006-03.html

Wallace, A.R. 1889.
Darwinism: An Exposition of the Theory of Natural Selection with Some of its Applications.
London: Macmillan.

Note: Sankhya should carry a bar accent on
its final "a".

Also see

Online: lpoly