help for distplot(SJ17-4: gr41_5; SJ10-1: gr41_4; SJ5-3: gr41_3;
SJ3-4: gr41_2; SJ3-2: gr41_1; STB-51: gr41)

Distribution function plots

distplot
varname
[if] [in]
[weight]
[,
over(varname)
by(varname[, sub_options])
{ frequency
|
midpoint
}
missing
reverse[(ge)]
trscale(transformation_syntax)
graph_options
]

distplot
varlist
[if] [in]
[weight]
[,
by(varname[, sub_options])
{
frequency
|
midpoint
}
reverse[(ge)]
trscale(transformation_syntax)
graph_options
]

fweights and aweights may be specified.

Description

distplot produces a plot of the cumulative distribution
function(s) for the variables in varlist. By default this shows the
proportion (or if desired the frequency) of values less than or equal to
each value.

With the reverse option, distplot produces a plot
of the reverse cumulative probabilities (or frequencies), a.k.a., or a
multiple of, the converse or complementary distribution, reliability,
survival or survivor function. This shows the proportion (or if desired
the frequency) of values greater than each value, or (if
reverse(ge) is specified) of values greater than or equal to each
value.

The plot is by default a line plot. Note the possibility of
using recast() to recast the plot as another
twoway type, such as connected, dot,
dropline, scatter, or spike.

Remarks

In principle the cumulative distribution function has values ranging
between 0 and 1. Hence people are sometimes puzzled by graphs from
finite data, which do not stretch over that entire range. The puzzle is
easily explained. To make discussion simple and concrete, forget about
possible ties and weights, and imagine a simple set of data values 1 2 3
4 5.

By default distplot shows probabilities of being less than or
equal to any value, so the cumulative probabilities shown for the
example data run 1/5 ... 5/5 or 0.2 0.4 0.6 0.8 1 and 0 is never
plotted. With the reverse option the cumulative probabilities are
the complements (subtracted from 1) and run 0.8 0.6 0.4 0.2 0. With the
reverse(ge) option they run 1 0.8 0.6 0.4 0.2, which has a
particular advantage that all can be shown on a logarithmic scale. So,
with reverse 1 is never plotted and with reverse(ge) 0 is
never plotted.

Logically there seems scope to plot probability of being less, which
would mean plotting 0 0.2 0.4 0.6 0.8, but that does not seem
conventional and is not currently supported by distplot.

The midpoint option splits probabilities upwards and downwards.
Consider Tukey's (1977, 496) apt summary: "It is a long-used statistical
practice to treat any observed value exactly equal to a cutting value as
if it were half below and half above." The principle goes back at least
to Galton. This option thus yields cumulative probabilities of 0.1 0.3
0.5 0.7 0.9 or their complements in 1, and so treats tails
symmetrically.

The midpoint convention is helpful for showing distribution functions
for which cumulative probabilities are on various transformed scales.
The convention is crucial for quantile plotting in which one axis is
expected quantiles calculated by feeding cumulative probabilities to a
quantile function (inverse cumulative distribution function, if you
insist). Results of quantile functions are often undefined for arguments
of 0 or 1. The normal quantile function is the first example of this
kind met in most people's statistical education. A twist now, which
takes us beyond the scope of distplot, is that there can be
grounds in quantile plotting for preferring variations on (rank - 0.5) /
#values, which is the implied rule here.

For any sufficiently large sample, cumulative probabilities approach 0
and 1 so closely that these nuances are not discernible on graphs, but
the principle is the same.

For other information on what to calculate as cumulative probabilities
- in other contexts often called plotting positions or percentile
ranks - see Cox (2014a) and its references.

Options

by(varname[,sub_options]) specifies
that calculations be carried out separately for each distinct value of a
specified single variable. Results will be shown separately in distinct
panels. See by_option.

Note that although the total
suboption of by() is not prohibited, it does not implement calculation
of the distribution function of all values. A work-around to do that is
outlined in the Examples. See also Cox (2014b).

over(varname) specifies that calculations be
carried out separately for each distinct value of a specified single
variable. Curves will be shown together within the same panel.
over() is only allowed with a single varname.

frequency specifies calculation of cumulative frequency
rather than cumulative probability.

midpoint specifies the use of midpoints of cumulative
probability for each distinct value. This is especially appropriate for
showing distributions of graded (ordinal) data with a relatively small
number of categories. For more explanation and examples, see the
Appendix below.

frequency and midpoint may not be combined.

missing, used only with over(), permits the use of
nonmissing values of varname corresponding to missing values for
the variable named by over(). The default is to ignore such
values.

reverse or reverse(ge) specifies plotting of the
reverse distribution function. Note an important detail (setting on one
side the effects of midpoint): With reverse the highest
observed value is plotted as probability or frequency 0, as no values
are greater than that. That prohibits plotting probability or frequency
on a logarithmic scale. With reverse(ge) the highest observed
value is plotted as a positive probability or frequency, as at least one
value is greater than or equal to that. That allows plotting of
probability or frequency on a logarithmic scale.

trscale() specifies the use of an alternative transformed scale
for cumulative probabilities (or frequencies) on the graph. Stata syntax should
be used with @ as placeholder for untransformed values. To show
probabilities as percents, specify trscale(100 * @). To show probabilities
on an inverse normal scale, specify trscale(invnorm(@)); on a logit scale,
specify trscale(logit(@)); on a folded root scale, specify
trscale(sqrt(@) - sqrt(1 - @)); on a loglog scale, specify
trscale(-log(-log(@))); on a cloglog scale, specify
trscale(cloglog(@)). Tools to make associated labels and ticks
easier are available on SSC: see ssc desc mylabels.

graph_options refers to any options allowed by
graph.
In particular, note that the default y axis title is always
simple, such as "Cumulative probability". If you wish to show more
algebra and use Stata 11 up, know that the text option supports
"less than or equal to" and "greater than or equal to" symbols
through {&le} and {&ge}.

Examples

. sysuse citytemp, clear

* degrees as below need Stata 11 up

. label var tempjan "Mean January temperature ({&degree}F)"

. distplot tempjan, over(region)

. distplot tempjan, by(region)

. distplot tempjan, by(region) reverse

. distplot tempjan, by(region) reverse(ge)

. distplot tempjan tempjul, by(region) legend(order(1 "January" 2 "July")) xtitle("Mean temperature ({&degree}F)")

. count

. local np1 = r(N) + 1

. expand 2

. replace region = 5 in `np1'/L

. label def region 5 "Total", add

. distplot tempjan, by(region)

Author

Nicholas J. Cox, Durham University

n.j.cox@durham.ac.uk

Acknowledgments

Elizabeth Allred, Ronán Conroy and Roger Harbord made
helpful comments during development of earlier versions of this
or related programs. László Kardos found a bug and raised
the detail of by(, total). A question from Maria Sundaram on
Statalist prompted fuller explanation of exactly what is plotted.

Also see

Manual: [R] cumul, [R] diagnostic plots

Online: help for graph, cumul, quantile,
qplot (if installed), mylabels (if installed), stripplot (if installed)

Appendix: the midpoint option and graded data

The focus of this Appendix is conveyed by its title, but the
examples illustrate a variety of additional points of technique.

The cumulative probability P is defined under the midpoint
option as

SUM counts in categories below + (1/2) count in this category

.
SUM counts in all categories

With terminology from Tukey (1977, 496-497), this could be
called a `split fraction below'. It is also a `ridit' as defined by
Bross (1958): see also Fleiss et al. (2003, 198-205) or Flora (1988).
Yet again, it is also the mid-distribution function of Parzen (1993,
3295) and the grade function of Haberman (1996, 240-241). The numerator
is a `split count'. Using this numerator, rather than

SUM counts in categories below

or

SUM counts in categories below + count in this category,

means that more use is made of the information in the data. Either
alternative would always mean that some probabilities are identically 0
or 1, which tells us nothing about the data. In addition, there are
fewer problems in showing the cumulative distribution on any
transformed scale (e.g., logit) for which the transform of 0 or 1 is
not plottable. Using this approach for graded data was suggested by
Cox (2001, 2004).

A plot of the complement of this cumulative probability, 1 - P,
may be obtained through the reverse option.

Further information on working with counted fractions and folded
transformations for probability scales is available in Tukey (1960,
1961, 1977), Atkinson (1985), Cox and Snell (1989) and Emerson (1991).
Some of the possible transformations appear as link functions in
the literature on generalized linear models (e.g., McCullagh and Nelder
1989; Aitkin et al. 1989).

Example 1 Aitkin et al. (1989, 242) reported data
from a survey of student opinion on the Vietnam War taken at the University of
North Carolina in Chapel Hill in May 1967. Students were classified by sex,
year of study and the policy they supported, given choices of

A The US should defeat the power of North Vietnam by widespread bombing
of its industries, ports and harbours and by land invasion.

B The US should follow the present policy in Vietnam.

C The US should de-escalate its military activity, stop bombing North
Vietnam, and intensify its efforts to begin negotiation.

D The US should withdraw its military forces from Vietnam immediately.

(They also report response rates (page 243), averaging 26% for males and 17%
for females.)

Here are the data as code:

. input str1 policy float freq long(female year)

. "A" 175 0 1

. "B" 116 0 1

. "C" 131 0 1

. "D" 17 0 1

. "A" 160 0 2

. "B" 126 0 2

. "C" 135 0 2

. "D" 21 0 2

. "A" 132 0 3

. "B" 120 0 3

. "C" 154 0 3

. "D" 29 0 3

. "A" 145 0 4

. "B" 95 0 4

. "C" 185 0 4

. "D" 44 0 4

. "A" 118 0 5

. "B" 176 0 5

. "C" 345 0 5

. "D" 141 0 5

. "A" 13 1 1

. "B" 19 1 1

. "C" 40 1 1

. "D" 5 1 1

. "A" 5 1 2

. "B" 9 1 2

. "C" 33 1 2

. "D" 3 1 2

. "A" 22 1 3

. "B" 29 1 3

. "C" 110 1 3

. "D" 6 1 3

. "A" 12 1 4

. "B" 21 1 4

. "C" 58 1 4

. "D" 10 1 4

. "A" 19 1 5

. "B" 27 1 5

. "C" 128 1 5

. "D" 13 1 5

. end

. label values female female

. label def female 0 "male" 1 "female"

. label values year year

. label def year 5 "Graduate"

distplot needs a numeric outcome variable. Here are some sample plots.

. encode policy, gen(Preference)

. local opts recast(connected) mid over(year) xla(, valuelabel)

. distplot Pref [w=freq] if !female, `opts'

. distplot Pref [w=freq] if female, `opts'

. distplot Pref [w=freq], `opts' by(female, note(""))

Example 2 Fienberg (1980, 54-55) reports data from
Duncan, Schuman and Duncan (1973) from 1959 and 1971 surveys of a large
American city asking "Are the radio and TV networks doing a good job, just a
fair job, or a poor job?". Here are the data as code:

. clear

. input float(year freq) long(Opinion race)

. 1959 81 3 2

. 1959 23 2 2

. 1959 4 1 2

. 1959 325 3 1

. 1959 253 2 1

. 1959 54 1 1

. 1971 224 3 2

. 1971 144 2 2

. 1971 24 1 2

. 1971 600 3 1

. 1971 636 2 1

. 1971 158 1 1

. end

. label values Opinion Opinion

. label def Opinion 1 "Poor" 2 "Fair" 3 "Good"

. label values race race

. label def race 1 "White" 2 "Black"

* for groups, see Cox (2017)

. groups race year Opinion [w=freq] , percent(race year) sepby(race year)

. mylabels 2 5 10(10)80, myscale(logit(@/100)) local(myla)

. distplot Opinion [w=freq], recast(connected) mid over(year) by(race, note("")) trscale(logit(@)) xla(1/3, valuelabel) yla(`myla', ang(h)) ytitle(Percent) xsc(r(0.8 3.2))

This shows a clear shift of opinion towards Poor from 1959 to 1971,
and a narrowing gap between Black and White.

Example 3 Clogg and Shihadeh (1994, 156) give data from
the 1988 General Social Survey on answers to the question "When a marriage is
troubled and unhappy, do you think it is generally better for the children if
the couple stays together or gets divorced?". Responses were "much better to
divorce", "better to divorce", "don't know", "worse to divorce" and "much worse
to divorce". Here are the data as code, with "don't know" treated as in the middle of the scale:

. clear

. input float freq long(female Opinion)

. 84 0 5

. 205 0 4

. 135 0 3

. 121 0 2

. 56 0 1

. 154 1 5

. 330 1 4

. 178 1 3

. 72 1 2

. 49 1 1

. end

. label values female female

. label def female 0 "male" 1 "female"

. label values Opinion Opinion

. label def Opinion 1 "much worse" 2 "worse" 3 "don't know" 4 "better" 5 "much better"

However, it is not clear that the "don't know"s do
belong in the middle of the
scale. The point can be explored by graphs with and without those
values. Either way, there is a distinct separation between males and females,
and a logit scale gives a more nearly linear pattern.

. local opts recast(connected) mid over(female) xla(, valuelabel)

. local opts `opts' legend(col(1) pos(5) ring(0))

. distplot Opinion [w=freq], `opts' xsc(r(0.7,5.3) titlegap(*5))

. mylabels 2 5 10(10)90 95 98, myscale(logit(@/100)) local(myla)

. distplot Opinion [w=freq], `opts' xsc(r(0.7,5.3)) trscale(logit(@)) yla(`myla', ang(h)) ytitle(Percent)

. egen Opinion2 = group(Opinion) if Opinion != 3, label

. label var Opinion2 "Opinion"

. distplot Opinion2 [w=freq], `opts' xsc(r(0.7, 4.3)) trscale(logit(@)) yla(`myla', ang(h)) ytitle(Percent)

Example 4 Knoke and Burke (1980, 68) gave data from the
1972 General Social Survey on church attendance. Here are the data as code:

. clear

. input float freq long(group Attendance)

. 322 1 1

. 122 1 2

. 141 1 3

. 250 2 1

. 152 2 2

. 194 2 3

. 88 3 1

. 45 3 2

. 106 3 3

. 28 4 1

. 24 4 2

. 119 4 3

. end

. label values group group

. label def group 1 "young non-Catholic" 2 "old non-Catholic"

. label def group 3 "young Catholic" 4 "old Catholic", modify

. label values Attendance Attend

. label def Attend 1 "low" 2 "medium" 3 "high"

The reverse option ensures that higher attendance groups plot
higher on the graph. There are clear age and denomination effects and an
indication of an interaction between the two.

. mylabels 0.05 0.1(0.2)0.9 0.95, myscale(logit(@)) local(myla)

. distplot Attend [w=freq], recast(connected) mid over(group) trscale(logit(@)) reverse xla(1/3, valuelabel) xsc(titlegap(*5)) yla(`myla', ang(h))

Example 5 Box, Hunter and Hunter (1978, 145-149)
gave data on five hospitals on the degree of restoration (no improvement,
partial functional restoration, complete functional restoration) of certain
joints impaired by disease effected by a certain surgical procedure. (It is
not clear whether these data are real.) Hospital E is a referral hospital.
Box et al. carry out chi-square analyses, focusing on the difference between
Hospital E and the others. Here are the data as code:

. clear

. input str1 hospital float freq long restore

. "A" 13 1

. "B" 5 1

. "C" 8 1

. "D" 21 1

. "E" 43 1

. "A" 18 2

. "B" 10 2

. "C" 36 2

. "D" 56 2

. "E" 29 2

. "A" 16 3

. "B" 16 3

. "C" 35 3

. "D" 51 3

. "E" 10 3

. end

. label values restore restore

. label def restore 1 "none" 2 "partial" 3 "complete"

. label var restore "How far restored?"

. mylabels 5 10(20)90 5, myscale(logit(@/100)) local(myla)

. distplot restore [w=freq], recast(connected) mid over(hospital) trsc(logit(@)) xla(1/3, valuelabel) xsc(r(0.9,3.1) titlegap(*5)) yla(`myla', ang(h)) ytitle(Percent)

References

Aitkin, M., D. Anderson, B. Francis, and J. Hinde. 1989.
Statistical Modelling in GLIM. Oxford: Oxford University Press.

Atkinson, A. C. 1985. Plots, Transformations, and Regression.
Oxford: Oxford University Press.

Box, G. E. P., W. G. Hunter, and J. S. Hunter. 1978.
Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building. New York: Wiley.

Bross, I. D. J. 1958. How to use ridit analysis. Biometrics
14: 38-58.

Clogg, C. C. and E. Shihadeh. 1994.
Statistical Models for Ordinal Variables.
Thousand Oaks, CA: Sage.

Cox, D. R. and E. J. Snell. 1989. Analysis of Binary Data.
London: Chapman & Hall.

Cox, N. J. 2001. Plotting graded data: a Tukey-ish approach.
Presentation to UK Stata users meeting, Royal Statistical Society,
London, 14-15 May.
http://www.stata.com/support/meeting/7uk/cox1.pdf

Cox, N. J. 2004. Speaking Stata: Graphing categorical and
compositional data. Stata Journal 4(2): 190-215.

Cox, N. J. 2014a.
How can I calculate percentile ranks? How can I calculate plotting positions?
/https://www.stata.com/support/faqs/statistics/percentile-ranks-and-plotting-positions/

Cox, N. J. 2014b.
Stata tip 119: Expanding datasets for graphical ends.
Stata Journal 14: 230-235.

Cox, N. J. 2017. Speaking Stata: Tables as lists: The groups
command. Stata Journal 17(3): !!!-!!!.

Duncan, O. D., H. Schuman, and B. Duncan. 1973.
Social Change in a Metropolitan Community.
New York: Russell Sage Foundation.

Emerson, J. D. 1991. Introduction to transformation.
In Fundamentals of Exploratory Analysis of Variance,
ed. D. C. Hoaglin, F. Mosteller, and J. W. Tukey, 365-400.
New York: Wiley.

Fienberg, S. E. 1980.
The Analysis of Cross-Classified Categorical Data.
Cambridge, MA: MIT Press.

Fleiss, J. L., B. Levin, and M. C. Paik. 2003.
Statistical Methods for Rates and Proportions.
Hoboken, NJ: Wiley.

Flora, J. D. 1988. Ridit analysis. In
Encyclopedia of Statistical Sciences,
ed. S. Kotz and N. L. Johnson, (8) 136-139.
New York: Wiley.

Haberman, S. J. 1996.
Advanced Statistics Volume I: Description of Populations.
New York: Springer.

Knoke, D. and P. J. Burke. 1980. Log-linear Models.
Beverly Hills, CA: Sage.

McCullagh, P. and J. A. Nelder. 1989. Generalized Linear Models.
London: Chapman & Hall.

Parzen, E. 1993. Change PP plot and continuous sample quantile
function. Communications in Statistics -Theory and Methods
22: 3287-3304.

Tukey, J. W. 1960. The practical relationship between the common
transformations of percentages or fractions and of amounts. Reprinted in
Mallows, C.L. (ed.) 1990.
The Collected Works of John W. Tukey. Volume VI: More Mathematical.
Pacific Grove, CA: Wadsworth & Brooks-Cole, 211-219.

Tukey, J.W. 1961. Data analysis and behavioral science or learning to
bear the quantitative man's burden by shunning badmandments. Reprinted in
Jones, L.V. (ed.) 1986. The Collected Works of John W. Tukey. Volume III: Philosophy and Principles of Data Analysis: 1949-1964. Monterey, CA:
Wadsworth & Brooks-Cole, 187-389.

Tukey, J. W. 1977. Exploratory Data Analysis.
Reading, MA: Addison-Wesley.