*! version 1.3.1 21January2015 Dirk Enzmann
help divcat

Title

divcat -- calculates five measures of diversity for multiple categories: Generalized
variance (GV), entropy (H), its normalized counterparts (NGV, NH), and polarization (RQ).

Syntax

divcat varname [if] [in] [weight] [, options ]

options

tableout display a frequency table of varname

base(#) use the logarithm to base # when calculating entropy (H) (default: 2)

nolabel omit labels of subgroups specified using the by prefix

gv show the generalized variance (GV) and its normalized counterpart (NGV) in a separate table (default: show
all diversity measures in a common table)

entropy show the entropy (H) and its normalized counterpart (NH) in a separate table (default: show
all diversity measures in a common table)

rq show the polarization measure (RQ) in a separate table (default: show
all diversity measures in a common table)

nodetail omit common table of all diversity measures

gen_gv(newvar) generate a new variable newvar containing the generalized variance (GV)

gen_ngv(newvar) generate a new variable newvar containing the normalized generalized variance (NGV)

gen_h(newvar) generate a new variable newvar containing the entropy (H)

gen_nh(newvar) generate a new variable newvar containing the normalized entropy (NH)

gen_rq(newvar) generate a new variable newvar containing the polarization measure (RQ)

replace replace the contents of newvar if newvar exists already

by is allowed (see by);

aweights, fweights, and iweights are allowed (see weight).

Description

divcat calculates five measures of diversity of a categorical
variable (i.e. for multiple categories): generalized variance (GV), entropy
(H), its normalized counterparts (NGV and NH, resp.), and polarization
(RQ). The formulas are given in Budescu and Budescu (2012) (GV, NGV, H, and
NH) and in Montalvo & Reynal-Querol (2002, 2008) (RQ). divcat allows to
generate new variables containing these measures, which is especially useful
when calculating diversity measures separately for subgroups (specified using
the by prefix).

There is a large variety of diversity measures. Related concepts are
variance, heterogeneity, inequality, entropy, or concentration. Depending
on the field of study, the same measures are known under different
names:

- The GV is also known as the Blau Index (Blau, 1977) or the Hirschman-Herfindahl
Index (HHI) (Hirschman, 1945; Herfindahl, 1950), although some equate
the HHI with the Simpson Index (SI) (Simpson, 1949), whereas GV is actually
1-SI (compare -ineq-: click here). The
GV can be interpreted as the probability that two randomly paired members of
a population belong to two different subgroups.

- The NGV is the normalized GV bounded by 0 and 1. Another name for the NGV is
the Index of Qualitative Variation (IQV) (Mueller, Schuessler, & Costner, 1970). The
index can be interpreted as the proportion of the observed variation to the
maximum possible variation. The normalizing transforms GV to a relative measure allowing
comparisons with diversity measures from studies with a different number of categories because
its size does not depend on the number of categories (as in the case of GV).

- The entropy measure H shares many properties of GV (the abbreviation actually
stands for the Greek letter Eta); an attractive property is its additivity (see Budescu &
Budescu, 2012). Early on, it has been described by Shannon (1948); its formulas differ as to
the base of the logarithm used: The Shannon formula uses the logarithm to base e (see
also -ineq-: click here), whereas
Budescu and Budescu (2012) are using the logarithm to base 2 (which is the default
of divcat). In the case of two groups, when using the logarithm to the base 2, H is
equal to its normalized counterpart NH.

- The NH is the normalized H bounded by 0 and 1. As in the case of GV and its
normalized counterpart NGV, NH is a relative measure: its size no longer depends on
the number of categories (as in the case of H).

- RQ differs from the previous measures as it is a polarization
measure for discrete variables (see Montalvo & Reynal-Querol, 2002, 2008). As do the
normalized measures, it is bounded between 0 and 1. However, in contrast to the other
measures of diversity that reach a maximum if the cases are distributed equally across all categories (or
if all groups are of the same size), RQ reaches a maximum if there are two (large) groups of equal size (and
all other groups are small). This makes RQ attractive for the study of social conflicts.

A helpful discussion of the properties of the "fractionalization" measures GV and H as well as
its normalized counterparts can be found in Budescu and Budescu (2012).

Options

Main

tableout displays a frequency table of the categorical variable varname
specified with divcat. If subgroups are specified using the by prefix, a frequency
table for each subgroup will be produced.

base(#) sets the base of the logarithm used when calculating entropy (H) (default: 2). Possible
alternatives are base 10 or the natural logarithm (base e). Note that the latter
can be specified using the option base(e).

nolabel omits labels of subgroups specified using the by prefix. Note that the maximum
width of row labels of the results table is 32 characters. nolabel can help to make the values
separating the subgroups visible in spite of this space restriction.

gv shows the generalized variance (GV) and its normalized counterpart (NGV) in a separate
table (default: show all diversity measures in a common table)

entropy shows the entropy (H) and its normalized counterpart (NH) in a separate
table (default: show all diversity measures in a common table)

rq shows the polarization measure (RQ) in a separate table (default: show all diversity
measures in a common table)

nodetail suppresses the common table of all diversity measures (default: show all diversity
measures in a common table)

gen_gv(newvar) generates a new variable newvar containing the generalized variance (GV)

gen_ngv(newvar) generates a new variable newvar containing the normalized generalized
variance (NGV)

gen_h(newvar) generates a new variable newvar containing the entropy (H)

gen_nh(newvar) generates a new variable newvar containing the normalized entropy (NH)

gen_rq(newvar) generates a new variable newvar containing the polarization measure (RQ)

replace replaces the variable specified by gen_gv(), gen_ngv(), gen_h(),
gen_nh(), or gen_rq() if newvar exists already.

Examples

Example 1 shows how to calculate the diversity of "rep78" over the subgroups of "foreign", and
how to save GV into the new variable "gv" (to replicate example 1, copy and paste the two command lines
into Stata's command window):

sysuse auto, clear
bys foreign: divcat rep78, gen_gv(gv)

Examples 2 and 3 demonstrate that the "fractionalization" measures (GV to NH) reach a maximum
if all cases are distributed equally across all categories of "cat" (first set of input data), whereas the
polarization measure (RQ) moves towards a maximum if the majority of cases is distributed equally
across only two categories of "cat" (second set of input data) (to replicate the examples, copy and paste the
command lines into Stata's command window):

clear
input cat cases
1 33
2 33
3 34
end
divcat cat [fw = cases], t base(e)

clear
input cat cases
1 48
2 48
3 4
end
divcat cat [fw = cases], t base(e)

Saved Results

divcat saves the following in r():

Scalars

r(N_total) total number of cases

r(bygroups) number of groups defined by the variables specified with prefix by

r(categs) number of categories of varname (of last by group)

r(N) number of cases (of last by group)

r(GV) generalized variance GV (of last by group)

r(NGV) normalized generalized variance NGV (of last by group)

r(H) entropy H (of last by group)

r(NH) normalized entropy NH (of last by group)

r(RQ) polarization measure RQ (of last by group)

Macros

r(base) base of logarithm used to calculate the entropy H

r(by) group variables specified using the by prefix (if used)

r(wgt) weights (if used)

Matrices

r(div) matrix of diversity measures (over by groups)

References

Blau, P. M. (1977). Inequality and Heterogeneity. New York:
Free Press.

Budescu, D. V. & Budescu,
M. (2012). How to measure diversity when you must.
Psychological Methods, 17, 215-227.

Herfindahl, O. C. (1950). Concentration in the U.S. Steel Industry (unpublished
doctoral dissertation). New York, NY: Columbia University.

Hirschman, A. O. (1945). National Power and the Structure of Foreign Trade. Berkeley,
CA: University of California Press.

Montalvo, J. G. & Reynal-Querol,
M. (2002). Why ethnic fractionalization? Polarization, ethnic conflict, and growth
(mimeo). [URL: https://ideas.repec.org/p/upf/upfgen/660.html].

Montalvo, J. G. & Reynal-Querol, M. (2008). Discrete polarization with an application to the determinants of genocides. The Economic Journal,
118, 1835-1865.

Mueller, J. H., Schuessler, H. L., & Costner, H. L. (1970). Statistical Reasoning in Sociology. Boston: Houghton
Mifflin.

Shannon, C. (1948). A mathematical theory of communications. Bell System Technical Journal, 27,
397-423, 623-656.

Simpson, E. H. (1949). Measurement of diversity. Nature, 163, 688.

Author

Dirk Enzmann

Institute of Criminal Sciences, Hamburg

email: mailto:dirk.enzmann@uni-hamburg.de