*! version 5 Nov2021
help dsegdialog:

dseg -- Decomposable Multigroup Segregation Indices

Syntax

Segregation using index index of varlist1 given varlist2

dseg index varlist1 [if] [in] [weight] , given(varlist2) [options]

index description

atkinson, alt_atkinson symmetric Atkinson, A

theil, alt_theil Theil's H (Entropy), H

mutual, n_mutual mutual information, M

diversity, alt_diversity relative diversity, R

Mandatory option
given(varlist2) the units in "groups-given-units" indices; the groups in "units-given-groups" indices

Model
addindex(namelist) additional indices to be calculated

by(varlist) subsamples over which the index is to be calculated

within(varlist[,components]) the partition in the between-within decomposition

missing treats missing values like other values

fast uses contributed command ftools to speed up computing time with big data; see ftools

Bootstrapping & simulation
bootstraps(#[,opt]) performs # bootstrap replications

random(#) computes the index with # simulated samples under the assumption of no-segregation

rseed(#) initial value of the random-number seed

Replacing and saving
clear replaces current data

saving(filename[,opt]) saves index values in Stata data file filename

prefix(name) sets the prefix for names of new index variables

Reporting
nolist do not list index values

format(%fmt) set index display format; default is %9.4f; see format

fweights are allowed; see weight.

Description

The dseg command computes all known multigroup indices of segregation that are additively decomposable into a between and a within term:
the symmetric Atkinson index (A), the Theil's H index (H), the mutual information index (M), and the relative diversity index (R).
For A, H, and R, two versions are available:
One version captures how groups shares differ across organizational units.
This is the "groups-given-units" notion of segregation.
The other version captures how groups are distributed along different organizational units.
This is the "units-given-groups" notion of segregation.
The indices H and R described in Reardon and Firebauch (2002) are "groups-given-units" indices that are weakly unit decomposable.
Alternatively, the "units-given-groups" versions of H (proposed in Mora and Ruiz-Castillo, 2011) and R are weakly group decomposable.
The A index proposed in Frankel and Volij (2011) is a "units-given-groups" index that is also weakly unit decomposable. The "groups-given-units" version is weakly group decomposable.
Index M, proposed by Theil and Finizza (1971), is both a "groups-given-units" and a "groups-given-units" index. Two versions are also available for M: the original index (using natural logarithms) and a normalized version.
The latter uses as base of the logarithm the minimum between the number of groups or the number of units.
This is a normalization of the index in a weak sense as it is the proportion of maximum segregation reached by the original index (see Mora and Ruiz-Castillo 2011).
The original version is both strongly unit and group decomposable.
The normalized version is strong unit decomposable if normalized with the natural logarithm of the number of groups.
It is strong group decomposable if normalized with the natural logarithm of the number of units.

The variables specified in varlist1 identify the value combinations that define the groups in "groups-given-units" indices and the units in "units-given-groups" indices.
The variables specified in varlist2 identify the units in "groups-given-units" indices and the groups in the "units-given-groups" indices.
Hence, the user specifies the index to compute by setting the name of the index and by choosing which variables are included in varlist1 and which variables are included in varlist2.
As the mutual information index is both a "groups-given-units" and a "units-given-groups" index, which variable list is in varlist1 and which is in varlist2 has no effects on the results.

By default, dseg assumes that each observation in the data set represents one individual.
When data is aggregated (i.e., each observation represents several individuals), a variable in the dataset must contain the number of duplicated observations and fweight must be used.

Options

Model

addindex(namelist) is a list of additional indices to be computed.
Alternative indices (alt_atkinson, alt_theil, alt_diversity) refer to the "units-given-groups" versions when varlist2 identifies the units and to the "groups-given-units" versions when varlist2 identifies the groups.

by(varlist) identifes the subsamples over which the index is to be calculated. It is useful for computing the same index and its decomposition for different years, countries, etc.

within(varlist[,opt]) specifies the partition that defines the between-within decomposition.
It admits the components suboption.
For A, H, and R, dseg automatically computes the only posible decomposition available for each of these indices.
Hence, for the "units-given-groups" version of A, varlist identifies the clusters that partition the organizational units defined by varlist1.
In contrast, for the "groups-given-units" version of H and R, varlist identifies the clusters that partition the organizational units defined by varlist2.
Without loss of generality, for M, or its normalized version, varlist specifies the partition of varlist2 that defines the between-within decomposition.
When the components suboption is used, the procedure additionally returns the components of the within term, namely the weights and the local segregation indices in the within term.

missing treats missing values as other values. By default, dseg assumes missing values are instances of random incomplete information.
Hence, all observations with missing value in at least one of varlist1, varlist2, the variables in by(varlist), or the variables in within(varlist), are dropped before the calculations are made.
The missing option reverts this behavior and forces dseg to interpret missing values as categories.

fast uses contributed command ftools to speed up computing time with big data. It requires all variables to be numeric; see ftools.

Bootstrapping & simulation

bootstraps(#[,opt]) sets the number of bootstrap samples.
This option invokes the bsample command to generate a bootstrap sample with replacement with the same number of observations that the original sample.
Bootstrap options are passed through by opt; see bsample.
The bootstraps(#) option cannot be used with weights or simultaneously with random(#).
It results in a new dataset with index values for each of the # bootstrap samples.
In the new dataset, the bootstrap samples are identified with variable bsn, the bootstrap sample number.
The new dataset includes an additional observation with the indices calculated with the original dataset.
This observation is identified with bsn = 0.
The new dataset replaces the current dataset in memory when the clear option is used.
It is saved when the saving(filename,opt) option is used.
If none of these two options are used, dseg stops and displays an error message before doing the bootstrap.

random(#) computes the index with # simulated samples under the assumption of no-segregation.
Each simulated sample is obtained after randomly reshuffling values of varlist1.
Otherwise, random(#) closely follows the behavior of bootstraps(#):
(a) it cannot be used with weighted data or simultaneously with bootstraps(#);
(b) the output is a new dataset that includes index values for all simulated samples;
and (c) the new dataset must replace the current dataset and/or be saved.

rseed(#) sets the seed for the random number generator.

Returning, replacing, and saving

The dseg command internally creates a temporary data set with the results.
If the within(varlist) option is not used, the new dataset has as variables the indices calculated.
If the within(varlist) option is used, the new dataset also includes, for each index calculated, the between and the within term in the decomposition.
If the components suboption is used, the new dataset additionally includes the weights and the local segregation indices whose weighted average is the within term for each index.
If neither the by(varlist) option nor the components suboption are used, the new dataset has only one observation.
If only the by(varlist) option is used, the new dataset has as many observations as the number of value combinations in by(varlist).
If within(varlist,components) is also used, the number of observations in the new dataset equals the number of combinations of the categories defined by the varlists of by(varlist) and within(varlist,components) together.

By default dseg lists the indices and returns matrix r(S) with all of them.
(In addition to the number of observations, the command name, the names of the indices, and the notion of segregation defined by varlist1 and varlist2.)
There are three cases in which matrix r(S) is not returned:
(a) when either bootstraps(#) or random(#) are used;
(b) when the resulting matrix is too large;
(c) in the presence of a string variable in option by(varlist) or if components is used as in within(varlist, components).
In the last two cases (b) and (c), a warning message is displayed.

clear replaces current data in memory with the new dataset.

saving(filename[,opt]) saves the new dataset in filename.
Saving options are passed through by opt; see save.

prefix(name) attaches name in front of the default name for each index, each between/within term, and each local weight in the new dataset.
This option is useful in the presence of conflicting names in the new dataset.
There are four types of variable names in the new dataset:
(a) the indices names;
(b) the variable names in the list of by(varlist) if by(varlist) is used;
(c) the variable names in the list of within(varlist,components) if components is used; and
(d) bsn and ssn if bootstraps(#) or random(#) are used.
For the index versions implicit in the notion of segregation defined by varlist1 and varlist2, the default names are A, H, M, and R.
For the alternative versions of A, H, and R, the names are AltA, AltH, and AltR, respectively.
For the normalized mutual information, the default name is NM.
When the within(varlist) option is used, postfixes _B and _W are added after the default name for the between and the within term, respectively.
If the components suboption is used, postfixes _w and _l are used to name the weights and the local segregation indices.
In case of a conflicting name, dseg stops and issues an error message.

Reporting

nolist omits the display of index values.
By default, dseg lists the values of the index.
Option nolist surpresses the list.
When used together with option clear, a description of the new data set is displayed.

format(%fmt) sets index output display format; default is %9.4f; see format

Remarks

Several papers extensively discuss the properties of the indices
(Frankel and Volij, 2011, Mora and Ruiz-Castillo, 2011, Reardon and Firebaugh, 2002, and Reardon, Yun, and Eitle, 2000).

The mutual information index proposed by Theil and Finizza (1971) and the symmetric Atkinson index proposed by Frankel and Volij (2011) are both characterized in terms of ordinal axioms by Frankel and Volij (2011).
The latter was previously characterized by Hutchens (2004) using an alternative set of axioms for the case when the number of groups equals two.
Reardon and Firebaugh (2002) describe the properties of the "groups-given-units" versions of H and R.
Mora and Ruiz-Castillo (2011) propose the "units-given-groups" version of H and the normalized mutual information, and discuss the decomposability properties of M, and the two versions of H.

All indices have a minimum value of zero.
The mutual information index has no global maximum.
For a fixed number of groups and organizational units, its maximum is the minimum between the logarithm of the number of groups and the logarithm of the number of units.
This justifies using this maximum as the base of the logarithms.
All other indices reach a maximum value of one.

All indices are relative indices of segregation and are invariant to population size.
Only the symmetric Atkinson proposed by Frankel and Volij (2011) is group composition-invariant (i.e., invariant to the groups proportions).
Only the "units-given-groups" version of the Atkinson index is unit composition-invariant (i.e., invariant to the units proportions).

Examples

Setup

. use dseg.dta

The data are obtained from the Alabama subsample of the 2017 Common-Core of Data compiled by the U.S. National Center for Educational Statistics and contain enrollment counts by school and race.
Variables race, white, school, district, and cbsa identify the racial/ehtnic group, a white dummy, the school, the local educational authority, and the CBSA codes of each observation, respectively.
The variable student_count includes the number of students in each cell.

Displays the original mutual information index of race segregation in schools proposed by Theil & Finizza (1971)

. dseg mutual race [fw=student_count], given(school)

Displays both the original and the normalized mutual information indices

. dseg mutual race [fw=student_count], given(school) addindex(n_mutual)

Displays the Theil's H index first used in Theil & Finizza (1971), i.e. the one based on the "groups-given-units" notion of segregation

. dseg theil race [fw=student_count], given(school)

Displays both versions of the Theil's H index

. dseg theil race [fw=student_count], given(school) addindex(alt_theil)

Displays the group decomposition between white-minority supergroups, and within minority students of the relative diversity index based on the "units-given-groups" notion of segregation.
The format display is changed to improve display precision

. dseg diversity school [fw=student_count], given(race) within(white) format(%9.6f)

Displays the decomposition of race segregation in schools into a between- and a within-districts term in all unit-decomposable indices.
Note that the normalized mutual is unit-decomposable in this example because the number of units is larger than the number of groups

. dseg theil race [fw=student_count], given(school) within(district) addindex(mutual n_mutual diversity alt_atkinson)

Displays the relative diversity index presented by Reardon and Firebough (2002) of race segregation in schools for each Alabama CBSA (and an aggregate of non-CBSA areas).
Note that cbsa is a string variable and the results cannot be stored in Stata matrix r(S).

. dseg diversity race [fw=student_count], given(school) by(cbsa)

Displays the mutual information index of race segregation in schools controlling for districts for each Alabama CBSA

. dseg mutual race [fw=student_count], given(school) by(cbsa) within(district)

As before plus it saves (replacing if necessary) dataset Mutual_Alabama2017.dta with the indices, the terms of the decomposition, and the district indices and weights used to compute the within term.
The call further replaces the current data in memory, and suppresses the automatic display of indices.

. dseg mutual race [fw=student_count], given(school) by(cbsa) within(district,components) saving(Mutual_Alabama2017.dta,replace) nolist clear

Saved results

dseg saves the following in r():

Scalars

r(N) number of observations

Macros

r(cmd) command name

r(index) index (indices) name

r(notion) notion of segregation

Unless bootstraps() or random() are used and whenever is possible (i.e. when the results do not include a string variable and the matrix dimension is not too large):

Matrices

r(S) matrix of indices

References

Hutchens, R. (2004). One Measure of Segregation. International Economic Review 45(2): 555–578.

Frankel, D. M. and Volij, O. (2011). Measuring school segregation. Journal of Economic Theory 146(1):1-38.

Mora, R. and Ruiz-Castillo, J. (2011) Entropy-Based Segregation Indices. Sociological Methodology 159–194-41.

Reardon, S. F. and Firebaugh, G. (2002). Measures of Multigroup Segregation.
Sociological Methodology 32:33-67.

Reardon, S.F.; Yun, J.T. and T. M. Eitle (2000). The Changing Structure of School Segregation: Measurement and Evidence of
Multiracial Metropolitan-Area School Segregation, 1989-1995. Demography 37(3):351-364.

Theil, H., and A. J. Finizza (1971). A Note on the Measurement of Racial Integration of
Schools by Means of Informational Concepts. The Journal of Mathematical Sociology 1(2): 187–193.

Author

Ricardo Mora, Department of Economics, Universidad Carlos III Madrid. Email: ricmora@eco.uc3m.es

Acknowledgements

I would like to thank Daniel Guinea-Martín for so many comments to improve the syntax and usefulness of the command.

Also see

duncan, duncan2, hutchens, seg, segregation, ftools if installed.