help missings (SJ23-2: dm0085_3)

Title

missings -- Various utilities for managing missing values

Syntax

missings report [varlist] [if] [in] [,
numeric string sysmiss
observations minimum(#) percent format(format)
identify(varlist)
sort(specification) show(#) list_options]

missings breakdown [varlist] [if] [in] [,
numeric string sysmiss minimum(#) sort(specification) show(#)
list_options]

missings list [varlist] [if] [in]
[,
numeric string sysmiss
minimum(#)
identify(varlist)
list_options]

missings table [varlist] [if] [in]
[,
numeric string sysmiss
minimum(#)
identify(varlist)
tabulate_options]

missings tag [varlist] [if] [in], generate(newvar)
[numeric string sysmiss]

missings dropvars [varlist]
[,
numeric string sysmiss
force]

missings dropobs [varlist] [if] [in]
[,
numeric string sysmiss
force]

by: may be used with any of missings report, breakdown, missings list,
or missings table. See [D] by.

Description

missings is a set of utility commands for managing variables that
may have missing values. By default, "missing" means numeric missing
(that is, the system missing value . or one of the extended missing
values .a to .z) for numeric variables and empty or "" for
string variables. See [U] 12.2.1 Missing values for further
information.

If varlist is not specified, it is interpreted by default as all
variables.

missings report issues a report on the number of missing values in
varlist. By default, counts of missings are given by variables;
optionally, counts are given by observations.

missing breakdown issues a report on different missing values
in varlist, that is the numbers present of (1) empty strings "" if string
variables are included and (2) system missing and extended missing
values if numeric variables are included. This subcommand is most obviously
useful as a check on the presence of extended missing values for numeric
variables.

missings list lists observations with missing values in varlist.

missings table tabulates observations by the number of missing
values in varlist.

missings tag generates a variable containing the number of missing
values in each observation in varlist.

missings dropvars drops any variables in varlist that are
missing on all values.

missings dropobs drops any observations that are missing on all
values in varlist.

Options

numeric (all subcommands) indicates to include numeric
variables only. If any string variables are named explicitly, such
variables will be ignored.

string (all subcommands) indicates to include string variables
only. If any numeric variables are named explicitly, such variables will
be ignored.

sysmiss (all subcommands) indicates to include system missing
. only. This option has no effect with string variables, for which
missing is deemed to be the empty string "", regardless.

observations (with missings report) indicates counting of
missing values by observations, not the default of counting by
variables.

minimum(#) (with missings report; missings breakdown; missings list; and
missings table) specifies the minimum number of missings to be
shown explicitly. With missings table, the default is minimum(0);
otherwise, it is minimum(1).

percent (with missings report) reports percents missing as well as
counts. Percents are calculated relative to the number of observations or
variables specified.

format(format) (with missings report) specifies a display
format for percents. The default is format(%5.2f). This option has no
effect unless percent is also specified.

identify(varlist) or identify(varname)
(with missings report, observations; missings list; and missings table)
insists on showing varlist or varname in the display of results.
This can be especially useful to show (for example) identifier variables, which
typically will not be missing, or key categorical variables such as education
or gender. With missings report, observations
and missings list, varlist is included in the list results.
With missings table, varname is used to produce a two-way table in
contrast to a one-way table; two or more variables may not be specified.

sort(specification) (with missings report and missings breakdown) specifies output should be
sorted as specified. The specification must include either missings
or alpha or any abbreviation of either keyword. missings means
sorting by number of missing values. alpha means sorting by variable
name. The specification may include descending to indicate sorting in
descending order, for example, that variables with the most missing values will
be shown first.

Note: to maintain compatibility with previous versions, the bare
option sort is also supported, although not indicated in the syntax
diagram. sort is equivalent to sort(missings descending).

show(#) (with missings report and missings breakdown) specifies that at most the first #
variables be shown. This option has no effect unless sorting is also specified
and is most obviously useful whenever the sort is on the number of missing
values.

list_options (with missings report, missings breakdown and missings list) are options
listed in [D] list that may be specified when list is used to
show results.

tabulate_options (with missings table) are options listed in
[R] tabulate oneway or
[R] tabulate twoway
that may be specified when tabulate is used to show results.

generate(newvar) (with missings tag) specifies the name of a new
variable. generate() is required.

force (with missings dropvars and missings dropobs) signals
that the dataset in memory is being changed and is a required
option when data are being dropped and the dataset in memory has not been
saved as such.

Remarks

missings is intended to unite and supersede the main ideas of
nmissing (Cox 1999, 2001a, 2003, 2005) and
dropmiss (Cox 2001b, 2008).

Creating entirely empty observations (rows) and variables (columns)
is a habit of many spreadsheet users, but neither is helpful in Stata
datasets. The subcommands dropobs and dropvars should
help users clean up. Conversely, there is no explicit support here for
dropping observations or variables with some missing and some
nonmissing values. Users so minded will find other subcommands of use
as an intermediate step, but multiple imputation might be a better way
forward.

Examples

. webuse nlswork, clear

. missings report

. missings report, minimum(1000)

. missings report, sort(miss desc)

. missings report, sort(miss desc) show(10)

. missings list, minimum(5)

. missings list, minimum(5) id(race)

. missings table

. bysort race: missings table

. missings table, identify(race)

. missings tag, generate(nmissing)

. generate frog = .

. generate toad = .a

. generate newt = ""

. missings breakdown, sort(missings descending)

. missings breakdown, numeric sort(missings descending)

. missings dropvars frog toad newt, force sysmiss

. missings dropvars toad, force sysmiss

. set obs 30000

. missings dropobs, force

Stored results

missings stores the following in r():

Scalars

r(N) number of observations checked (all)

r(n_dropped) number of observations dropped (missings dropobs)

Macros

r(varlist) varlist used (missings report, missings breakdown,
missings list, missings table, and missings dropvars)

Author

Nicholas J. Cox, Durham University, Durham, UK

n.j.cox@durham.ac.uk

Acknowledgments

Jeroen Weesie, Eric Uslaner, and Estie Sid Hudes contributed to the
earlier development of nmissing and dropmiss.

A question from Fahim Ahmad on Statalist prompted the addition of sorting
and show(#) options to missings report. A question from
Martyn Sherriff on Statalist prompted the addition of the identify()
option. Discussion with Richard Goldstein led to clarification of the
scope of identify(). A question from Jørgen Carling on Statalist
led to adding missings breakdown.

References

Cox, N. J. 1999.
dm67: Numbers of missing and present values.
Stata Technical Bulletin 49: 7-8.
Reprinted in Stata Technical Bulletin Reprints, vol. 9, pp. 26-27.
College Station, TX: Stata Press.

------. 2001a.
dm67.1: Enhancements to numbers of missing and present values.
Stata Technical Bulletin 60: 2-3.
Reprinted in Stata Technical Bulletin Reprints, vol. 10, pp. 7-9.
College Station, TX: Stata Press.

------. 2001b.
dm89: Dropping variables or observations with missing values.
Stata Technical Bulletin 60: 7-8.
Reprinted in Stata Technical Bulletin Reprints, vol. 10, pp. 44-46.
College Station, TX: Stata Press.

------. 2003. Software Updates:
dm67_2: Numbers of missing and present values.
Stata Journal 3: 449.

------. 2005.
Software Updates: dm67_3: Numbers of missing and present values.
Stata Journal 5: 607.

------. 2008. Software Updates:
dm89_1: Dropping variables or observations with missing values.
Stata Journal 8: 594.

Also see

Article: Stata Journal, volume 17, number 3: dm0085_1,

Stata Journal, volume 15, number 4: dm0085

Help: [U] 12.2.1 Missing values,
[D] codebook, [D] egen, [D] ipolate
[R] misstable, [D] mvencode, [D] recode,
[MI] intro,

findname (if installed), mipolate (if installed)