*! version 1.2.0 23Mar2019

Title

glevelsof -- Efficiently get levels of variable using C plugins

Important: Please run gtools, upgrade to update gtools to
the latest stable version.

Syntax

This is a fast option to Stata's levelsof. It can additionally take
multiple variables.
It is 3 to 13 times faster in Stata/IC and 2.5-7 times faster in MP

glevelsof
varlist
[if] [in]
[, options]

Instead of varlist, it is possible to specify

[+|-]
varname
[[+|-]
varname ...]

To change the sort order of the results.

Options
clean display string values without compound double quotes

local(macname) insert the list of values in the local macro macname

missing include missing values of varlist in calculation

separate(separator) separator to serve as punctuation for the values of returned list; default is a space

Extras
nolocalvar Do not store the levels of varlist in a local macro.

silent Do not display the levels of varlist. For use with gen() and matasave

matasave[(str)] Save results in mata object (default name is GtoolsByLevels)

gen([prefix], [replace]) Store the levels of varlist in new varlist (prefix) or replace varlist with its levels

colseparate(separator) separator to serve as punctuation for the columns of returned list; default is a pipe

numfmt(format) Number format for numeric variables. Default is %.16g (or %16.0g with matasave).

unsorted do not sort levels (ignored if inputs are integers)

Gtools
compress Try to compress strL to str#.

forcestrl Skip binary variable check and force gtools to read strL variables.

verbose Print info during function execution.

bench[()] Benchmark various steps of the plugin. Optionally specify depth level.

hashmethod(str) Hash method (default, biject, or spooky). Intended for debugging.

oncollision(str) Collision handling (fallback or error). Intended for debugging.

Description

glevelsof displays a sorted list of the distinct values of varlist.
It is meant to be a fast replacement of levelsof. Unlike levelsof,
it can take a single variable or multiple variables.

glevelsof is part of the [R] gtools project.

Options

Options

clean displays string values without compound double quotes.
By default, each distinct string value is displayed within compound double
quotes, as these are the most general delimiters. If you know that the
string values in varlist do not include embedded spaces or embedded
quotes, this is an appropriate option. clean
does not affect the display of values from numeric variables.

local(macname) inserts the list of values in
local macro macname within the calling program's space. Hence,
that macro will be accessible after glevelsof has finished.
This is helpful for subsequent use, especially with foreach.

missing specifies that missing values of varlist
should be included in the calculation. The default is to exclude them.

separate(separator) specifies a separator
to serve as punctuation for the values of the returned list.
The default is a space. A useful alternative is a comma.

colseparate(separator) specifies a separator
to serve as punctuation for the columns of the returned list.
The default is a pipe. Specifying a varlist instead of a
varname is only useful for double loops or for use with
gettoken.

numfmt(format) Number format for printing. By default numbers
are printed to 16 digits of precision, but the user can specify
the number format here. By default, only "%.#g|f" and "%#.#g|f" are
accepted since this is formated internally in C. However, with option
matasave this is formated in mata and has to be a mata format.

unsorted Do not sort levels. This option is experimental and
only affects the output when the input is not an integer (for integers,
the levels are sorted internally regardless; the user would request the
spooky hash method via hash(), which obeys the unsorted
option, but this is intended for debugging). While not sorting the
levels is faster, glevelsof is typically used when the number
of levels is small (10s, 100s, 1000s) and thus speed savings will be
minimal.

nolocalvarDo not store the levels of varlist in a local macro.
This is specially useful with option gen().

silentDo not display the levels of varlist. Mainly for use with
gen() and matasave. With matasave, the levels are
not sepparately stored as a string matrix, but the raw levels are
kept.

matasave[(str)]Save results in mata object (default
name is GtoolsByLevels). See GtoolsByLevels.desc() for more.
This object contains the raw variable levels in numx and charx
(since mata does not allow matrices of mixed-type). The levels are saved
as a string in printed (with value labels correctly applied) unless
option silent is also specified.

gen([prefix], [replace]) Store the levels of varlist in new
varlist (prefix) or replace varlist with its levels.
These options are mutually exclusive.

Gtools

compress Try to compress strL to str#. The Stata Plugin Interface
has only limited support for strL variables. In Stata 13 and earlier
(version 2.0) there is no support, and in Stata 14 and later (version
3.0) there is read-only support. The user can try to compress strL
variables using this option.

forcestrl Skip binary variable check and force gtools to read strL
variables (14 and above only). Gtools gives incorrect results when there is binary data in strL variables.
This option was included because on some windows systems Stata detects
binary data even when there is none. Only use this option if you are
sure you do not have binary data in your strL variables.

verbose prints some useful debugging info to the console.

benchmark and benchmarklevel(int) print how long in
seconds various parts of the program take to execute. The user can also
pass bench(int) for finer control. bench(1) is the same
as benchmark but bench(2) and bench(3) additionally print
benchmarks for internal plugin steps.

hashmethod(str) Hash method to use. default automagically
chooses the algorithm. biject tries to biject the inputs into the
natural numbers. spooky hashes the data and then uses the hash.

oncollision(str) How to handle collisions. A collision should never
happen but just in case it does gtools will try to use native commands.
The user can specify it throw an error instead by passing oncollision(error).

Remarks

glevelsof serves two different functions. First, it gives a
compact display of the distinct values of varlist. More commonly, it is
useful when you desire to cycle through the distinct values of
varlist with (say) foreach; see [P] foreach.
glevelsof leaves behind a list in r(levels) that may be used in a
subsequent command.

glevelsof may hit the limits imposed by your Stata. However,
it is typically used when the number of distinct values of
varlist is modest. If you have many levels in varlist then
an alternative may be gtoplevelsof, which shows the largest or smallest
levels of a varlist by their frequency count.

Examples

. sysuse auto

. glevelsof rep78

. display "`r(levels)'"

. glevelsof rep78, miss local(mylevs)

. display "`mylevs'"

. glevelsof rep78, sep(,)

. display "`r(levels)'"

. glevelsof foreign rep78, sep(,)

. display `"`r(levels)'"'

. glevelsof foreign rep78, gen(uniq_) nolocal

. desc uniq_*

. glevelsof foreign rep78, mata(uniq) nolocal

. mata uniq.desc()

See the
online documentation
for more examples.

Stored results

glevelsof stores the following in r():

Macros

r(levels) list of distinct values

Scalars

r(N) number of non-missing observations

r(J) number of groups

r(minJ) largest group size

r(maxJ) smallest group size

With matasave, the following data is stored in GtoolsByLevels:

real scalar anyvars
1: any by variables; 0: no by variables

real scalar anychar
1: any string by variables; 0: all numeric by variables

real scalar anynum
1: any numeric by variables; 0: all string by variables

string rowvector byvars
by variable names

real scalar kby
number of by variables

real scalar rowbytes
number of bytes in one row of the internal by variable matrix

real scalar J
number of levels

real matrix numx
numeric by variables

string matrix charx
string by variables

real scalar knum
number of numeric by variables

real scalar kchar
number of string by variables

real rowvector lens
> 0: length of string by variables; <= 0: internal code for numeric variables

real rowvector map
map from index to numx and charx

real rowvector charpos
position of kth character variable

string matrix printed
formatted (printf-ed) variable levels (not with option -silent-)

Author

Mauricio Caceres Bravo

mauricio.caceres.bravo@gmail.com

mcaceresb.github.io

Website

glevelsof is maintained as part of [R] gtools at github.com/mcaceresb/stata-gtools

Acknowledgment

This help file was based on StataCorp's own help file for levelsof.

This project was largely inspired by Sergio Correia's ftools:
https://github.com/sergiocorreia/ftools.

The OSX version of gtools was implemented with invaluable help from @fbelotti;
see https://github.com/mcaceresb/stata-gtools/issues/11.

Also see

help for
gtoplevelsof,
gtools;
flevelsof (if installed),
ftools (if installed)