help tuples

Title

tuples -- Select tuples from a list

Syntax

tuples
list
[ , options ]

where list, if it is a varlist, is unabbreviated. If
items in list contain spaces, these items must be enclosed in double
quotes.

List
asis treat list as is; do not unabbreviate

varlist treat list as varlist; issue an
error if it is not

Selection
max(#) specify maximum number of items in a tuple

min(#) specify minimum number of items in a tuple

conditionals(string) eliminate tuples according
to specified conditions

Reporting
display show created tuples

Method
nopython do not use Python; seldom used

nomata do not use Mata; seldom used

nosort do sort tuples; seldom used

Description

tuples produces a set of local macros, each containing
a list of the items defining a tuple selected from a given list.

By default the given list is tried as a variable list, but if it is not a
variable list any other kind of list is acceptable, except that no other
kind of expansion takes place.

More details are discussed in the Remarks section.

Options

List

asis specifies that the list supplied should be treated as is,
and thus not unabbreviated as a varlist. asis
may not be combined with varlist.

varlist specifies that the list supplied should be a varlist,
so that it is an error if the list is not in fact a varlist. varlist
may not be combined with asis.

Selection

max(#) specifies a maximum value for the number of items in a
tuple. Default # is n, i.e., the number of items in the supplied
list.

min(#) specifies a minimum value for the number of items
in a tuple. Default # is 1.

conditionals() specifies conditional statements to eliminate possible
tuples according to the rule(s) specified.

conditionals() accepts the logical operators &
for intersections or "and" statements, | for unions or "or" statements,
() for binding statements and giving statements priority, and ! for complements or "not" statements.

Other than the foregoing logical operators, conditionals() only accepts
positional arguments. That is, to refer to the first element of the list, use
1; to refer to the second element, use 2; and so forth. Inapplicable
positional arguments (e.g., referring to 4 in a list of size 3) will
produce an error.

Spaces are used to separate conditional statements with conditionals(). A
single statement must, then, contain no spaces.

For a worked example, see: Using conditionals().

conditionals() is not allowed with Stata based
methods to produce tuples.

Reporting

display specifies that tuples should be displayed.

Method

nopython does not call Python to produce the tuples. This option
is implied for Stata versions prior to version 16 or if Python is not
available. nopython is seldom used.

nomata produces tuples outside of the Mata
environment. This option is implied for Stata versions prior to
version 10 or for Stata versions greater or equal to version 16
with Python installed. If combined with nopython,
nomata is generally slow; nomata is seldom used.

nosort does not sort the produced tuples. By default, tuples
first produces all singletons, starting with the last item in the list,
then all distinct pairs, and so on. nosort, if specified with the
default method, produces tuples in a different
sort order. The exact sort order depends on other options; specifying
nosort implies that the exact sort order is irrelevant. The speed
gain, if any, is typically trivial, unless nopython and nomata
are also specified or implied. In general, nosort is seldom used.

Remarks

Remarks are presented under the following headings

Introduction

Using conditionals()

Methods to produce tuples

Introduction

Given a list of n items, tuples by default produces 2^n - 1
macros, named tuple1 upwards, which are all possible distinct singletons
(each individual item); all possible distinct pairs; and so forth. Thus
given frog toad newt, local macros tuple1 through tuple7
contain

newt

toad

frog

toad newt

frog newt

frog toad

frog toad newt

Here n = 3, 2^n - 1 = 7 = comb(3,1) + comb(3,2) + comb(3,3);
see comb().

By default the set of created macros is complete, other than the tuple
containing no selections. Users wishing to cycle over a set of tuples
including the empty tuple can exploit the fact that the local macro
tuple0 is undefined, and so empty (unless the user has previously
defined it explicitly), so that tuple0 can be invoked with the correct
result.

Remember that the number of possible macros will explode with the number
of items supplied. For example, if 10 items are supplied, there will be
1,023 macros, 15 items will result in 32,767, and 20 items imply 1,048,575
macros. The number of macros created by tuples is returned in local
macro ntuples.

As of January 2011, tuples is declared to supersede Nicholas
J. Cox's selectvars.

Using conditionals()

conditionals() is useful for eliminating potential tuples with
combinations of the items from the list based on logical statements.

What is most important to remember about the use of conditionals()
is that the conditional statements apply across all tuples. Thus,
conditionals(1) will force all tuples to contain the first
element in the list.

For example, conditionals() can be used to eliminate combinations of
variables to model in an estimation command that contain products without
first-order (linear) terms (see Example 2 and
Example 4 below). To do so, consider what is to be
done. Imagine 2 variables and their product: A, B, and
A#B (using factor variable notation). Assume they
are listed as

A B A#B

in the list offered to tuples. You need to make sure that A#B
never appears without both A and B. The challenge is then
translating that language into a logical statement.

Begin with an easy component:
"...A#B never appears without both A and B"
contains
"A and B"
which can be represented as
"A&B"
or - because conditionals() requires a
positional statement - "1&2". Thus, you are left with
"...A#B never appears without both 1&2".

In addition,
"...A#B never appears without both 1&2"
contains the term "both". The "both"
implies that "1&2" is a unit and, therefore, should be put in
parentheses, leading to
"...A#B never appears without (1&2)".
Next, consider the word "without" which can be
represented as a "and not" statement. Including the
"and not" statement,
"...A#B never appears &!(1&2))".

Finally, the most tricky component: you need to represent the
fact that A#B and not both A and B cannot be allowed.
Hence, the language "appears" can be translated first into a
statement binding the positional statement for A#B to the existing
logical statement, producing "...never 3&!(1&2)". The last
component is simpler, as the "never" is clearly a
"not" statement. Because that "never" refers to the
notion of A#B appearing with A and B, the
statement must be bound in parentheses, then negated. Incorporating the
last component results in "!(3&!(1&2))". Note that there are no
spaces in the statement.

In most cases, eliminating specific sets of combinations will
require the skillful use of the "!" operator.

Methods to produce tuples

This section is technical and it is of little practical relevance.

For Stata versions 16 or higher, tuples is implemented as a
Python script file in terms of the
combinations()
method from the
itertools
module. Note that tuples has only been tested in Python versions 3.6
through 3.9 and, if Python has not been initialized prior to using tuples, tuples will initialize the Python environment as configured in
python query.

For Stata versions prior to version 16, or if Python is not available,
tuples is implemented in Mata as a variation of
algorithm AS 88 (Gentleman, 1975).

For Stata versions prior to version 10, tuples is implemented in
terms of nested forvalues loops in Stata. The Stata implementation
is generally slow.

Earlier implementations of tuples used different methods to produce
the tuples. These methods are still available as options. The available
methods are

noncr combined with nopython creates an
n x (2^n-1) indicator matrix to produce the tuples

kronecker implements a method based on staggered
Kronecker products

cvp implements a method based on permutations; see
cvpermute()

naive implements the Stata based method in Mata

These alternative methods are typically slower than the default methods and
sometimes require more memory; the respective options are rarely ever used.

Examples

Example 1

Obtain all possible tuples from a list

. tuples a b c d, asis

Example 2

Obtain tuples where two words ("the" and "car") appear in all tuples while
two synonyms ("big" and "large") are not to appear together in any tuple

. tuples the big large red fast car, asis conditionals(1&6 !(2&3))

Example 3

Use tuples to collect and display e(r2) following
regress

. sysuse auto, clear

. generate rsq = .

. generate predictors = ""

. tuples headroom trunk length displacement

. quietly forvalues i = 1/`ntuples' {

. regress mpg `tuple`i''

. replace rsq = e(r2) in `i'

. replace predictors = "`tuple`i''" in `i'

. }

. generate p = wordcount(predictors) if predictors != ""

. sort p rsq

. list predictors rsq in 1/`ntuples'

Example 4

Extension of Example 3, with AIC and an interaction using
conditionals()

. sysuse auto, clear

. generate aic = .

. generate predictors = ""

. tuples headroom trunk length displacement c.trunk#c.length, conditionals(!(5&!(2&3)))

. quietly forvalues i = 1/`ntuples' {

. regress mpg `tuple`i''

. estat ic

. mata: st_store(`i', "aic", st_matrix("r(S)")[1,5])

. replace predictors = "`tuple`i''" in `i'

. }

. generate p = wordcount(predictors) if predictors != ""

. sort p aic

. list predictors aic in 1/`ntuples'

Acknowledgments

Sebastian Orbe, Dejin Xie, Fred Lee, and Raymond Zhang reported problems
that led to bug fixes.

Volodymyr Vovchack suggested including the min() option.

E-mail communication
with William Buchanan and a discussion with John Mullahy on
Statalist
led to the implementation of the new default nopython method to produce
the tuples. Mike Lacy pointed to algorithm AS 88 (Gentleman, 1975) and shared
code on which the default nopython implementation is based.

Thanks to Regina Chua for assistance in testing tuples version 4.0.

References

Gentleman, J. F. 1975. Algorithm AS 88: Generation of All NCR Combinations
by Simulating Nested Fortran DO Loops. Journal of the Royal Statistical
Society. Series C (Applied Statistics), 24(3), pp. 374--376.

Authors

Joseph N. Luchman, Fors Marsh Group LLC

jluchman@forsmarshgroup.com

Daniel Klein, Universität Kassel

klein.daniel.81@gmail.com

Nicholas J. Cox, Durham University

n.j.cox@durham.ac.uk

Also see

Online: foreach, mata, python