Sequence analysis using Stata -- software demonstration

Ulrich Kohler, University of Potsdam
ulrich.kohler@uni-potsdam.de

Christian Brzinsky-Fay, WZB
brzinsky-fay@wzb.eu

Magdalena Luniak, WZB

Contents

Preliminaries

o Long versus wide
o sqset the data

The egen functions

o Generate variables with descriptive information on sequences
o Display contents of e-generated variables

Descriptive tables

o Tabulate sequences
o Describe sequences

Graph sequences

o Parallel-coordinates plot
o Sequence index plots

Optimal matching

o The sqom command
o Accessing the results
o On speed

Long versus Wide

Sequences are entities of their own; i.e., one thinks about
sequences in "wide" form, and that is how datasets are usually
structured.

. use http://www.uni-potsdam.de/soziologie-methoden/dokumente/kohler/data/youthemp.dta , clear

. list id st1-st10 in 1/10

Many Stata users prefer sequences in "long" form. The programs are
therefore written for data in long form. Hence, to use the programs,
one has to reshape from wide to long.

. reshape long st, i(id) j(order)

. list id st in 1/10

sqset the data

To work with the SQ-Ados, one has to sqset the data. This
command works similar to tsset, stset, or xtset.

Syntax

sqset
elementvar idvar ordervar [, trim rtrim
ltrim keeplongest]

sqset [, clear]

Example

. sqset st id order

Among other things, sqset checks for gaps, confirms integer
order and uniqueness of sequence-IDs, and confirms order.

Generate variables with summary descriptions

The SQ-egen functions are used to generate variables that
hold a summary description of each sequence.

General usage

egen [type] newvar = sqfcn() [, options]

Examples

. egen seqlen = sqlength() <- Overall length of sequence
. egen dur1 = sqlength(), element(1) <- Overall length of sequence of element 1
. egen gaplen = sqgaplength() <- Length of gaps
. egen gapcount = sqgapcount() <- Number of episodes with gaps
. egen elemnum = sqelemcount() <- Number of different elements in sequence
. egen chnum = sqepicount() <- Number of episodes
. egen epi1 = sqepicount(), element(1) <- Number of episodes of element 1

. describe

Stata keeps track of all variable names that are generated
with the SQ-egen functions. Other SQ-commands automatically use the
e-generated variables. The names of the e-generated variables are
stored as together with the dataset.

Display contents of e-generated variables

The sqstat bundle provides convenient displays for
the variables generated with the SQ-egen functions.

List features of sequences

. sqstatlist if sex
. sqstatlist dur1 elemnum chnum, ranks(1/10)

. preserve
. sqstatlist sex dur1, replace
. describe
. tab sex, sum(dur1)
. restore

Summarize features of sequences

. sqstatsum
. sqstatsum dur1 epi1 if sex

Tabulate features of sequences

. sqstattab1
. sqstattab1 dur1 gaplen

. sqstattab2 elemnum sex

. sqstattabsum sex
. sqstattabsum sex, sum(dur1)

Tabulate sequences

sqtab is used to produce a frequency table of the
sequences in the dataset.

Syntax

sqtab [varname] [if] [in] [, ranks(numlist)
se so nosort gapinclude
tabulate_options]

Example

. sqtab
. sqtab, ranks(1/10)

"Same order" and "Same elements"

sqtab allows a simple definition of similarity of
sequences. With the option so, all sequences that have the same
order of elements are collapsed together. The option se
collapses sequences that consist of the same elements.

. sqtab, so
. sqtab, se

Describe sequences

sqdes produces a descriptive overview of the sequences
in the dataset. More specifically, it shows

o the number of elements observable over all sequences
(k),

o the maximum length of the sequences (l),

o the number of possible sequences that might be formed with k
elements of length l,

o the number of different sequences in the dataset, and

o the number of sequences that are shared by ... persons

Syntax

sqdes [if] [in] [, so se graph gapinclude]

Examples

. sqdes
. sqdes, so
. sqdes, se graph

Graph sequences as parallel-coordinates plot

sqparcoord produces parallel-coordinates plots of the
sequences in the dataset. In its simplest form, such plots are useful
only for very small numbers of sequences. Therefore, sqparcoord
provides several options to produce meaningful displays even with larger
numbers of sequences.

Syntax

sqparcoord [if] [in] [, ranks(numlist) so
offset(#) wlines(#) gapinclude twoway_options]

Examples

. sqparcoord <- All sequences (useless)
. sqparcoord, ranks(1/10) offset(.5) <- 10 most frequent sequences, with offset
. sqparcoord, wlines(7) <- Plot frequent sequences much thicker

. sqparcoord, so ranks(1/10) offset(.5) <- Using "same order" sequences
. sqparcoord, so wlines(7) <- Plot frequent sequences much thicker

Graph sequences as sequence index plot

sqparcoord produces a sequence index plot (Brüderl
and Scherer 2006). In these plots, the episodes of the sequences are
plotted as stacked horizontal bars with colors to separate the
different elements.

As stressed elsewhere, the results of sequence index plots
depend on the order the sequences in the graph. A simple algorithm
is used to order of the sequences in the plot, but results of more
sophisticated algorithms can also be used (for example, results from
sqom).

Syntax

sqindexplot [if] [in] [, ranks(numlist) se so
order(varname) by(varname) color(colorstyle)
gapinclude twoway_options]

Examples

. sqindexplot, color(blue green black yellow red)
. sqindexplot, ranks(1/10)
. sqindexplot, so
. sqindexplot, se

Note

With sequence index plots, one might overstate the frequency
of elements on "high" levels. This can be minimized by (a) decent
ordering and (b) tuning the aspect ratio.

Perform optimal matching

sqom performs a cluster analysis of sequences on the basis of
a distance matrix produced by the Needleman-Wunsch algorithm. It
allows free specification of "Indel" and "substitution" cost, as well
as different kinds of standardizations. Results are stored for
later use.

Syntax

sqom [if] [in] [, indelcost(#)
subcost(#|rawdistance|matexp|matname)}
name(varname)
refseqid(spec)
full
k(#)
standard(#|cut|longer|longest|none)

Examples

. sqom <- Default: Indel = 1, subcost = 2
. sqom, indelcost(3) <- Indel = 3, subcost = indelcost*2
. sqom, subcost(rawdistance) <- Indel = 1, subcost = abs(value1-value2)

. matrix sub = 0,8,7,3,2\8,0,8,7,3\7,8,0,8,7\3,7,8,0,7\2,3,7,7,0
. sqom, subcost(sub) <- subcosts from matrix "sub"

. sqom, standard(cut) <- cut at length of shortest
. sqom, standard(6) <- cut at length of 6
. sqom, standard(longer) <- divide by the longer of two

. sqom, full k(2) <- full dissimilarity matrix

Accessing results of optimal matching

Results from sqom can be accessed for further
analysis. Distances are either saved as a variable or as a Stata
matrix named SQdist. The convenience programs sqclusterdat
and mdsadd helps adding results of cluster analyses and/or multidimensional
scaling to the sequence data.

Examples

. sqom, name(om1)
. describe om1
. sqindexplot, order(om1)

. sqom, full k(2)
. matrix dir
. sqclusterdat
. clustermat wardslinkage SQdist, name(myname) add
. cluster tree myname, cutnumber(20)
. sqclusterdat, return

. mdsmat SQdist
. predict mdsdim1, saving(mds)
. sqmdsadd using mds