help for ingap (Roger Newson)

Insert gap observations in a dataset

ingap [ numlist ] [if] [in]
[ , after

gapindicator(newvar[,replace]) neworder(newvar[,replace])

rowlabel(varname) growlabels(string_list)
grexpression(gap_row_label_expression)
rstring(string_replacement_option)
fast
]

where numlist is an optional list of integers, string_list is a list of strings,
gap_row_label_expression is a string-valued expression,
and string_replacement_option can be

order | name | type | format | varlab | char characteristic_name | label | labname

and characteristic_name is the name of a variable characteristic.

The by prefix can be used with ingap; see help for prefix.

Description

ingap inserts gap observations into a list of positions in an existing dataset.
All existing variables in the dataset (apart from by-variables)
will have missing values in the gap observations, unless the user specifies otherwise.
Often, the user specifies non-missing values in the gap observations for one particular existing string variable,
known as the row label variable.
This row label variable may then be output with a list of other variables to form a publication-ready table,
using the listtab package (or possibly the listtex package).
Alternatively, the row label variable may be encoded, using the sencode package,
to form a numeric variable with value labels,
which can then be plotted on one axis of a graph to define axis labels.
The sencode, listtab and listtex packages are downloadable from SSC.

ingap inserts a gap observation next to (before or after) each of a list of observations
specified by the numlist.
A positive number i in the numlist specifies the ith existing observation in the dataset,
or in each by-group if the by prefix is specified.
A negative number -i in the numlist specifies the ith existing observation,
in reverse order, from the end of the dataset,
or from the end of each by-group if the by prefix is specified.
A zero or out-of-range number in the numlist is ignored.
The numlist is set to 1 if not specified by the user.
ingap assumes that the dataset in memory has up to 3 classes of variables.
These are the by-variables (which define by-groups possibly representing the pages of a table),
a row label variable (possibly containing the row labels in the left margin of the table),
and the remaining variables (which may form the entries in the table rows).
A gap observation inserted by ingap has the same values for the by-variables
as the observation next to which it was inserted,
a row label value specified by the growlabels() or grexpression() options,
and missing values in the remaining variables (unless the user specifies otherwise).
ingap may also generate new variables,
indicating whether the observation is a gap observation
and/or the new order of the observation in the dataset (or by-group) after the gap observations have been inserted.

Options

after specifies that each gap observation will be inserted
after the corresponding existing observation in the dataset or by-group specified in the numlist.
If after is not specified,
then each gap observation will be inserted before the corresponding existing observation.

gapindicator(newvarname[,replace]) specifies the name of a new variable to be generated,
equal to 1 for the newly-inserted gap observations and 0 for all other observations.
The replace suboption specifies that any existing variable with the same name will be replaced.

neworder(newvarname[,replace]) specifies the name of a new variable to be generated,
equal to the new sequential order of the observation within the dataset
(or within the by-group if the by prefix is specified),
after the gap observations have been inserted.
The new variable has no missing values.
After execution of ingap, the dataset in memory is sorted primarily by the by-variables (if specified),
and secondarily by the neworder() variable (if specified).
The replace suboption specifies that any existing variable with the same name will be replaced.

rowlabel(string_varname) specifies the name of an existing string variable,
used as the row labels for a table whose rows are the observations.
In the gap observations,
this string variable is set to the value specified by the corresponding string listed in the growlabels() option
if that option is specified (see below),
or to a missing value otherwise.
The rowlabel() variable may not be a by-variable.

Note that the neworder(), gapindicator() and rowlabel() options may not specify the same variable names,
and may not specify the names of by-variables.
Also, note that the neworder() and gapindicator() variables are always non-missing,
even in observations not included in the sample defined by the if and in qualifiers.
These qualifiers only specify that an observation may have observations inserted before it
(or after it, if after is specified),
if its sequential order in the dataset or by-group is included in the numlist.

growlabels(string_list) specifies a string value for each of the row labels in the gap observations.
The jth string in the string_list is written to the rowlabel variable
in the newly-inserted gap observation inserted next to the jth observation mentioned in the numlist.
If the rowlabel option is present and the growlabel() option is absent,
then the rowlabel() variable is initialised to missing in the gap observations.

grexpression(gap_row_label_expression) specifies a string expression,
to be evaluated in all gap observations to give the final values of the rowlabel() variables
in these gap observations.
If grexpression() and growlabels() are both specified,
then the result of grexpression() replaces any values set by growlabels().
(However, the name of the rowlabels() variable may appear in the grexpression() expression,
so that the values of the rowlabels() variable can be modified
in ways depending on the original values set by the growlabels() list.)
Note that, when the grexpression() expression is evaluated,
all variables other than the rowlabels() variable have been set to their final values,
which are missing for all variables except the by-variables and the rowlabel() variable,
except if they have been set to other values by the rstring() option (see below).
However, the grexpression() expression may access values of variables in adjacent observations using subscripting.
If by-variables are present, then any subscripts in the expression specified by grexpression() are defined within by-groups,
and are defined including the gap observations.
For instance, if a gap observation is inserted at the beginning of each by-group,
then the value of _n in these gap observations will be 1.

rstring(string_replacement_option) specifies a rule for replacing the values of
string variables (other than the by-variables and row label variables) in gap observations.
If rstring() is not set,
then these variables will be set to a missing value (an empty string) in the gap observations.
rstring() can be set to
order, name, type, format, varlab, char characteristic_name,
label, or labname.
The options order, name, type, format, varlab and char characteristic_name
imply that the value of each string variable, in the gap observations,
will be set to the order of the variable in the existing dataset,
the storage type of the variable,
the display format of the variable,
the variable label of the variable,
or the ccharacteristic of the variable with the name characteristic_name,
respectively.
The option label is a synonym for varlab.
The option labname specifies that the value of each string variable, in the gap observations,
will be set to its variable label, if that label exists,
and to its name otherwise.
(Note that numeric variables that are not by-variables, gapindicator() variables or neworder() variables
are always set to the numeric missing value . in gap observations.)
The rstring() option allows the user to add a row of column headings to a dataset of string variables,
or to add a row of column headings to each by-group of a dataset of string variables.
Note also that numeric variables may be converted to string variables using the sdecode package,
downloadable from SSC,
before using ingap and listtab.
This allows the user to use the rstring() option,
and also to format numeric variables in ways not possible using Stata formats alone,
such as adding parentheses to confidence limits.

fast is an option for programmers.
It specifies that ingap will do no work to ensure
that the original dataset is preserved in the event that ingap fails,
or if the user presses the Break key.
If fast is not specified,
and ingap fails, or the user presses the Break key,
then the original existing dataset is preserved, with no additional gap observations.

Remarks

ingap is typically used to convert a Stata dataset to a form with 1 observation per table row
(including gap rows), or 1 observation per graph axis label (including gap axis labels).
The user can then list the dataset as a TeX, LaTeX, HTML or Microsoft Word table,
using the listtab package (downloadable from SSC).
Alternatively, for immediate impact, the user can use the sencode package
(downloadable from SSC) to encode the row labels to a numeric variable,
and then plot this numeric variable against other variables using Stata graphics programs.
For instance, a user of Stata 8 or above might use eclplot (downloadable from SSC)
to produce horizontal confidence interval plots, with the row labels on the vertical axis.
It is often advisable for the user to type preserve before a sequence of commands including
ingap, and to type restore after a sequence of commands using ingap,
because ingap modifies the dataset by adding new observations.
It is often also advisable for the user to place the whole sequence of commands in a do-file,
and to execute this do-file,
rather than to type the sequence of commands one by one at the terminal.

The listtab package is described in Newson (2012).
It inputs a list of variables in a Stata dataset,
and outputs a text table, in a file or on the screen,
containing these variables as columns and the observations as rows,
and formatted using a row style.
This row style may correspond to table rows in plain TeX, LaTeX, HTML, XML, or RTF tables,
or to rows of tab-delimited, column-delimited or ampersand-delimited generic text spreadsheets,
or to rows in other styles that may be invented in future.
The row style is defined using a row-beginning string, a row-end string,
and a between-column delimiter string.
listtab is a successor to the listtex package,
described in Newson (2006), Newson (2004) and Newson (2003).
The main change introduced in listtab is that empty delimiter strings are now allowed.
Users of Stata versions 10 and above
are advised to use listtab in preference to listtex,
although both packages are still downloadable from SSC.

Examples

. ingap, g(toprow)

. ingap 1 53, g(toprow) row(make) grow("US cars" "Non-US cars")

. by foreign: ingap, g(gind) row(make) grow("Car model")

. sort foreign rep78 make

. by foreign rep78: ingap

. by foreign: ingap -1, after

. by foreign: ingap, row(make) grow("Car model")

. list

The following example works in the auto data if the user has installed the listtex package,
downloadable from SSC.
It outputs to the Results window a generic ampersand-delimited text table,
which can be cut and pasted into a Microsoft Word document,
and then converted to the rows of a table inside Microsoft Word,
using the menu sequence Table->Convert->Text to Table.
(Note that the listtex command can alternatively create table rows suitable for input
into a TeX, LaTeX or HTML file.)

. preserve

. by foreign: ingap, row(make) grexp(cond(foreign,"Non-US cars","US cars"))

. listtab make mpg weight, delim(&) type

. restore

The following example works in the auto data if the user has installed the listtab package,
and also the sdecode package, both of which
can be downloaded from SSC.)
It outputs to the Results window a generic ampersand-delimited
text table, which can be cut and pasted into a Microsoft Word document (as in the previous example),
and then converted into two tables, one for American cars and one for non-American cars, each with a title
line containing the variable labels in the auto data.
Note that, to do this, the user must convert the numeric variables to string variables,
and this is done using sdecode.

. preserve

. sdecode mpg, replace

. sdecode weight, replace

. sdecode price, replace

. by foreign: ingap, rstring(labname)

. listtab make mpg weight price, delim(&) type

. restore

The following example works in the auto data if the user has installed
the sdecode and sencode packages, downloadable from SSC.
It produces a graph of mileage by car type (US or non-US) and
repair record.

. preserve

. sdecode rep78, gene(row) miss

. by foreign: ingap, row(row) grexp(cond(foreign,"Others:","US cars:")) gap(gapind)

. sencode row, replace many gs(foreign -gapind rep78)

. lab var row "Repair record"

. scatter row mpg, yscale(reverse range(0 13)) ylab(1(1)12, valuelabel angle(0)) xlab(0(10)50)

. restore

Other examples of the use of ingap, together with other packages, can be found in
Newson (2012), Newson (2006), Newson (2004) and Newson (2003).

Author

Roger Newson, National Heart and Lung Institute, Imperial College London, UK.
Email: r.newson@imperial.ac.uk

References

Newson, R. B. 2012.
From resultssets to resultstables in Stata.
The Stata Journal 12(2): 191-213.
Download from The Stata Journal website.

Newson, R. 2006.
Resultssets, resultsspreadsheets and resultsplots in Stata.
Presented at the 4th German Stata User Meeting, Mannheim, 31 March, 2006.

Newson, R. 2004.
From datasets to resultssets in Stata.
Presented at the 10th United Kingdom Stata Users' Group Meeting, London, 29 June, 2004.

Newson, R. 2003.
Confidence intervals and p-values for delivery to the end user.
The Stata Journal 3(3): 245-269.
Download from The Stata Journal website

Acknowledgement

I would like to thank Nicholas J. Cox, of the University of Durham, U.K.,
for writing the hplot package, downloadable from SSC.
This package gave me a lot of the ideas used in ingap,
and was also my preferred package for producing confidence interval plots under Stata Versions 6 and 7,
before I had access to the improved graphics of Stata Version 8.

Also see

Manual: [U] 11 Language syntax, [D] by, [D] expand, [P] byable, [R] ssc

On-line: help for by, byprog, expand, ssc

help for listtab, listtex, sencode, sdecode, hplot, eclplot if installed