help for descgen (Roger Newson)

Add Stata dataset attributes to a xdir or xframedir resultsset

descgen [if] [in] [ ,
filename(varname) dirname(varname) nodn

framename(varname)

isdta(newvarname) nobs(newvarname) nvar(newvarname) width(newvarname) size(newvarname)

sortedby(newvarname) nosb allvars(newvarname) noav

label dslabel(newvarname)

charlist(enamelist) charprefix(stub)

replace
]

where enamelist is a list of items that are either valid Stata names or *,
and stub is a valid prefix for variable names.

Description

descgen is intended for use in output datasets (or resultssets)
produced by the xdir or xframedir module of the xdir package,
which have one observation for each of a list of files or frames,
and data on directory names and file or frame names.
It inputs the file or frame name variable and (optionally) the directory name variable,
and generates a list of new variables,
containing, in each observation, Stata dataset attributes
describing the Stata dataset stored in the corresponding file or frame.
These attributes include numbers of variables and observations and sizes of an observation or of the dataset,
and (optionally) dataset labels, dataset characteristics or lists of variables in the dataset.
If the corresponding file or frame does not exist, or does not contain a Stata dataset,
then these dataset attribute variables will have missing values.

Options

filename(varname) specifies an existing string variable,
containing the file names.
If absent, then it is set to filename.

dirname(varname) specifies an existing string variable,
containing the directory names.
If absent, then it is set to dirname.

nodn specifies that the directory name variable specified by dirname() will not be used.
This option can be useful,
if the file name variable specified by filename() contains the full file path for each file,
including the directory name.

framename(varname) specifies an existing string variable,
containing the frame names.
If absent, then it is set to framename.
If the filename() and framename() variables are both present,
then the filename() variable is used, and the framename() option is ignored.

isdta(newvarname) specifies the name of the output variable containing an indicator
that the file or frame name belongs to a Stata dataset that can be described using describe.
It is 1 for Stata dataset files, 0 for non-dataset or non-existent files or frames,
and missing for observations out of the sample selected by the if and in qualifiers.
If absent, then it is set to isdta.

nobs(newvarname) specifies the name of the output variable containing the numbers of observations in the datasets.
If absent, then it is set to nobs.

nvar(newvarname) specifies the name of the output variable containing the numbers of variables in the datasets.
If absent, then it is set to nvar.

width(newvarname) specifies the name of the output variable containing the widths (in bytes) of observations in the datasets.
If absent, then it is set to width.

size(newvarname) specifies the name of the output variable containing the sizes (in bytes) of the full data areas in the datasets.
These sizes are equal to the product of the nobs() output variable and the width() output variable.
If absent, then it is set to size.

sortedby(newvarname) specifies the name of the output string variable containing the lists of variables by which the datasets are sorted.
If absent, then it is set to sortedby.

nosb specifies that the sortedby() output variable will not be produced.
This option can be useful to save space if the sort keys are very long,
which may be the case for datasets with many variables.

allvars(newvarname) specifies the name of the output string variable containing the lists of all variables in each dataset.
If absent, then it is set to allvars.

noav specifies that the allvars() output variable will not be produced.
This option can be useful to save space, if the datasets have many variables.

label specifies that the dslabel() output variable will be produced,
containing the dataset label.
(See below.)

dslabel(newvarname) specifies the name of the output string variable
containing the dataset label in each dataset.
If absent, then it is set to dslabel.
If the label option is absent, then any dslabel() option is ignored.

charlist(enamelist) specifies a list of items that are either valid names for dataset characteristics
or the character *.
These specify a list of dataset charateristics for which new variables will be created,
containing, in each observation,
the value of the named dataset characteristic for the corresponding Stata dataset.
If a * is specified,
then new variables will be created for all dataset characteristics present in any of the datasets.
The new variables will be named according to the charprefix() option.

charprefix(stub) specifies a prefix for the variables to be created by the charlist() option.
The variables will have names of the forn stubn,
where n is a number with a value ranging from 1 to the number of variables created.
If charprefix() is not specified,
then it is set to dschar.
The numbers will order the variables in order of first appearance of the corresponding characteristic name
in the enamelist specified by the charlist() option.

replace specifies that any existing variables with the same names as the generated output variables will be replaced.

Output variables created by descgen

The descgen package creates the following list of variables:

Default name Description

isdta Stata dataset status indicator

nobs N of observations

nvar N of variables

width Width of observation (bytes)

size Size of dataset (bytes)

sortedby Sort list of variables

allvars List of all variables

dslabel Dataset label

dscharn Characteristic _dta[characteristicname]

The variables sortedby and allvars can be absent,
if the user sets the options nosb and noav,
respectively.
The variable dslabel is only created if the label option is specified.
The variables dscharn are only created if the charlist() option is specified.
All these variables can be renamed,
using the options of the same names,
or using the charprefix() option in the case of the dscharn variables.

Remarks

descgen is designed for use in output datasets (or resultssets),
with one observation for each of a list of files,
created using the xdir package,
which can be downloaded from SSC.
It inputs the variables whose default names are filename or framename,
and (optionally) dirname,
containing the file or frame names and directory names, respectively.
It outputs, in each observation,
the dataset attributes that may be output by the describe command,
with the using modifier,
if the file or frame specified by the observation contains a Stata dataset.
If a file or frame specified by an observation is does not exist,
or does not contain a Stata dataset,
then the values of the corresponding output variables will be missing.

descgen only stores a basic summary of the attributes of a dataset,
optionally including lists of variables.
If the user wants more detailed information on the attributes of the individual variables,
then the user should use describe,
or the SSC package descsave.

Examples

The examples are designed to be demonstrated in an output dataset (or resultsset)
produced by the xdir package.
This resultsset has one observation for each of a list of Stata dataset files,
all of which are copied from the Stata datasets downloadable using the sysuse command.
These Stata dataset files are all stored in a subdirectory of the current working directory,
produced using the commands in the set-up section.
Once these files have been created,
the xdir command is used to create a resultsset in the memory,
with one observation for each of the Stata data files,
and two variables dirname and filename,
containing the names of the directory and the file, respectively.
Once this resultsset is created,
the user can use descgen to add new variables,
containing the various dataset attributes.

Set-up:

. capture noisily mkdir ./dta

. local sysdta "auto auto2 autornd bplong bpwide"

. foreach DS in `sysdta' {

. sysuse `DS', clear

. describe, full

. save ./dta/`DS', replace

. }

. xdir, dirname(./dta) pattern(*.dta) norestore

. describe, full

. list, abbr(32)

The following example creates the default full set of generated variables,
with the default names,
and describes and lists them.

. descgen

. describe, full

. list, abbr(32)

The following example discards the variables created in the previous example,
and creates a new reduced set of variables,
using the nosb and noav options
to omit the variable-list string variables sortedby and allvars.
These are also described and listed,
producing a more concise output.

. keep dirname filename

. descgen, nosb noav

. describe, full

. list, abbr(32)

The following example discards the variables created in the previous example,
and creates a new increased set of variables,
using the label option
to create an additional variable dslabel containing the dataset labels.
These are also described and listed,
producing a more informative output.

. keep dirname filename

. descgen, label

. describe, full

. list, abbr(32)

The following example discards the variables created in the previous example,
and creates a new set of variables,
containing the same information as in the first example,
but with non-default names.
These are also described and listed.

. keep dirname filename

. describe, full

. descgen, isdta(isdset) nobs(N_obs) nvar(N_var) width(obswidth) size(datasize) sortedby(sortvars) allvars(fullvars) label dslabel(datlab)

. describe, full

. list, abbr(32)

The following example discards the variables created in the previous example,
and creates a new set of variables,
including variables dschar1 and dschar2,
containing the dataset characteristics _dta[note1] and _dta[note2].

. keep dirname filename

. describe, full

. descgen, charlist(note1 note2)

. describe, full

. list, abbr(32)

The following example discards the variables created in the previous example,
and creates a new set of variables,
including variables (cmd:c_1}, c_2, c_3, and c_4,
containing the full set of characteristics present
in any of the datasets represented by the observations.

. keep dirname filename

. describe, full

. descgen, charlist(*) charprefix(c_)

. describe, full

. list, abbr(32)

The following example uses the xframedir module of the xdir package
to create a temporary data frame frankie,
containing a dataset with 1 observation for each frame in memory
and a variable framename containing the frameee names.
In this data frame,
we use descgen to create the new variables describing the datasets in those frames,
and then describe and list them.
The frame frankie is then dropped.

. xframedir, frame(frankie, replace)

. frame frankie {

. descgen, label charlist(*)

. describe, full

. list, abbr(32)

. }

. frame drop frankie

Saved results

descgen saves the following in r():

Macros

r(charlist) Full list of dataset characteristic names specified by charlist()

descgen also stores a characteristic charname
for each of the generated dataset characteristic variables,
containing the corresponding dataset characteristic name.

Author

Roger Newson, Imperial College London, UK.
Email: r.newson@imperial.ac.uk

Also see

Manual: [D] dir, [D] describe

On-line: help for dir, describe

help for xdir, xframedir, descsave if installed