help for specc

Title

specc -- manages component dofiles and alternative specifications
for the creation of specification curves in Stata.

Description

specc is designed to automate the creation of specification curves.
First, it enables the assisted creation of sets of alternative options ("methods")
for each choice in the estimation of a given statistical parameter
(such as outcome definition, covariate choice, and functional form; i.e. "the garden of forking paths").
Second, it automatically assembles all of these individual decisions
into the full set of possible specifications formed by interacting these various choices.
In other words, the intent of specc is to reduce the programming task
for an exponential number of specifications to a linear amount of coding.

Functions

specc initialize
creates the directory and database
that the command will use to build a particular specification curve,
and need only be used at the first call of the command.

specc new
is used to create a new method
which represents a possible choice at some stage in the choice tree;
it creates a dofile to hold the instructions and a registry entry for the method.

specc remove
is used to delete a method
which represents a possible choice at some stage in the choice tree;
it deletes the corresponding dofile and registry entry for the method.

specc set
is used to inform the command
the order in which the choices ("classes") are to be iterated over.
This sequence of choices must end with the estimation ("model") class
so that the estimated parameters are passed back to the results.

specc report
returns the current state
of the specc registry (i.e., the available choice classes and methods).
It also reports whether an execution order has been set
and what the corresponding methods for the ordered classes will be.

specc run
calculates and executes the full set
of possible choices over the entire range of classes that are set
and returns the results in the form of a specification curve,
reporting the combination of methods that corresponds to each estimate.

Syntax

Setting up the specc directory

specc initialize using "/specc/directory/"

This function will initialize the specc.dta registry in the specified directory.
It will use the directory /specc/ in the current working directory if none is specified.
It will also create the "main" method for the "model" choice class
since this is always required.

Creating choice classes and methods

specc new class_name method_name

Method Description

using "/specc/directory/"

[, code("starter Stata code")] [replace]

This function will initialize a new method for the specified class.
Each "class" groups a set of mutually exclusive choices for that point
in the decision tree; the choices within each class are referred to as "methods".
Class names and method names must be one word and must be valid Stata variable names.
Method descriptions can be anything but should be kept short for legibility.
If the class does not exist, the command will create a directory for it;
if the method does not exist, the command will create a registry entry for it.
It will also create a corresponding dofile for the method in the class directory.
Finally, it can pass up to one line of code into the template dofile.
You typically must manually edit the dofile to achieve the desired result.
Additionally, methods of the "model" class will come with template instructions
to create a matrix holding the key parameters for the specification curve.

Removing choice classes and methods

specc remove class_name method_name

using "/specc/directory/"

This function will remove any method matching the specified class and method names.
You can specify drop as an alternative to remove.

Setting the execution order

specc set class_name [class_name] [class_name] [...] model

using "/specc/directory/" , reset

This function instructs specc to prepare to execute
a series of possible specifications using the methods stored in the registry.
For each of the classes specified in the set execution order,
specc will iterate over each of the available combinations of methods.
Since model is required to be last and at least one model method is required,
the estimation results will then be returned to the command for visualization.

Reporting the current status

specc report [class_name method_name]

using "/specc/directory/" , [sort]

This function requests specc to return the current status of the registry.
If the command is specified without a class or model,
it will return a list of all registered classes and models
as well as the currently set execution order, if any.
Specifying sort will list them in alphabetical order rather than in timestamp order.
If a class and model are specified, it will report the contents of the corresponding dofile.
Note that macro characters will not display properly and the dofile is definitive.

Running the execution order

specc run using "/specc/directory/"

, [sort] [save] [twoway_options]

This function requests specc to run the currently set execution order.
In the order specified, it will iteratively substitute each method dofile in the given class,
and obtain an estimation result from each registered model.
The results will be displayed in the form of a specification curve.
If sort is specified the curve will be displayed in the order of methods;
by default it is ordered such that the estimated parameter is increasing.
If save is specified it will save results.dta in the directory
containing the results of each of the estimated specifications.

Authors

Benjamin Daniels

Please send bug reports, suggestions and requests for clarifications
writing "specc" in the subject line to bbdaniels@gmail.com.

You can also see the code, make comments to the code, see the version
history of the code, and submit additions or edits to the code through
the GitHub repository for specc:

https://github.com/bbdaniels/specc

References and Acknowledgements

The Garden of Forking Paths

https://statmodeling.stat.columbia.edu/2019/08/01/the-garden-of-forking-paths/

Simonsohn, Uri, Joseph P. Simmons, and Leif D. Nelson.
"Specification curve: Descriptive and inferential statistics on all reasonable specifications."
Available at SSRN 2694998 (2015).

Hans H. Sievertsen "Specification curve using Stata"
https://github.com/hhsievertsen/speccurve

Notes

specc is open for development on GitHub.
Submit bugs and feature requests here.
If you like specc, be sure to visit my homepage.