*! version 2.0.1 31jul2026 Eric A. Booth and Elizabeth Teas

help for projectbuilder

Title

projectbuilder -- scaffold a data-analysis project folder with a
numbered do-file pipeline, then optionally ingest data, convert it, combine it
into one analytic file, and build a documentation page.

Syntax

projectbuilder Source[/Subsource] [, options]

Source[/Subsource] is one token. Enclose it in quotation marks
if it contains a space; an unquoted second word is rejected with error 198
rather than folded into the folder name. Leading and trailing spaces are
trimmed; interior ones are kept, in the folder name and in r(project).
On macOS and Windows, where the filesystem ignores case, a name differing from
an existing project only in case is the same project and trips the 602
guard.

path(string) base directory, created if its parent exists; default is the current directory

description(string) one-line description of the project

url(string) source URL; recorded, and fetched now if reachable

data(string) one local file, or one folder, to copy into 01_raw/ now, then convert and combine

topic(string) free-text topic tag(s)

publicfacing(string) must be yes, no, or unsure

timeline(string) refresh cadence (for example, monthly)

othernotes(string) free-text caveats or provenance

outcomes(string) up to 10 outcome variable names for the profiler; recorded, not validated

over(string) up to 10 by-variable names for the profiler; recorded, not validated

descsave write a codebook-export call into 300_labels.do, to run when you run that file

rebuild refresh an existing project (re-ingest, re-document)

replace with rebuild, also overwrite edited code files

builddocs render the documentation with webdoc2 if installed

noautoconvert skip the automatic convert/combine pass

All options are optional. The project name is not: projectbuilder with no
name stops with "source[/subsource] required". Each option may be given at most
once; repeating one raises Stata's "option ... not allowed". Two abbreviations
sit close together: des() is description(), while a bare
desc is descsave.

The convert-and-combine step named on the data() row needs
convertanything and combineall, which are optional and not
installed by default. Without them you still get the folder tree, the
do-files, and the documentation. See
Optional dependencies.

Quick start

Start one of two ways. If the data has not arrived yet, build the folder
now:

. projectbuilder MyProject, description("What this project is for")

If you already have the files, point at the folder holding them and get
them ingested in the same call:

. projectbuilder MyProject, data("the_folder_the_files_are_in") ///
description("What this project is for")

Use one or the other, not both: the second call would stop with error 602,
because MyProject already exists. From then on, every data refresh is the
same one command:

. projectbuilder MyProject, rebuild

That is the whole daily loop. The folder is created under the current
directory unless you name another in path(); a plain rerun never
overwrites an existing project; and the dataset in memory is left alone. To
click through a full worked example in the Viewer, see
Try it now.

Description

projectbuilder creates a project folder under the current working
directory (or under path()) and fills it with a numbered do-file
pipeline, an analytic-data folder, an output folder, and a documentation
folder. Everything is written by the command itself with
file write; there is no template folder and no shell call, so it
behaves the same on macOS, Windows, and Linux.

There are two ways to use it. In Workflow A the data exists now, so you
point data() at local files and/or url() at a source address;
projectbuilder copies the files in, converts them, combines them into one
analytic file, and builds the documentation. In Workflow B you scaffold
first and add data later, then rerun with rebuild on every refresh. Both
are shown below.

projectbuilder never overwrites an existing project: a plain call on a
folder that already exists stops with error 602. Use rebuild to opt in to
working on an existing project. A rebuild preserves any do-file in
_code/ you have edited unless you also give replace; the
documentation, being a derived artifact, is regenerated on every rebuild.
The dataset in memory is left untouched throughout; see
Your data in memory.

For a guided tour you can click through in the Viewer, see
Try it now.

Options

path(string) sets the base directory under which the project folder is
created. The default is the current working directory. The project is created
at path/Source (or path/Source/Subsource).
A relative path is resolved against the current directory, so r(path) and
the $root stamped into 000_control.do are always absolute. The base
itself is created if its own parent already exists; path("new/deeper")
where neither level exists stops with error 601 rather than making two levels
at once.

description(string) is a one-line description of the project. It is
stamped into the documentation and into the header of
_code/000_control.do, so the first file a colleague opens says what the
project is for.

url(string) records a source URL. The fetch is written into
100_data_download.do so it is reproducible, and it is attempted once at
scaffold time; if the address is reachable the file lands in 01_raw/.

data(string) names one local file, or one folder of files, to copy into
01_raw/ now, after which the automatic convert/combine pass runs unless
noautoconvert is given. This is the Workflow A entry point for data you
already have on disk. It takes a single path: wildcards are not expanded and a
space-separated list is not split, so data("*.csv") looks for a file
literally named *.csv, does not find one, and says so. Point it at the
folder instead. A relative path is read from the current directory.

topic(string), publicfacing(string), timeline(string), and
othernotes(string) are metadata stamped into the documentation.
publicfacing() must be yes, no, unsure, or empty; case
does not matter, and surrounding spaces are trimmed.

outcomes(string) and over(string) seed the suggested locals in
400_data_profiler.do. Each is capped at 10 items, with a note when
trimmed. Both are recorded as strings, not validated as varlists: at
scaffold time the analytic file usually does not exist yet, so there is nothing
to validate against. Two consequences follow. Names are written through
verbatim, so abbreviations and wildcards such as pri* are recorded but
never expanded. Names that turn out not to exist are not an error either: the
generated profiler tests each name with confirm variable and
skips the ones the analytic file does not have.

descsave writes a codebook-export call (via descsave from SSC) into
300_labels.do. Nothing is exported at scaffold time: the codebook appears
in _documentation/ when you run 300_labels.do, as both
project_codebook.dta (which is what descsave produces, one
observation per variable) and project_codebook.xlsx, exported from it
with Stata's own export excel so it can be read without
Stata. The block is wrapped in preserve, so running
300_labels.do leaves the analytic file loaded as before. Like the other
options it is remembered, so a later rebuild keeps it. Note the
abbreviations here: des() is description(), while a bare
desc is descsave.

rebuild refreshes an existing project: it re-runs the convert/combine
pass over 01_raw/ and regenerates the documentation. Every data refresh
is just another rebuild. Metadata recorded earlier survives a bare
rebuild; see Recorded metadata below. Given
for a project that does not exist yet, rebuild is not an error: it
scaffolds one, and says that is what it did. That is what makes the command
safe to put in a scheduled script, but it also means a mistyped name plus
rebuild produces a second, empty project rather than a complaint. Check
r(rebuilt), which is 1 only when an existing project was
refreshed.

replace has effect only with rebuild: it allows the numbered
do-files to be overwritten. Without it, a rebuild leaves your edited
do-files untouched. One consequence catches people out: adding an option that
changes what a do-file should contain, such as descsave or a new
outcomes(), has no visible effect on a bare rebuild, because the
do-file it would rewrite is one of the guarded ones. Add replace to make
those options reach _code/. The documentation, which is not guarded,
updates either way.

builddocs renders _documentation/website/index.html with
webdoc2 if it is installed. Documentation is always built either way; this
option only makes it prettier. The render runs _documentation/_runall.do,
so its output appears in your log, and a render that fails part way -- most
often because the webdoc2 package directory is not on the adopath -- will
print its own error there. That failure is caught: the built-in
index.html is kept and the run still succeeds.

noautoconvert skips the automatic convertanything and
combineall pass. The calls still appear in 200_data_management.do so
you can run them yourself.

Workflow A -- the data already exists (local files or a source URL)

You have a folder of CSV or Excel files on disk (or a public download URL), and
you want a project built around them in one command.

Step 1. Have the files in a folder. In real work that folder is whatever
the agency sent you; to follow along here, make one:

. mkdir "budget_drop"
. sysuse auto, clear
. export delimited using "budget_drop/budgets_fy24.csv", replace
. export delimited using "budget_drop/budgets_fy25.csv", replace

Step 2. Scaffold and ingest in a single call. Point data() at that
folder (and/or url() at the source). Run it from the directory that should
hold the new project folder, or name that directory in path():

. projectbuilder CountyBudgets, ///
data("budget_drop") ///
description("County budget CSVs, one row per dept per FY") ///
topic("local government, budgets") publicfacing(unsure) ///
timeline("annual") outcomes(total_budget) over(year dept) ///
descsave

data() is read relative to the current directory unless you give a full
path. If it does not exist, projectbuilder says so in red and still builds
the scaffold, so an empty 01_raw/ after this step means the path was
wrong, not that the command failed.

This copies every file from budget_drop/ into CountyBudgets/01_raw/
and writes the documentation page. What happens next depends on which optional
companions are installed, and neither is installed by default -- check with
which convertanything and which combineall before reading the output.
With convertanything installed, projectbuilder converts
01_raw/ into 01_raw/_converted/; with combineall also
installed, it appends the converted files into
02_cleaned/CountyBudgets_analytic.dta. Without them you get the full
folder tree, the do-files, and the documentation, and the run tells you what it
skipped and how to install it. See
Optional dependencies.
(If a source lives online, add url("https://.../data.csv"); it is fetched
now and the fetch is recorded in 100_data_download.do.)

Step 3. Open _code/000_control.do and run it to set the path
globals. Then check whether 02_cleaned/CountyBudgets_analytic.dta exists,
because the rest of the pipeline reads it. The "Next steps" block printed at the
end of Step 2 already tells you which case you are in. If the file exists, work
down from 300_labels.do. If it does not, convertanything or
combineall was missing, and the run printed the install command for
whichever one it could not find; install them and rerun with rebuild, or
run 200_data_management.do yourself. Skipping this check makes
300_labels.do stop with error 601 on its opening use.

Workflow B -- scaffold now, data later, rebuild on every refresh

You want the project structure now, before the data has arrived.

Step 1. Scaffold with no data() and no url(). You get the full
tree, an empty 01_raw/, and printed next steps:

. projectbuilder VendorFeed, description("Monthly vendor extract")

Step 2. When the files arrive, drop them into
VendorFeed/01_raw/.

Step 3. Rerun with rebuild. projectbuilder detects the new
files, re-runs the convert/combine pass, and regenerates the documentation:

. projectbuilder VendorFeed, rebuild

Every later refresh is the same rebuild. It is idempotent: it will not
overwrite a do-file you have edited in _code/ unless you add
replace, and the description, topic, and other metadata you recorded at
scaffold time are carried forward rather than reset to placeholders.

Optional dependencies

projectbuilder has no hard dependencies. A few companion tools make it do
more when installed; each is detected with capture which. If a
tool is missing, the generated do-file still contains the call, so it
remains a working example, and the step is skipped rather than failing.

Where you see the note depends on when the tool is needed.
convertanything, combineall, and webdoc2 do their work while
projectbuilder runs, so a missing one is named on screen along with its
install command. descsave and srctag/srcfind do their work
later, when you run 300_labels.do, so their notes live inside that file
and appear only when you run it.

convertanything bulk-convert mixed formats in 01_raw/ to .dta in 01_raw/_converted/ (author's GitHub)

combineall append or merge the converted files into the analytic file (author's GitHub)

descsave write a codebook (.dta plus .xlsx) from 300_labels.do (SSC: ssc install descsave)

srctag / srcfind tag and search each variable's source lineage (author's GitHub)

webdoc2 render a richer index.html (author's GitHub; needs ssc install webdoc, and see the note below)

webdoc2 needs one extra step. Its renderer injects a file called
header.html, and webdoc2 ships that as an ancillary file, so
net install webdoc2 does not place it:

. ssc install webdoc
. net install webdoc2, from("https://raw.githubusercontent.com/ericabooth/webdoc2-stata-public/main/") replace
. net get webdoc2, from("https://raw.githubusercontent.com/ericabooth/webdoc2-stata-public/main/") replace

The last line drops header.html into your current directory. builddocs
looks for it there, and on the adopath, before it renders, and carries a copy
into _documentation/ for the duration of the render. Put it somewhere on
your adopath -- your PERSONAL directory is the usual place -- and it will be
found from anywhere. Without it the render stops, projectbuilder says so
and names this remedy, and the built-in website/index.html is left
untouched.

Keep non-data files out of 01_raw/. The automatic pass hands
every file it finds there to convertanything, which reads delimited text
by extension. A README.txt or a notes file sitting beside the data is
therefore read as a dataset and appended into the analytic file, and a
.pdf or a file with no extension is skipped without comment. Two files
whose names differ only by extension, such as report.csv and
report.xlsx, both convert to report.dta and the second overwrites
the first. When the converted count does not match the raw count,
projectbuilder says so; r(nraw) and r(nconverted) let a
script check the same thing. Put documentation in _documentation/ and
leave 01_raw/ for data.

The analytic file is a straight append of the converted files and carries no
column saying which raw file each row came from. If you need that, add it in
200_data_management.do, or install srctag, which is what it is
for.

The convert-then-combine chain is the heart of 200_data_management.do:
convertanything turns every raw file into a cleaned .dta, then
combineall with cmethod(append) stacks those into one analytic file.
When webdoc2 is absent, projectbuilder writes a plain but complete
index.html and Readme.md directly, so the documentation always
exists.

What gets built

After projectbuilder MyProject you have:

MyProject/
+-- 01_raw/ raw source files (write-once)
| +-- _archive/
| +-- _converted/ convertanything output (.dta per raw file)
+-- 02_cleaned/ <project>_analytic.dta lives here
| +-- _archive/
+-- 03_output/ logs, tables, exhibits
| +-- _archive/
+-- _code/
| +-- 000_control.do every path in one place; run-all block
| +-- 100_data_download.do
| +-- 200_data_management.do convertanything -> combineall
| +-- 300_labels.do
| +-- 400_data_profiler.do
| +-- 500_aggregation.do
| +-- 600_analysis.do
| +-- _archive/
+-- _documentation/
| +-- index.do webdoc2 source
| +-- _runall.do renders website/index.html
| +-- _project_meta.txt recorded metadata, read back on rebuild
| +-- Readme.md
| +-- website/index.html
| +-- _archive/
+-- _archive/

The _archive/ folders are made for your convenience and are never written
to by projectbuilder. Nothing is moved into them automatically -- not by
rebuild, and not by replace, which overwrites an edited do-file in
place with no copy kept. Move anything you want to keep before running
replace, or keep _code/ under version control.

000_control.do pins the language version, stamps $root with the
absolute path of the new folder (one loudly commented line to edit if the
project ever moves), derives $raw, $converted, $cleaned,
$output, $code, and $docs from it, and ends with a run-all
block over the numbered pipeline.

The version pin is version 16.0, this package's own floor, not the release
of the Stata that generated the file. Pinning the generating release would write
something like version 19.5, which every earlier installation rejects, so
the control file would not run for a teammate on Stata 16 through 19. Raise the
pin by hand if the project comes to depend on newer syntax.

Recorded metadata

The metadata options are recorded once and reused. projectbuilder writes
description(), url(), topic(), publicfacing(),
timeline(), othernotes(), outcomes(), over(), and the
creation date into _documentation/_project_meta.txt, one key=value
per line. On any later call, each value the call does not supply is read back
from that file. A bare projectbuilder MyProject, rebuild therefore keeps
the description and topic it was given at scaffold time instead of replacing
them with placeholders, and rebuild replace rewrites
400_data_profiler.do with the outcomes() and over() names
already on record. A non-empty option on the current call always wins, so
rebuild description("...") is how you change a recorded value. An empty
one is treated as not supplied: rebuild topic("") does not clear the
topic, it re-reads the recorded one. To clear a value, delete it from
_project_meta.txt, which is plain text and meant to be edited
directly.

The Created row of the documentation is the date of the first scaffold,
carried in the same file, so a rebuild does not restamp it with today. The date
and time of the current build appear separately in the footer line of
index.html and Readme.md. That footer is the only line a rebuild is
expected to change when nothing else about the project has moved.

_documentation/index.do is guarded in the same way as the numbered
do-files: a rebuild without replace leaves it exactly as written, on
the assumption that you may have edited it. website/index.html and
Readme.md are derived artifacts and are rewritten on every call. Both
sides draw on the same recorded metadata, so a bare rebuild leaves them
agreeing.

They do diverge in one ordinary case: changing a value on a refresh, as in
rebuild topic("procurement"), rewrites index.html and
Readme.md with the new topic but leaves the guarded index.do
showing the old one. That is the guard doing its job, on the assumption that
you may have edited index.do yourself. Add replace to bring
index.do back in line, or edit it directly. The same asymmetry applies to
hand edits: an edit to index.do reaches index.html only when
builddocs renders it with webdoc2, whereas an edit to the generated
index.html is overwritten by the next call.

Your data in memory

projectbuilder leaves the dataset in memory exactly as it found it. That
is worth stating because the automatic pass does load data: convertanything
runs with clear, and combineall does its own use and
save. The command wraps that pass in preserve, so a scaffold or a
refresh in the middle of an analysis session does not cost you your data.

Two consequences follow. Nothing you have in memory is written into the
project, so the analytic file is built only from what is on disk in
01_raw/. And because preserve writes a temporary copy, a
rebuild while a very large dataset is loaded costs one round trip to
disk; noautoconvert skips the pass, and with it the preserve.

Characters in option values

The free-text options are written into do-files and documentation pages, which
puts three characters worth knowing about.

Backtick. Rejected, with an error naming the option. A backtick opens a
macro reference Stata cannot close, and it used to stop the command partway
through, after the folders had been made but before the do-files were written.
Type an apostrophe instead.

Dollar sign. Stata expands globals while it parses the command line, so
description("costs $M per year") reaches projectbuilder with the
$M already gone. This is true of every Stata command and cannot be
fixed here. Write \$M, or put the value in
_documentation/_project_meta.txt by hand, where it is read back
literally and survives every later rebuild.

Tilde. Passes through unchanged, including in a URL such as
url("https://example.edu/~Dave/data.csv").

Try it now

Every line below is clickable. The demo is built in your temporary directory,
so nothing lands in a folder you care about, and your working directory is
never changed. Click them in order; each step assumes the one before it. The
whole sequence is identical on Windows, macOS, and Linux.

If you, or anyone else on this machine, has walked through this before, Step 1
will stop with error 602: projectbuilder does not overwrite an existing
project, which is the point Step 6 makes on purpose. Add rebuild to
Step 1 and read the rest as a refresh, or delete the folder Step 2 names and
start over. On a shared server, give path() a directory of your own
instead of the temporary one.

Step 1. Scaffold a project with no data yet, in the temporary
directory Stata already uses. Read the "Next steps" block it prints:

. projectbuilder pbdemo, path("`c(tmpdir)'") description("A walkthrough") topic(demo) publicfacing(no)

Step 2. Keep the folder it built. r(path) is the absolute path
of the new project, so nothing below has to guess at separators or spell out a
directory:

. global pbdemo "`r(path)'"

. display "$pbdemo"

Step 3. Look at what it wrote. The control file is the one that
matters; every path in the project is derived from its $root:

. type "$pbdemo/_code/000_control.do"

. type "$pbdemo/_documentation/Readme.md"

Step 4. Put some data in 01_raw/, the way a real drop would
arrive:

. sysuse auto, clear

. export delimited using "$pbdemo/01_raw/auto.csv", replace

Step 5. Refresh. This is the command you rerun on every data
update, and the only one you need to remember:

. projectbuilder pbdemo, path("`c(tmpdir)'") rebuild

. describe

The describe still shows the auto data: the refresh did not disturb it.
If convertanything and combineall are installed, the run also built
02_cleaned/pbdemo_analytic.dta; if not, it printed the install command for
whichever one was missing and built everything else.

Step 6. Confirm that the guards work. Both of these are supposed to
fail. The first stops with error 602, because a plain call never overwrites an
existing project. The second stops with error 198, before any folder is
created, because publicfacing() takes only yes, no, or unsure:

. projectbuilder pbdemo, path("`c(tmpdir)'")

. projectbuilder pbdemo2, path("`c(tmpdir)'") publicfacing(maybe)

Step 7. Change one recorded value and watch the rest survive. The
topic changes; the description, timeline, and creation date do not:

. projectbuilder pbdemo, path("`c(tmpdir)'") rebuild topic("procurement")

. type "$pbdemo/_documentation/_project_meta.txt"

Step 8. Last, run the control file. It sets the path globals every
other do-file in the project relies on, and it is the only file you edit if the
project ever moves:

. do "$pbdemo/_code/000_control.do"

. macro list root raw cleaned

Run this one last, because the control file begins with clear all: it
drops the auto data the steps above were using. (clear all clears data
and programs but leaves globals alone, so $pbdemo survives it.) That is
the control file doing its job at the top of a pipeline, not something
projectbuilder does to you.

Nothing needs cleaning up. The demo sits in your temporary directory, which the
operating system clears on its own schedule. projectbuilder makes no
shell call and never deletes anything, so removing a project is always
something you do yourself.

Examples

Every example below runs as written, top to bottom, in one directory. The names
are all distinct, so nothing trips the clobber guard, and the rebuild
examples refresh a project an earlier line created. Paths use forward slashes,
which Stata accepts on Windows as well as macOS and Linux.

Scaffold an empty project in the current directory, then look at the
documentation it wrote:

. projectbuilder Ex01Feed, description("Monthly vendor extract")
. type "Ex01Feed/_documentation/Readme.md"

Scaffold under a named base directory instead of the current one. The
base is created if its parent already exists:

. projectbuilder Ex02Feed, path("projects")

Scaffold a subsource under a source; the project label joins them with an
underscore, so the analytic file is Agency_Extract_analytic.dta:

. projectbuilder Agency/Extract, description("One agency extract")

Quote a name that contains a space. An unquoted second word is an error,
not a longer folder name:

. projectbuilder "Vendor Feed 2027", description("Quoted name")

Refresh a project after new files land in 01_raw/. The recorded
metadata is kept, and edited do-files in _code/ are left alone:

. projectbuilder Ex01Feed, rebuild

Change one recorded value on a refresh; everything else stays as it
was:

. projectbuilder Ex01Feed, rebuild topic("procurement")

Refresh and reset the numbered do-files to the shipped templates,
discarding your edits to them:

. projectbuilder Ex01Feed, rebuild replace

Seed the profiler with the variables you already know you care about.
They are recorded, not validated, so it is safe to name variables the analytic
file does not have yet:

. projectbuilder Ex03Rates, ///
outcomes(enroll_rate cost_pp) over(year region)

Use the stored results to drive the next step of a script rather than
retyping the path:

. mkdir "drop"
. sysuse auto, clear
. export delimited using "drop/auto.csv", replace
. quietly projectbuilder Ex04Auto, data("drop")
. display "built " r(project) " with " r(nraw) " raw file(s)"
. display "it lives at " r(path)

Refresh a whole set of projects on a schedule. Each call is idempotent,
so this is safe to rerun:

. foreach p in Ex01Feed Ex03Rates Ex04Auto {
2. capture noisily projectbuilder `p', rebuild
3. }

Scaffold and ingest in one call: copy a folder of files in, record every
piece of metadata at once, and seed the profiler. This reuses the drop
folder made two examples above:

. projectbuilder Ex07Budgets, ///
data("drop") ///
description("County budget CSVs, one row per dept per FY") ///
topic("local government, budgets") publicfacing(unsure) ///
timeline("annual") outcomes(total_budget) over(year dept) ///
descsave

Record a source URL. It is fetched now if the address is reachable, and
the fetch is written into 100_data_download.do either way. The address
below is a placeholder, so expect the fetch to miss and the scaffold to be
built anyway; that is the behavior to rely on when a source is temporarily
down:

. projectbuilder Ex08Open, url("https://example.com/data.csv")

Scaffold without running the convert/combine pass, so
01_raw/_converted/ and the analytic file are not built. Files named in
data() are still copied into 01_raw/. This is also the fastest
option when a large dataset is loaded, since it skips the preserve the
pass would otherwise do:

. projectbuilder Ex05Feed, noautoconvert

Rebuild and render the documentation with webdoc2 when it is
installed:

. projectbuilder Ex01Feed, rebuild builddocs

Write a literal dollar sign. Stata expands $total before
projectbuilder ever sees it, so escape it:

. projectbuilder Ex06Cost, description("Reported in \$ millions")

Stored results

projectbuilder is rclass and stores:

Scalars

r(nraw) number of files in 01_raw/

r(nconverted) number of .dta files in 01_raw/_converted/

r(rebuilt) 1 if this call refreshed an existing project, else 0

Macros

r(project) project label (slashes become underscores)

r(path) absolute path of the project folder

Authors

Eric A. Booth, Sr Researcher, Texas 2036

Support: eric.a.booth@gmail.com

Elizabeth Teas, Sr Research Scientist, Far Harbor, LLC

elizabeth@farharbor.com

Companion package to Applied Program Evaluation Using Stata.