*! version 1.0.0 2026-07-15

Title

reshapehelper -- Diagnose the data and suggest (never run) the likely reshape command

Syntax

reshapehelper [long|wide] [varlist] [, options]

Tell it what you know (all optional)
to(long|wide) the shape you want to END with; bare long/wide before the comma means the same thing

i(varlist) the identifier variable(s), if you know them

j(name|varname) going long: the NAME for the new j variable; going wide: the EXISTING j variable

stubs(names) the variable stubs, with @ allowed, when the scanner cannot find them

Control
sample(#) rows used for the dry run and probes; default sample(500)

notest skip the preserve/restore dry run

smcl(filename) write the unwrapped suggestion to this .smcl file (default: a session temp file)

replace overwrite an existing smcl() file

global(name) global macro to hold the suggestion; default $reshapehelper_cmd

detail also show the pattern scan and the i/j candidate lists

The marked stem of an option name is its minimum abbreviation, so
stubs() may be typed stub(), sample() samp(),
notest notes, global() gl(), and detail
d. The option names shown unmarked must be typed in full.

A bare reshapehelper with no arguments diagnoses the dataset in memory.

Description

reshape is the Stata command most likely to send an analyst to a
discussion board: is this data wide or long, which variables are i and j,
where does the string option go, why r(9), why
no xij variables found?
reshapehelper answers those questions before anything is
reshaped. It never modifies your data.

It works in four moves:

1. Scan the variable names for wide patterns: numeric suffixes
(inc80 inc81), string suffixes after an underscore
(bp_before bp_after), a j buried mid-name (inc80r - the
@ notation), a j hiding in the PREFIX (qld_p nsw_p vic_p), and
doubly-wide names carrying two indices (ht_k1_t2).

2. Hunt for i and j. Declared structure comes first: an
xtset/tsset panel and time variable and the svyset
primary sampling unit are taken as the analyst's own statement of the
design. Then come uniqueness tests (isid) over id-looking variables
and, when those fail, bounded searches over single variables and pairs
- which is how a compound i like (household, year) is found.

3. Compose and dry-run the command on up to sample() rows
inside preserve/restore. When the test fails, it iterates on
reshape's own error codes: r(109)/r(498) add the string option,
r(110) renames a colliding j, r(9) in a widening triggers a duplicates
diagnosis and one attempt to extend i() with a distinguishing variable.

4. Report, as described next. For a doubly-wide layout it
suggests (and tests) the two chained reshape commands. When the job is
really a transpose - observations stored as columns - it says so
and points at xpose (sxpose from SSC for string data).

What you get back

1. A diagram. A boxed before/after sketch in the style of the
reshape help file, built from your actual variable names and sample
values. The AFTER box is real: it shows the dry run's result, not a
mock-up.

2. The suggested syntax, with a verdict. The command line(s),
marked dry-run PASSED or FAILED r(#), plus any note (for
example, that i() keeps a time variable because the rows are a wide-long
panel) and any caution (for example, mixed suffix widths that smell
like the inc2-beside-inc80 false match). When nothing can be
composed, you get a numbered checklist of what to fix or supply instead
- confirm the row id, unify near-miss stubs, inspect duplicates, fold
crossed factors into one j with egen concat(), or state to(),
i(), j() directly.

3. The command, ready to use. Stored in r(cmd) (and
r(cmd2) for a chained pair), in the global $reshapehelper_cmd
(and $reshapehelper_cmd2), written unwrapped to a viewable SMCL file,
and offered as click-to-run and click-to-view links in the Results window.
Run it as-is with:

. $reshapehelper_cmd

Options

to(long|wide) states the target shape. Without it,
reshapehelper infers direction from the evidence: wide naming
patterns argue for reshape long; an (i, j) pair that uniquely
identifies rows argues the data are long and offers the widening. A bare
long or wide before the comma is shorthand for this option.

i(varlist) names the identifier(s) and skips the i
hunt. Supply it whenever you know it; it is the single biggest help you
can give.

j(name|varname) is direction-dependent, exactly as
in reshape: going long it NAMES the new variable to create; going
wide it names the EXISTING variable whose values become suffixes.

stubs(names) hands over the stubs when the scanner
cannot find them - above all for bare string suffixes like
incm/incf, which reshapehelper deliberately does not guess
(the reshape manual's own warning: beside an agenda variable, a
greedy scan would happily read stub age with j nda). @
notation is allowed.

sample(#) caps the rows used for value probes and
the dry run (default 500, minimum 10). The suggestion is composed from the
sample; on very tall data a larger sample buys confidence at the cost of
speed.

notest skips the dry run; the suggestion is then composed from
name patterns and identifier tests alone and printed unmarked.

smcl(filename) and replace persist the
unwrapped suggestion beyond the session; the default writes to a temp-dir
file that the click to VIEW link opens. A filename that does not
already end in .smcl has that extension appended, so
smcl(mysuggestion) writes mysuggestion.smcl; the path actually
used is returned in r(smcl).

global(name) renames the global holding the
suggestion (a chained second step lands in name2).

detail adds the scanner tallies and the i/j candidate lists to
the header - useful when the suggestion surprises you.

Examples

Tier 1 - the textbook case (reshape Example 1):

. input id sex inc80 inc81 inc82 ...

. reshapehelper

suggests and dry-runs reshape long inc, i(id) j(year)

. $reshapehelper_cmd

Tier 2 - string suffixes, string added for you:

. sysuse bpwide, clear

. reshapehelper

suggests reshape long bp_, i(patient) j(period) string

Tier 3 - tell it what you want and let the dry run find the rest
(here it discovers string by iterating on reshape's r(498)):

. input id kids incm incf ...

. reshapehelper, to(long) stubs(inc) i(id) j(sex)

Tier 4 - duplicates block a widening; get the remedy menu, not a
guess:

. reshapehelper, to(wide)

diagnosis counts the (i, j) collisions and offers: duplicates report / collapse / sequence with bysort ... gen seq = _n / fold a factor into j with egen concat()

Tier 5 - the wide-long panel: county-year rows whose sector
columns hide a second long dimension (note that i() keeps year):

. xtset county year

. reshapehelper

suggests reshape long emp_, i(county year) j(period) string

Tier 5 - doubly wide (UCLA OARC FAQ): two chained commands, both tested:

. input famid ht_k1_t1 ht_k1_t2 ht_k2_t1 ht_k2_t2 ...

. reshapehelper

step 1 reshape long ht_k1_t ht_k2_t, i(famid) j(time)

step 2 reshape long ht_k@_t, i(famid time) j(grp)

The bundled example_reshapehelper.do walks seventeen scenarios
across five tiers, from the textbook case through inconsistent stubs, the
inc2 trap, prefix-j names, crossed factors, spaces in a string j,
duplicate (i, j) pairs, doubly-wide names, long-long to wide-wide, and the
transpose case.

Stored results

reshapehelper stores the following in r():

r(status) ok (suggestion composed), blocked (composed but the dry run failed; see r(diagnosis)), or needinfo (checklist shown)

r(direction) wide2long, long2wide, doubly, or unknown

r(cmd), r(cmd2) the suggested command(s); r(cmd2) when applicable (a chained doubly-wide pair)

r(preclean) when applicable, a data-cleaning line to run first (for example, stripping spaces from a string j)

r(i), r(j), r(stubs) the pieces the suggestion uses

r(icands), r(jcands) the candidate lists considered

r(note), r(caution), r(diagnosis) the annotations printed under the suggestion, each when applicable

r(smcl) path of the SMCL suggestion file, returned only when that file was written - that is, when r(status) is ok or blocked; a needinfo run writes no file and returns no r(smcl). The path carries the .smcl extension that smcl() appends when the filename you supply lacks it, so it need not match what you typed

r(tested) 1 if the dry run passed, else 0

r(rc) the final dry-run return code

r(xpose) 1 when the layout smells like a transpose job

r(sparse) a would-be identifier that was set aside as too sparse (returned only when that is why no command was found)

A macro that does not apply to a run is not stored at all, so it is
absent from return list rather than returned empty, and which macros
survive varies with the run. A needinfo run, for instance, has no
r(cmd) and no r(smcl), yet may still report the piece it did
settle on in r(j) beside its r(diagnosis). Test a macro for
emptiness rather than assume it is there.

The suggestion also lands in $reshapehelper_cmd (and
$reshapehelper_cmd2); rename with global(). The globals mirror
the LAST run: a run with no second step clears the 2 global, and a
run that ends in the checklist clears both, so a stale command from an
earlier dataset can never fire.

Remarks

Suggestions are advisory. A dry run proves the command RUNS on a
sample; only you can say whether the result MEANS what you need. Two
built-in skepticisms help: small numeric suffixes (none above 12) draw a
caution that they may be item numbers rather than repeats over time, and
mixed suffix widths (a stray inc2 beside inc80-inc82)
draw a caution to restrict j explicitly or rename the stray - a case
where reshape itself runs happily and silently builds a mostly-missing j
group.

Why a stub is sometimes refused (name coincidences). A program
reading only variable names cannot reliably tell a real stub from a
coincidence. With variables incm incf uem uef
agem agef plus an unrelated agenda, an automatic split
would decide the repeated values are m, f, and nda and
read a stub age out of agenda - the greedy match the
reshape manual itself warns about. To avoid confidently-wrong
suggestions, reshapehelper does not auto-detect bare letter-suffix
stubs; you name the stub with stubs() and it works out the rest,
including the string option, from the dry run. Triple-nested single
commands are likewise never suggested; doubly-wide names get the two-step
chain, and deeper nesting gets the chain plus a rerun of
reshapehelper on the intermediate result.

The identifier plausibility bar (and sparse panels). When it
hunts for the row identifier, reshapehelper requires that a candidate
repeat: each unit should appear in more than one row (once per year,
wave, or condition), so a candidate that splits the sample into nearly as
many groups as there are rows is set aside as an accident of the data rather
than a design. Concretely, the distinct groups must number at most half the
rows examined. On sysuse auto the rule sets aside make: it does
pair with mpg to identify the rows uniquely, but it names a different
car in every row, so the run ends in the checklist with r(sparse)
reporting make rather than in a technically-valid but meaningless
widening on i(make) j(mpg). The one trade-off:
a genuinely sparse panel - most units observed once, only a few
repeated - can be sent to the checklist instead of getting a
suggestion. When that happens the checklist says so by name and offers to
force the identifier (reshapehelper, i(thatvar) ...); if
most rows really should share an id, the true identifier may be missing or
mis-typed (for example a name with trailing spaces), so clean it first.

The j-candidate ceiling. A second bar applies to j when the data
are being widened. A variable enters the j candidate list only if it takes
at least 2 distinct values and at most 60 or half the sampled rows (rounded
up), whichever is smaller, so beyond the half-the-rows rule there is an absolute
ceiling of 60 distinct j values: a variable with more than 60 levels is
read as an identifier or a measurement rather than an index, and is not
proposed as j even in a very tall dataset. The fallback pass, which runs
when no candidate carries a time-like name (year, wave,
visit, and so on), is stricter still and stops at 30 distinct values.
Naming the variable in j() bypasses both limits, which is the way to
widen on a high-cardinality index deliberately - bearing in mind that
each level becomes its own set of columns.

Rerun it. reshapehelper is built to be run again on its own
output: a wide-long panel widens one dimension per pass, and the note tells
you when another dimension remains.

Sampling. All probes and the dry run use the first
sample() rows, so a duplicate or a rename that only appears deep in a
tall dataset can escape the diagnosis; the final authority is running the
suggestion yourself (and, after a failure, reshape error, which lists
the offending observations).

Author

Eric A. Booth

Sr Researcher, Texas 2036

eric.a.booth@gmail.com

MIT License. Report issues at the GitHub repository.

Also see

Manual: [D] reshape

Help: reshape, xpose, stack, sxpose (SSC)