*! version 1.2.3 06may2026

Title

xtbalfront -- Trade-off frontier diagnostics and balanced-subsample
selection for unbalanced panel data

Syntax

xtbalfront varlist [if] [in],
id(varname)
time(varname)
[options]

options

Required
id(varname) numeric cross-section identifier

time(varname) numeric time identifier

Subsample target (optional, mutually exclusive)
years(#) choose a subsample of # balanced years

crosssections(#) choose a subsample with at least # balanced cross-sections

Anchor (optional - constrains where the balanced window may sit)
endperiod(#) window must end at period #

startperiod(#) window must start at period #

recent window must end at the most recent period (shortcut)

Apply the subsample
generate(newvar) create a 0/1 indicator marking the chosen subsample

keep drop observations not in the chosen subsample

dryrun show diagnostics for the proposed subsample without applying generate() or keep

Mode
any use per-firm survival count instead of consecutive-window logic

Output controls
nograph suppress all graphs

noheatmap suppress the data presence/absence heatmap

nohist suppress the distribution-of-availability histogram

nolist suppress the trade-off tables and L x N peak printout

nocompare suppress the full-vs-subsample comparison table

nohistcompare suppress the per-variable pre/post histogram plots

nogapnote suppress the internal-gap diagnostic and xtmispanel note

nosellist suppress the list of selected cross-sections

maxlist(#) maximum cross-sections to print in the selected list (default 50)

maxshow(#) maximum cross-sections to render in the heatmap (default 200)

Cosmetic
lcolor(colour) line colour for the best-window curve

mcolor(colour) marker colour for the best-window curve

Saving
savematrix(matname) save the best-window trade-off curve as a Stata matrix

saverecent(matname) save the anchored trade-off curve as a Stata matrix

Description

xtbalfront examines an unbalanced panel and characterises the trade-off
between balanced length (number of years) and balanced breadth (number of
cross-sections) implied by the pattern of missing data on a user-supplied
varlist. An observation is treated as valid if every variable in
varlist is non-missing for that cross-section / period.

The command gives the analyst high flexibility in choosing the specific
balanced window - any length, any anchor, optionally pinned to the most
recent period - and a quantitative read on how that choice differs
distributionally from the original unbalanced data. The default invocation
runs every diagnostic at once: panel structure, both trade-off curves
(best-window and recency-anchored), the L x N peak, a presence/absence
heatmap, a per-firm availability histogram, and an internal-gap note.

By default the command produces, in a single call:

1. A panel structure block via xtset +
xtdescribe on the if/in sample, so the analyst sees
the panel before any subsetting decision.

2. A trade-off table listing, for every balanced length
L = 1, 2, ..., the maximum number of cross-sections that can form a balanced
panel of that length, together with the start and end of the optimal
window.

3. A second trade-off table for the recency-anchored case,
in which the window is held fixed against the most recent period in the data.
This second table answers the question "if I want my balanced panel to end at
the latest available year, how many cross-sections survive at each
length?"

4. Two trade-off plots, one per curve, each with the L x N
peak (the largest balanced sample) marked. The plots are produced as named
graphs (xtbalfront_best and xtbalfront_recent or
xtbalfront_anchored) so the analyst can recall, modify or export them
independently.

5. A heatmap of data presence/absence across cross-sections
and time, suppressed automatically when the panel exceeds maxshow()
cross-sections.

6. A histogram of the per-cross-section count of valid
years.

7. An internal-gap diagnostic flagging cross-sections whose
valid observations are non-contiguous, with a suggestion to consider
xtmispanel or interpolation if appropriate.

When years() or crosssections() is supplied, the command additionally:

8. Identifies the implied balanced sub-panel and reports L, N
and the window endpoints.

9. Reports a comparison table of mean and standard deviation
for every variable in varlist, with a Welch t-test of the difference between
the proposed subsample and the cross-sections that would be discarded. A
small p-value warns that the subsetting is removing systematic variation, not
just observations.

10. Produces an overlaid pre/post histogram plot for each
variable in varlist, comparing the full sample with the selected subsample
and reporting the percentage change in the mean in the plot's note.

11. Lists the selected cross-sections (with value labels where
present), capped at maxlist().

12. Optionally creates an indicator variable (generate())
or restricts the dataset in memory (keep). Specifying dryrun
shows everything except the actual subsetting, so the analyst can review the
diagnostics before committing.

The default command does not restrict the dataset. Use generate()
or keep to materialise a subsample.

Options

Required

id(varname) numeric cross-section identifier (e.g. country,
firm, individual). Must be numeric.

time(varname) numeric time identifier (e.g. year, quarter
encoded as a Stata numeric). Must be numeric.

Subsample target

years(#) requests a balanced panel of exactly # years. The
command picks the L = # row of the trade-off curve and identifies the
optimal cross-sections that survive that window. When an anchor option is
also supplied, the anchored trade-off curve is used.

crosssections(#) requests a balanced panel containing at
least # cross-sections. The command picks the longest balanced
length consistent with that target.

years() and crosssections() are mutually exclusive.

Anchor

endperiod(#) forces the balanced window to end at period
#.

startperiod(#) forces the window to start at period
#.

recent is shorthand for endperiod(max-time). Equivalent
to "anchor at the most recent period in the data".

With no anchor option, the second curve drawn alongside the best-window curve
is automatically the recency-anchored one (so default behaviour matches the
common expectation: "best obtainable" vs. "what survives if I insist on the
latest period").

Apply the subsample

generate(newvar) creates a new 0/1 indicator equal to 1 for
observations in the chosen balanced subsample, 0 otherwise. Non-destructive.

keep removes from memory any observation outside the subsample.
Destructive - use after dryrun or in a preserve block.

dryrun shows the trade-off curves, the gap note, the comparison table,
the per-variable histograms and the selected-cross-section list, but does
not create the indicator variable or drop any rows. Re-run without
dryrun to apply.

Mode

any switches from consecutive-window logic to a per-firm survival count.
The y-axis of the trade-off then shows "cross-sections with at least L valid
years anywhere in time" rather than "cross-sections whose L valid years are
consecutive". Use any only if your downstream estimator does not
require consecutive observations.

Output controls

nograph, noheatmap, nohist, nolist,
nocompare, nohistcompare, nogapnote,
nosellist suppress the corresponding output element.

maxlist(#) controls how many cross-sections are printed in the
selected list before truncation; default 50.

maxshow(#) caps the heatmap; if the panel has more cross-sections
than this, the heatmap is suppressed (the histogram still appears).
Default 200.

What each chart and table shows

Panel structure block. Reports the panel variable, time variable, time
range, and the distribution of T_i (number of periods per cross-section) via
xtset and xtdescribe. Read this first to confirm the panel is
the shape you expect.

Trade-off plots. The horizontal axis is the balanced window length L
(in periods). The vertical axis is the number of cross-sections that can
form a balanced panel of that length. Two separate plots are produced:

Best-window plot. For each L, the best obtainable count when
the window may sit anywhere in time. Reading right means asking for a
longer balanced span, so the count weakly falls; the L x N peak (red diamond)
is the L at which L * N is maximised - i.e. the most data-rich balanced
sample. Saved as the named graph xtbalfront_best.

Recency-anchored or user-anchored plot. For each L, the count
when the window's endpoints are forced. With no anchor option, the window
is held to end at the most recent period; reading right then means
extending the window further back from the latest year. With
endperiod() or startperiod(), the anchor is wherever you put
it. Saved as the named graph xtbalfront_recent or
xtbalfront_anchored.

A caption beneath each plot reminds the reader of these
conventions. Plot titles are intentionally omitted; the named-graph
mechanism lets the analyst add a title via
graph display if desired.

Trade-off tables. Numerical equivalents of the curves, one row per L,
showing the start and end periods of the optimal window for that length.

L x N peak printout. The single largest balanced sample for each curve,
printed as L = ? N = ? L*N = ? window = ? to ? Use these numbers as the
arguments to a follow-up call: xtbalfront ..., years(L) gen(insample).

Heatmap (presence/absence). A matrix view: each row is one cross-section
(sequentially indexed), each column is a period, each square is filled (navy)
if the row has all variables present in that period and hollow (red) if not.
The heatmap is the fastest way to see whether unbalancedness is
concentrated (e.g. all attrition at the end) or scattered. Suppressed
automatically when the panel has more than maxshow() cross-sections.

Distribution histogram. A frequency histogram of the count of valid
years per cross-section. A bimodal distribution typically indicates two
populations of firms (e.g. survivors vs. short-lived); a long left tail
suggests entrants and exits.

Internal-gap diagnostic. For every cross-section we locate the first
and last valid period and ask whether all interior periods are valid too.
If not, the cross-section has an internal gap. The note reports the count
of gappy cross-sections and the total number of interior missing values -
i.e. how many observations interpolation could potentially recover. The
note suggests xtmispanel, mipolate or ipolate as
candidates but does not apply them.

Comparison table (full vs proposed subsample). For each variable in
varlist we report the mean and standard deviation in the full if/in
sample and in the proposed subsample, plus a Welch t-statistic for the
difference between the subsample and its complement (the cross-sections
that would be discarded). This is a warning test, not a randomness
test: significant differences mean the subsetting is removing systematic
variation.

Pre/post histograms. One overlaid plot per variable in varlist (named
xtbf_h_varname, with varname truncated to 24 characters
when needed to fit Stata's 32-character graph-name limit), showing the
density of the full sample (navy) on top of the selected subsample
(cranberry). The plot's note reports the percentage change in the mean
and the two mean values themselves. This is the visual companion to the
comparison table - a small mean shift may still hide a meaningful change
in the distribution's shape.

Selected cross-section list. Once a target is given, the unique IDs
that make the cut are printed, with their value labels where attached. Long
lists are truncated at maxlist().

Recommended workflow

Step 1. Run xtbalfront with no target. Inspect the panel
structure, both curves, the heatmap, the histogram and the gap note.

Step 2. Decide on a target. If you care about a specific length
(say, six years) use years(6). If you care about breadth (say, at
least 1000 firms) use crosssections(1000). If the most recent period
matters substantively, add recent.

Step 3. Re-run with the target and dryrun. Read the
comparison table and the pre/post histograms. If a t-test for any key
variable is large or the histograms shift visibly, the balanced-subsample
estimates may not generalise.

Step 4. Re-run without dryrun, supplying generate()
(non-destructive) or keep (destructive). Estimate your model on the
full and selected samples and compare the substantive results.

Examples

Setup:

. webuse nlswork, clear

. xtset idcode year

Diagnostics only - panel summary, both curves, heatmap, gap note:

. xtbalfront ln_wage grade union, id(idcode) time(year)

Pick a six-year balanced panel using the best window; preview only:

. xtbalfront ln_wage grade union, id(idcode) time(year) years(6) gen(in6) dryrun

Same, but anchored at the latest year and applied:

. xtbalfront ln_wage grade union, id(idcode) time(year) years(6) recent gen(in6_recent)

Pick the longest panel containing at least 1000 cross-sections, drop the rest:

. xtbalfront ln_wage grade union, id(idcode) time(year) crosssections(1000) keep

Anchor at a specific period:

. xtbalfront ln_wage grade union, id(idcode) time(year) years(6) endperiod(86) gen(in_anchored)

Save both curves to matrices for further work:

. xtbalfront ln_wage grade union, id(idcode) time(year) savematrix(C_BEST) saverecent(C_REC)

Stored results

xtbalfront stores the following in r():

Scalars

r(total_cross) number of unique cross-sections in the if/in sample

r(nodata_cross) cross-sections with no valid observation

r(max_years) number of unique periods in the data

r(gap_count) cross-sections with at least one internal gap

r(gap_severity) total interior missing values across all cross-sections

r(peak_best_L) L at the best-window L x N peak

r(peak_best_N) N at the best-window L x N peak

r(peak_anc_L) L at the anchored L x N peak (when available)

r(peak_anc_N) N at the anchored L x N peak (when available)

r(selected_years) L of the chosen subsample (when targeted)

r(selected_cross) N of the chosen subsample (when targeted)

r(selected_start) window start of the chosen subsample

r(selected_end) window end of the chosen subsample

r(selected_obs) L * N of the chosen subsample

Macros

r(mode) window or any

r(anchor) description of the anchored curve, when shown

Matrices

r(curve) best-window trade-off curve (L, N, start, end)

r(curve_anchored) anchored trade-off curve (L, N, start, end)

Methodological note

Why the best-window curve is monotone weakly decreasing. Any window of
length L+1 contains a window of length L; the cross-sections that are valid
in the longer window are a subset of those valid in some length-L window.
Hence N(L+1) <= N(L).

Recency-anchored vs best-window. The recency-anchored curve answers a
substantively different question. If your downstream analysis requires the
panel to end at the most recent period (because you're projecting forward
from "now"), the best-window count for length L is irrelevant - what
matters is how many cross-sections survive the L most recent years. This
is why xtbalfront draws both by default.

Acknowledgments

xtbalfront uses Stata's built-in xtset and xtdescribe to
print the panel-structure summary at the top of every run. Both are part
of the xt suite shipped with Stata; this command is grateful for
their availability.

Author

Noman Arshed, Sunway Business School, Sunway University, Malaysia.

Email: nomana@sunway.edu.my

Web: https://econistics.com

Bug reports and feature requests are welcome via email or the
Statalist forum.

Also see

Stata: xtdescribe, xtset, xtreg

SSC: xtmispanel