*! version 1.4.0 16Jan2026

Title

unicefdata_sync -- Sync UNICEF metadata from SDMX API

Syntax

unicefdata_sync
[, options]

What to sync
all sync all metadata types (default)

dataflows sync dataflows only

codelists sync codelists only

countries sync country codes only

regions sync regional codes only

indicators sync indicators only

history display sync history

Options
path(string) directory for metadata files

suffix(string) suffix for output filenames (e.g., "_stataonly")

verbose display detailed progress

force force sync even if cache is fresh

forcepython force use of Python parser

forcestata force use of pure Stata parser

enrichdataflows add dataflow mappings to indicator metadata

fallbacksequences also generate fallback sequences file

Description

unicefdata_sync synchronizes metadata from the UNICEF SDMX Data Warehouse
API to local YAML files. This includes dataflow definitions, codelists,
country codes, regional aggregates, and indicator mappings.

All generated YAML files follow the standardized _unicefdata_<name>.yaml
naming convention and include watermark headers matching the R and Python
implementations. Files are saved to src/_/ alongside the helper ado files.

Prerequisites: Some features require Python 3.6+ with specific packages:

• forcepython: Requires lxml package

• enrichdataflows: Requires requests package

Install via: pip install lxml requests

Note: For complete metadata extraction, especially for large XML files
like dataflow schemas, consider using the Python-assisted version or the
Python library directly. See Limitations below.

Generated Files

The following files are created in the src/_/ directory:

_unicefdata_dataflows.yaml SDMX dataflow definitions (69 dataflows)

_unicefdata_codelists.yaml Valid dimension codes (sex, age, wealth, etc.)

_unicefdata_countries.yaml Country ISO3 codes from CL_COUNTRY (453 codes)

_unicefdata_regions.yaml Regional aggregates from CL_WORLD_REGIONS (111 codes)

_unicefdata_indicators.yaml Indicator catalog (733 indicators)

_unicefdata_sync_history.yaml Sync timestamps and version history

_dataflows/*.yaml Dataflow schemas with dimension info (69 files)

Watermark Format

Each YAML file includes a _metadata block with:

_metadata:
platform: stata
version: '2.0.0'
synced_at: '2025-12-05T10:00:00Z'
source: <API URL>
agency: UNICEF
content_type: <type>
<counts>

Options

What to sync

all syncs all metadata types. This is the default behavior if no specific
type is selected.

dataflows syncs only the dataflows metadata from the API.

codelists syncs only the codelists metadata (excluding CL_COUNTRY and
CL_WORLD_REGIONS which are handled separately).

countries syncs only the country codes from CL_COUNTRY.

regions syncs only the regional aggregate codes from CL_WORLD_REGIONS.

indicators syncs only the indicator catalog metadata.

history displays the sync history without performing any sync operation.

Options

path(string) specifies the directory where metadata files should be saved.
If not specified, the command auto-detects the package installation directory.

suffix(string) appends a suffix to all output filenames before the .yaml
extension. This is useful for generating separate metadata files using different
parsers. For example, suffix("_stataonly") creates files like
_unicefdata_dataflows_stataonly.yaml.

verbose displays detailed progress messages including file names and counts.

force forces a sync operation even if the cached metadata is still fresh
(less than 30 days old).

forcepython forces use of the Python-based XML parser. Requires Python 3.6+
with the lxml package installed.

forcestata forces use of the pure Stata parser. No external dependencies
required but may be slower for large files.

enrichdataflows queries the API for all 70 dataflows and adds a dataflows
field to each indicator in _unicefdata_indicators_metadata.yaml. This shows
which dataflow(s) contain each indicator. Requires Python with the requests
package installed. Takes approximately 1-2 minutes to complete.

fallbacksequences also generates _dataflow_fallback_sequences.yaml which
maps indicator prefixes to dataflow sequences. Only works with enrichdataflows.
This file is used by unicefdata for automatic dataflow detection.

Examples

Basic sync with minimal output

. unicefdata_sync

Sync with detailed progress

. unicefdata_sync, verbose

Sync all metadata types

. unicefdata_sync, all

Sync indicators only

. unicefdata_sync, indicators

Sync indicators with dataflow enrichment (~1-2 min)

. unicefdata_sync, indicators force enrichdataflows

Sync indicators with enrichment and fallback sequences

. unicefdata_sync, indicators force enrichdataflows fallbacksequences

Sync dataflows only

. unicefdata_sync, dataflows

Sync countries only

. unicefdata_sync, countries

View sync history

. unicefdata_sync, history

Force sync even if cache is fresh

. unicefdata_sync, force verbose

Sync to specific directory

. unicefdata_sync, path("./metadata") verbose

Generate Stata-only metadata with suffix

. unicefdata_sync, suffix("_stataonly") verbose

Known Limitations

⚠️ Stata Macro Length Limits (Error 920)

Stata has internal limits on macro string length that can cause issues when
parsing large XML files from the UNICEF SDMX API. This affects:

_dataflow_index.yaml May have an empty or incomplete dataflows list

_dataflows/*.yaml Individual schema files may not be generated

These limitations occur because:

1. Large XML responses exceed Stata's local macro limit (~32,768 characters)

2. The pure Stata XML parser must accumulate content in macros

3. Some UNICEF dataflow schemas contain extensive metadata

Recommended Solutions:

Option 1: Use Python-assisted extraction (recommended)

The standard unicefdata_sync command (without suffix) uses Python
for large XML files when available. This provides complete metadata extraction:

. unicefdata_sync, verbose

Option 2: Use the Python library directly

For complete control and guaranteed full extraction:

python:

from unicef_api.schema_sync import sync_all

sync_all()

end

Option 3: Use the R package

The R implementation handles large XML files without macro limitations:

In R: unicefData::sync_all_metadata()

Metadata Consistency

The unicefData package is designed to generate consistent metadata across Python,
R, and Stata. All files should have matching record counts and metadata headers.
If you notice discrepancies, regenerate using the Python or R implementations
as the reference.

Stored results

unicefdata_sync stores the following in r():

Scalars

r(dataflows) number of dataflows synced

r(indicators) number of indicators synced

r(codelists) number of codelists synced

r(countries) number of country codes synced

r(regions) number of regional codes synced

Macros

r(vintage_date) date of this sync (YYYY-MM-DD)

r(synced_at) ISO 8601 timestamp of sync

r(path) path to metadata directory

Author

Joao Pedro Azevedo, UNICEF

jpazevedo@unicef.org

https://jpazvd.github.io/

Part of the unicefData package for accessing UNICEF Data Warehouse.

Also see

Online: UNICEF Data Warehouse,
UNICEF SDMX API

Help: unicefdata, yaml, wbopendata (similar command for World Bank data)