Title

collect -- Concatenate multiple files

Syntax

collect path_el [; path_el ...] [, append keep(varelist) pass(options) test]

Wildcards in path_el can be used to specify multiple files in a directory.

Description

collect concatenates multiple files into a single dataset in memory. The files may be in a mix of formats (though some formats would require StatTransfer). Variables can be combined and renamed.

Before collecting any data, collect gives a fairly detailed report of what it is about to do. It shows, for each data file that will be collected:

o- The paths and assigned id numbers

o- The number of variables that will be kept

(as a link showing all variables, and highlighting the ones to be kept)

o- Any naming/renaming errors

and then:

o- Anything specified in keep() that was not found in any dataset.

If there are errors, all of them will be reported before the command aborts.

collect adds a variable, _file, identifying the file each observation came from. _file contains id numbers, and is labeled with the file names.

Options

append causes the specified files to be appended to the data in memory, rather than replacing it.

keep(varelist) specifies the variables to be kept from any of the files collected. The variables in keep() do not need to be present in every (or indeed any) file.
If they are present in any of the collected files, they will be kept in the final data file.

varelist allows modifiers for combining/renaming variables:

Modifier Description

(-> varname) Rename any modified variables to varname

For example:

collect f*, keep(Rob*(-> Bobby) A-Z )

would collect all files starting with f. From each of those files, all variables starting with Rob, and from A to Z would be kept.
All variables starting with Rob (eg, Rob, Robby, Robert) would be renamed to Bobby.

If multiple variables from the same dataset would end up with the same name, an error will be generated.

test causes collect to report on what it would do (ie, which files it would use, variables used or not found, any errors, estimate of observations), without actually collecting the data.

pass() passes import/export options along to the appropriate handler.

o- For file extensions .txt or .csv, the options are those for import delimited.

o- For file extensions .xl, .xls or .xlsx, the options are described under portel xl.

o- For other file extensions (besides .dta), the options are those for callst.

Examples

Using unrelated semicolon-separated paths:

. collect a:/one/path.dta; b:/another/path.dta

Using wildcards to select multiple files from a single directory:

. collect a:/single*/dir*/allofthese*.dta