help for intext and tfconcat (Roger Newson)

Read text files into string variables in the memory (without losing blanks)

intext using filename , generate(prefix)
[ length(#) clear ]

tfconcat filename_list , generate(prefix)
[ length(#)
tfid(newvarname) tfname(newvarname)
obsseq(newvarname)
]

where filename_list is a list of filenames separated by spaces.

Description

intext inputs a single text file into a set of generated string variables in
the memory, generating as many string variables as is necessary to store the longest
records in full, without trimmming leading and trailing blanks (as infix does).
tfconcat takes, as input, a list of filenames, assumed to belong to text
files, and concatenates them (without losing blanks) to create a new data set in memory,
overwriting any pre-existing data. The new data set contains one observation for each
record in each text file, ordered primarily by source text file and secondarily by
order of record within source text file, and contains a set of generated string
variables containing the text, as created by intext.
Optionally, tfconcat
creates new variables, specifying, for each observation, the input text file of origin
and/or the sequential order of the observation in its input text file of origin.

Options for intext and tfconcat

generate(prefix) is not optional. It specifies a prefix for the names
of the new string variables generated, which will be named as prefix1 ... , prefixn,
where n is the number of string variables required to contain the longest text record
in any input data set, with length as specified by the length option.

length(#) specifies the maximum length of the generated text variables. If absent,
it is set to 80.
The value of length() must be an integer between 1 and the maximum length of a strL variable on the version of Stata being used.
This value is stored in the c-class value c(maxstrlvarlen).
See also help for limits.

Options for intext only

clear specifies that any existing data set in the memory is to be removed before the
generated text variables are created.
If clear is absent, then intext attempts to
add the generated variables to the existing data set, failing if there is an existing variable
with the same name as one of the generated variables.
(tfconcat always removes any
existing data set before generating new variables.)

Options for tfconcat only

tfid(newvarname) specifies a new integer variable to be created,
containing, for each observation in the new data set, the sequential order, in the
filename_list, of the input text file of origin of the observation.
If possible,
tfconcat creates a value label for the newvarname with the same name,
assigning, to each positive integer i from 1 to the number of input file names
in the list, a label equal to the filename of the ith input text file,
truncated if necessary to the maximum length of a label in the version of Stata being used,
which is 80 characters in Stata Version 9..
If a value label of that name already exists in one of the input data sets, and
nolabel is not specified, then tfconcat adds new labels,
but does not replace existing labels.

tfname(newvarname) specifies a new string variable containing,
for each observation in the new data set, the name of the input text file of origin
of that observation,
ttruncated if necessary to the maximum length of a string variable in the version of Stata being used.
This maximum value is stored in the c-class value c(maxstrvarlen).
(See also help for limits.)

obsseq(newvarname) specifies a new integer variable containing,
for each observation in the new data set, the sequential order of that observation
as a text record in its input text data set of origin.

Remarks

intext is an inverse of outfile with the runtogether option.
That is to say, if the user inputs a text file into a list of generated string variables
using intext and then outputs them to a second text file using outfile
with the runtogether option, then the second text file will be identical
to the first text file.
tfconcat
works by calling intext multiple times to create a data set for each text file,
and concatenating these data sets into the memory.
Both programs make it possible to use
Stata for text file processing, especially when the text files may be indented Stata programs.
This cannot be done properly using infix, which uses fixed-field input, but trims leading and trailing blanks from strings.
Therefore, the intext package enables Stata programs to read Stata programs,
just as outfile with the runtogether option enables Stata programs to write
Stata programs.

Examples

. intext using intext.ado, gene(sect) clear

. tfconcat auto1.txt auto2.txt auto3.txt auto4.txt, gene(piece) tfid(tfseq) obs(recnum)

. sort tfseq recnum

The following example is equivalent to copy tfconcat.ado trash1.txt,text replace

. intext using tfconcat.ado, gene(slice) clear

. outfile slice* using trash1.txt, runtogether replace

The following advanced example works under Windows, and might be used if the user has
a library of Stata ado-files in the current directory.
It inputs the ado-files into the memory
and lists the lines beginning with "*!", which are echoed by the which command.

. tempfile dirf

. shell dir/b *.ado > `dirf'

. intext using `dirf', gene(fn) length(244) clear

. levelsof fn1

. tfconcat `r(levels)', gene(line) tfid(adofile) obs(lseq)

. list adofile line* if substr(line1,1,2)=="*!"

Author

Roger Newson, Imperial College London, UK.
Email: r.newson@imperial.ac.uk

Also see

Manual: [D] infix, [D] append, [D] outfile, [P] file

On-line: help for infiling, infix, append, outfile, file