help for linuxlsd1(Roger Newson)

Create a dataset of file records from the output of a Linux ls -d1 command

linuxlsd1 [ , fspec(file_spec) lsoptions(ls_options)
finfo(newvarname)
clear
]

llinuxlsd1 [ , fspec(file_spec)) lsoptions(ls_options)
finfo(name)
]

where file_spec is a Linux or Mac OS X file list specification
and ls_options is a list of extra options for the Linux ls command.
For more about these specifications and options, see the Linux online help under ls --help.

Description

linuxlsd1 inputs a file list specification recognised by Linux, Unix or Mac OS X,
and generates in the memory a new Stata dataset, with 1 observation per file in the list,
and data on information about the file,
including the file name and possibly other file-specific information.
This new Stata dataset can then be used for mass-processing of the specified files.
llinuxlsd1 inputs a file list specification recognised by Linux or Mac OS X,
and generates a local macro containing a list,
with 1 list item per file in the list,
containing information about the file, including the file name and possibly other items.
linuxlsd1 and llinuxlsd1 work by using the Linux ls command,
with the options -d1 to specify that directories in the file list will be listed as files (and not expanded),
and that the output will contain 1 line per file.
They are designed to work only under a Linux, Unix or Mac OS X operating environment.

Options for linuxlsd1 and llinuxlsd1

fspec(file_spec) specifies a list of files,
in the language of Linux or Mac OS X.
If not specified, then it is set to .,
specifying the local directory under Linux or Mac OS X.
These file specifications may contain wildcards such as *.
For instance, filespec("*.txt") specifies a list of all files in the current directory with the extension .txt.
It is usually a good idea to enclose the fspec() option in quotes,
because Linux file specifications often contain forward slashes (/),
which Stata uses for an escape character, at least in do-files.

lsoptions(ls_options) specifies a list of options acceptable to the ls command under Linux.
For instance, lsoptions(-t) specifies that the files will be sorted by the time when they were created,
and lsoptions(l) specifies that the output variable or macro will contain list items for each file,
instead of just the file name.

Options for linuxlsd1 only

finfo(newvarname) specifies the name for a variable to be generated in the output dataset,
containing the file information output by the Linux ls -d1 command,
with the files specified by the fspec() option.
If finfo() is unset, then the variable will have the name finfo.

clear specifies that the output dataset will overwrite any existing dataset that may be present in the memory.
If clear is not specified,
then linuxlsd1 will refuse to create an output dataset if data are already in memory.
This convention protects the user from deleting important data.

Options for llinuxlsd1 only

finfo(name) specifies the name for a local macro to be generated,
containing the list of file information records output by the Linux ls -d1 command
for the files specified by the fspec() option.
If finfo() is unset, then the macro will have the name finfo.

Remarks

The linuxlsd1 package is the Linux, Unix or Mac OS X equivalent of the msdirb package,
which creates a dataset in memory, or a local macro, containing a user-specified list of Microsoft Windows file names,
and which can be downloaded from SSC.

Examples

The following example creates in memory a dataset of all filenames with the extension .txt in the directory ./mysub1,
with a single variable finfo containing the file name. Note that the file specification is enclosed in quotes:

. linuxlsd1, fspec("./mysub1/*.txt") clear

. describe, full

. list

The following example creates in memory a dataset of all filenames with the extension .txt in the directory ./mysub1,
but this time the a single variable containing the file name is named filepath:

. linuxlsd1, fspec("./mysub1/*.txt") finfo(filepath) clear

. describe, full

. list

The following example creates in memory a dataset with 1 observation per file with the extension .txt in the directory ./mysub1.
However, this time the file-information variable is named filedata,
and the lsoptions option ensures that it contains not only the filename but also a list of other information (specified by the {cmd option -l),
and the observations are sorted in descending order of file size (specified by the ls option -S).

. linuxlsd1, fspec("./mysub1/*.txt") lsoptions(-lS) finfo(filedata) clear

. describe, full

. list

The following example uses the llinuxlsd1 module to create a list of all filenames with the extension .txt in the directory ./mysub1,
storing their names in a local macro named foobar.
It then uses the tfconcat module of the SSC package intext
to concatenate all the files in the list into a dataset in memory,
with a string variable line containing the file line,
and a string variable filename containing the file from which the line came:

. llinuxlsd1, fspec("./mysub1/*.txt") finfo(foobar)

. macro list _foobar

. tfconcat `foobar', tfname(filename) gene(line)

. describe, full

. list

Author

Roger Newson, Imperial College London, UK.

Email: r.newson@imperial.ac.uk

Also see

Manual: [D] shell, [D] insheet, [D] cd

On-line: help for shell, insheet, pwd, cd

help for msdirb, intext, tfconcat if installed

Linux help for the ls command under ls --help in Linux