{smcl}
{hline}
help for {hi:xcollapse}{right:(Roger Newson)}
{hline}

{title:Make dataset of means, medians, and other summary statistics.}

{p 8 17 2}{cmd:xcollapse} {it:clist} [{it:weight}] [{cmd:if} {it:exp}]
        [{cmd:in} {it:range}] [{cmd:,}
         {break}
         {cmdab:li:st}{cmd:(} [{it:varlist}] [{cmd:if} {it:exp}] [{cmd:in} {it:range}] [ , [{it:list_options}] ] {cmd:)}
         {cmdab:sa:ving}{cmd:(}{it:filename}[{cmd:,replace}]{cmd:)} {cmdab::no}{cmdab:re:store} {cmd:fast}
	 {cmdab:fl:ist}{cmd:(}{it:global_macro_name}{cmd:)}
         {break}
         {cmd:cw}
         {cmd:by(}{it:varlist}{cmd:)}
         {cmdab:idn:um}{cmd:(}{it:#}{cmd:)} {cmdab:nidn:um}{cmd:(}{it:newvarname}{cmd:)}
         {cmdab:ids:tr}{cmd:(}{it:string}{cmd:)} {cmdab:nids:tr}{cmd:(}{it:newvarname}{cmd:)}
         {break}
         {cmdab:fo:rmat}{cmd:(}{it:varlist_1 format_1 ... varlist_n format_n}{cmd:)}
        ]


{p 4 4 2}where {it:clist} is either

{p 8 17 2}[{cmd:(}{it:stat}{cmd:)}] {it:varlist} [ [{cmd:(}{it:stat}{cmd:)}]
        {it:...} ]{p_end}
{p 8 17 2}[{cmd:(}{it:stat}{cmd:)}] {it:target_var}{cmd:=}{it:varname}
        [{it:target_var}{cmd:=}{it:varname} {it:...}]
        [ [{cmd:(}{it:stat}{cmd:)}] {it:...}]

{p 4 4 2}or any combination of the {it:varlist} or {it:target_var} forms, and
{it:stat} is one of

{p 8 20 2}{cmd:mean}{space 8}means{p_end}
{p 8 20 2}{cmd:sd}{space 10}standard deviations{p_end}
{p 8 20 2}{cmd:sum}{space 9}sums{p_end}
{p 8 20 2}{cmd:rawsum}{space 6}sums ignoring optionally specified weight{p_end}

{p 8 20 2}{cmd:count}{space 7}number of nonmissing observations{p_end}
{p 8 20 2}{cmd:max}{space 9}maximums{p_end}
{p 8 20 2}{cmd:min}{space 9}minimums{p_end}

{p 8 20 2}{cmd:median}{space 6}medians{p_end}
{p 8 20 2}{cmd:p1}{space 10}1st percentile{p_end}
{p 8 20 2}{cmd:p2}{space 10}2nd percentile{p_end}
{p 8 20 2}{it:...}{space 9}3rd -- 49th percentiles{p_end}
{p 8 20 2}{cmd:p50}{space 9}50th percentile (same as {cmd:median}){p_end}
{p 8 20 2}{it:...}{space 9}51st -- 97th percentiles{p_end}
{p 8 20 2}{cmd:p98}{space 9}98th percentile{p_end}
{p 8 20 2}{cmd:p99}{space 9}99th percentile{p_end}
{p 8 20 2}{cmd:iqr}{space 9}interquartile range{p_end}

{p 4 4 2}
If {it:stat} is not specified, {cmd:mean} is assumed.

{p 4 4 2}
{cmd:aweight}s, {cmd:fweight}s, {cmd:pweight}s, and {cmd:iweight}s are
allowed; see help {help weights} and see note concerning weights below.
Statistic {cmd:sd} (standard deviation) is not allowed with {cmd:pweight}s.

{p 4 4 2}
{it:varlist} and {it:varname} in {it:clist} may contain time-series operators;
see help {help varlist}.


{title:Description}

{p 4 4 2}
{cmd:xcollapse} is an extended version of {help collapse}.
It creates an output data set of means, sums, medians, and other summary statistics.
This output data set may be listed to the Stata log, or saved to a disk file,
or written to the memory (overwriting any pre-existing data set).


{title:Options for use with {cmd:xcollapse}}

{p 4 4 2}
The options are listed in the following 2 groups:

{p 4 6 2}{bf:1.} Output-destination options.
(These specify where the output data set will be written.){p_end}

{p 4 6 2}
{bf:2.} Other options. (These specify what the output data set will contain.)


{title:Output-destination options}

{p 4 8 2}{cmd:list(}{it:varlist} [{cmd:if} {it:exp}] [{cmd:in} {it:range}] [, {it:list_options} ] {cmd:)}
specifies a list of variables in the output
data set, which will be listed to the Stata log by {cmd:xcollapse}.
The {cmd:list()} option can be used with the {cmd:format()} option (see below)
to produce a list of summary statistics
with user-specified numbers of decimal places or significant figures.
The user may optionally also specify {help if} or {help in} qualifiers to list subsets of combinations
of variable values,
or change the display style using a list of {it:list_options} allowed as options by the {help list} command.

{p 4 8 2}{cmd:saving(}{it:filename}[{cmd:,replace}]{cmd:)} saves the output data set to a disk file.
If {cmd:replace} is specified, and a file of that name already exists,
then the old file is overwritten.

{p 4 8 2}{cmd:norestore} specifies that the output data set will be written to the memory,
overwriting any pre-existing data set. This option is automatically set if {cmd:fast} is
specified. Otherwise, if {cmd:norestore} is not specified, then the pre-existing data set is restored
in the memory after the execution of {cmd:xcollapse}.

{p 4 8 2}{cmd:fast} is a stronger version of {cmd:norestore}, intended for use by programmers.
It specifies that the pre-existing data set in the memory will not be restored,
even if the user presses {help break:Break} during the execution of {cmd:xcollapse}.
If {cmd:norestore} is specified and {cmd:fast} is absent,
then {cmd:xcollapse} will go to extra work so that
it can restore the original data if the user presses {help break:Break}.

{p 4 8 2}Note that the user must specify at least one of the four options {cmd:list()}, {cmd:saving()},
{cmd:norestore} and {cmd:fast}. These four options specify whether the output data set
is listed to the Stata log, saved to a disk file, or written to the memory
(overwriting any pre-existing data set). More than one of these options can be specified.

{p 4 8 2}{cmd:flist(}{it:global_macro_name}{cmd:)} specifies the name of a global macro, containing
a filename list (possibly empty). If {cmd:saving()} is also specified, then
{cmd:xcollapse} will append the name of the data set specified in the
{cmd:saving()} option to the value of the global macro specified in {cmd:flist()}. This
enables the user to build a list of filenames in a global macro, containing the
output of a sequence of output data sets.
These files may later be concatenated using {help append}, or using {help dsconcat}
(downloadable from {help ssc:SSC}) if installed.


{title:Other options}

{p 4 8 2}{cmd:cw} specifies casewise deletion.  If not specified, all
observations possible are used for each calculated statistic.

{p 4 8 2}{cmd:by(}{it:varlist}{cmd:)} specifies the groups over which the summary statistics
are to be calculated.  If not specified, the resulting dataset will
contain one observation.  If specified, {it:varlist} may refer to either
string or numeric variables.
Note that, if the {help if} expression or the {help weight} expression contains the
reserved names {hi:_n} and {hi:_N}, then these will be interpreted as the observation sequence number
and the number of observations, respectively, within the whole data set, not within the by-group.

{p 4 8 2}{cmd:idnum(}{it:#}{cmd:)} specifies an ID number for the output data set.
It is used to create a numeric variable, with default name {hi:idnum}, in the output data set,
with that value for all observations.
This is useful if the output data set is concatenated with other {cmd:xcollapse} output data sets
using {help append}, or using {help dsconcat} if installed.

{p 4 8 2}{cmd:nidnum(}{it:newvarname}{cmd:)} specifies a name for the numeric ID variable
evaluated by {cmd:idnum()}. If {cmd:idnum()} is present and {cmd:nidnum()} is absent,
then the name of the numeric ID variable is set to {hi:idnum}.

{p 4 8 2}{cmd:idstr(}{it:string}{cmd:)} specifies an ID string for the output data set.
It is used to create a string variable, with default name {hi:idstr} in the output data set,
with that value for all observations.
This is useful if the output data set is concatenated with other {cmd:xcollapse} output data sets
using {help append}, or using {help dsconcat} if installed.

{p 4 8 2}{cmd:nidstr(}{it:newvarname}{cmd:)} specifies a name for the string ID variable
evaluated by {cmd:idstr()}. If {cmd:idstr()} is present and {cmd:nidstr()} is absent,
then the name of the string ID variable is set to {hi:idstr}.

{p 4 8 2}{cmd:format(}{it:varlist_1 format_1 ... varlist_n format_n}{cmd:)}
specifies a list of pairs of {help varlist:variable lists} and {help format:display formats}.
The {help format:formats} will be allocated to
the variables in the output data set specified by the corresponding {it:varlist_i}
lists.
If the {cmd:format()} option is absent, then the percent variables have the format {hi:%8.2f},
the frequency variables have the format {hi:%12.0g},
and the other variables have the same formats as the variables of the same names
in the input data set.


{title:Examples}

{p 4 4 2}
The following examples use the {cmd:list()} option to list the output data set to the Stata log.
After these examples are executed, there is no new data set either in the memory or on disk.

{p 4 8 2}{cmd:. xcollapse mpg weight price, list(,)}

{p 4 8 2}{cmd:. xcollapse (median) mpg weight price, list(,)}

{p 4 8 2}{cmd:. xcollapse mpg weight price, by(foreign rep78) list(,clean)}

{p 4 8 2}{cmd:. xcollapse (mean) mpg weight price, by(foreign rep78) format(mpg weight price %8.2f) list(*, sepby(foreign))}

{p 4 8 2}{cmd:. xcollapse (count) nmpg=mpg nweight=weight nprice=price (median) medmpg=mpg medweight=weight medprice=price, by(foreign rep78) format(med* %8.2f) list(foreign rep78 *mpg *weight *price, sepby(foreign) abbrev(16))}

{p 4 4 2}
The following examples use the {cmd:norestore} option to create an output data set in the memory,
overwriting any pre-existing data set.

{p 4 8 2}{cmd:. xcollapse mpg weight price, norestore}

{p 4 8 2}{cmd:. xcollapse (median) mpg weight price, norestore}

{p 4 8 2}{cmd:. xcollapse mpg weight price, by(foreign rep78) norestore}

{p 4 8 2}{cmd:. xcollapse (mean) mpg weight price, by(foreign rep78) format(mpg weight price %8.2f) norestore}

{p 4 8 2}{cmd:. xcollapse (count) nmpg=mpg nweight=weight nprice=price (median) medmpg=mpg medweight=weight medprice=price, by(foreign rep78) format(med* %8.2f) norestore}

{p 4 4 2}
The following examples use the {cmd:saving()} option to create an output data set in a disk file.

{p 4 8 2}{cmd:. xcollapse mpg weight price, saving(mysumm1)}

{p 4 8 2}{cmd:. xcollapse (median) mpg weight price, saving(mysumm2,replace)}

{p 4 8 2}{cmd:. xcollapse mpg weight price, by(foreign rep78) saving(mysumm3,replace)}

{p 4 8 2}{cmd:. xcollapse (mean) mpg weight price, by(foreign rep78) format(mpg weight price %8.2f) saving(mysumm4,replace)}

{p 4 8 2}{cmd:. xcollapse (count) nmpg=mpg nweight=weight nprice=price (median) medmpg=mpg medweight=weight medprice=price, by(foreign rep78) format(med* %8.2f) saving(mysumm5,replace)}


{title:Note concerning the choice among weights}

{p 4 4 2}
{cmd:xcollapse} allows all four weight types.  The weight normalization only
impacts the {cmd:sum}, {cmd:count}, and {cmd:sd} statistics.

{p 4 4 2}
The normal restrictions on weights apply:

{p 8 12 2}1.  {cmd:aweight}s are required to be zero or positive;{p_end}
{p 8 12 2}2.  {cmd:fweight}s are additionally required to be integers;{p_end}
{p 8 12 2}3.  {cmd:iweight}s can be anything and it is your responsibility to ensure the validity of what is produced;{p_end}
{p 8 12 2}4.  {cmd:pweight}s are not required to be integers, just zero or positive.{p_end}

{p 4 4 2}
The weights have the standard interpretation:

{p 8 12 2}1.  {cmd:aweight}s are intended for cell-mean data;{p_end}
{p 8 12 2}2.  {cmd:fweight}s are intended for replicated data;{p_end}
{p 8 12 2}3.  {cmd:iweight}s are intended for use by programmers;{p_end}
{p 8 12 2}4.  {cmd:pweight}s are intended for survey-sample data.{p_end}

{p 4 4 2}
Here are the definitions for {cmd:count} and {cmd:sum} with weights:

    {cmd:count}:
{p 8 32 2}unweighted: {space 11} the number of physical observations{p_end}
{p 8 32 2}{cmd:aweight}: {space 14} the number of physical observations{p_end}
{p 8 32 2}{cmd:f}, {cmd:i}, and {cmd:pweight}s {space 4} sum of the weights{p_end}

    {cmd:sum}:
{p 8 32 2}unweighted: {space 11} the sum of the variable{p_end}
{p 8 32 2}{cmd:aweight}: {space 14} sum(vj*xj); vj = weights normalized to sum to _N{p_end}
{p 8 32 2}{cmd:f}, {cmd:i}, and {cmd:pweight}s {space 4} sum(wj*xj); wj = user supplied weights.{p_end}

{p 4 4 2}
The {cmd:sd} statistic with weights returns the bias-corrected standard
deviation, which is based on the factor sqrt(N/(N-1)), where N is the number
of observations.  {cmd:sd} is not allowed with {cmd:pweight}ed data.
Otherwise {cmd:sd} is changed by the weights through the computation of the
count (N) as outlined above.


{title:Author}

{p}
Roger Newson, Imperial College London, UK.
Email: {browse "mailto:r.newson@imperial.ac.uk":r.newson@imperial.ac.uk}


{title:Also see}

{p 4 13 2}
Manual:  {hi:[R] collapse}, {hi:[R] contract}
{p_end}

{p 4 13 2}
Online:  help for {help collapse}, {help contract}, {help egen},
{help statsby}, {help summarize},
{help tabdisp}, {help table}
{break} help for {help xcontract}, {help dsconcat} if installed
{p_end}
