------------------------------------------------------------------------------- help for mahascores -------------------------------------------------------------------------------
Generate a set of Mahalanobis distance measures
mahascores varlist [weight] , [ idvar(idvar) varprefix(varprefix) genmat(genmat) genfile(filename) name1(name1) name2(name2) scorevar(scorevar) replace treated(treatedvar) invcovarmat(invcovarmat) compute_invcovarmat display(display_options) full all unsquared euclidean verbose float transpose nocovtrlimitation ]
mahascores generates a Mahalanobis distance measure between every pair of observations, or possibly between selected pairs of observations (under the treated() option). By default, the result is actually the square of the proper Mahalanobis measure. You can use the unsquared option to give you the unsquared value, but note that in most cases, the resulting values are used in comparisons or sortings; the proportional magnitude is not significant, so the squared values are just as good.
varlist (the "covariates") is a list of numeric variables on which to build the distance measure. These variables should be of numeric significance, not categorical; any categorical variables should be replaced by a set of indicator variables.
See mahascore for an explanation of the Mahalanobis measure.
Weights are allowed, but apply only under the compute_invcovarmat option.
There are three means of getting the output: - a set of generated variables, using the varprefix() option; - a matrix, using the genmat() option; - a separate file, using the genfile() option.
In what follows, let p denote the number of variables in varlist.
idvar(idvar) is an identifying variable which is used to mark the components of the output: - with the varprefix() option, its values become part of the new variable names; - with the genmat() option, its values are used as matrix row and column names; - with the genfile() option, its values go into the primary and secondary identifying variables.
idvar can be of any type, but it must be a single variable. If the existing identifying scheme consists of multiple variables, you should devise a way to combine them uniquely into a single variable. Numbers are acceptable, but they should be integers.
It is desirable and often essential (depending on the output options) that idvar uniquely identify observations. Under the the varprefix() option, it is essential that the contents of idvar be acceptable as suffixes on variable names; avoid embedded spaces and characters that are not acceptable in variable names.
-------------------------------------------------------------------- technical notes:
With the varprefix() option, illegal characters, embedded spaces, or non-unique values may result in a fatal error.
With the genmat() option, embedded spaces may cause the row or column names to "slip over" to the wrong row or column. Non-unique values do not cause an immediate error, but may cause confusing labelling of the columns and rows and may cause errors in later use of the matrix.
With the genfile() option, the form of the values in idvar is not critical, but if they don't uniquely identify observations, then it will be difficult to use the resulting file. --------------------------------------------------------------------
idvar() is optional. If it is omitted then the following values are used: - with the varprefix() option, 1, 2, etc., that is, the variable names are varprefix1, varprefix2, etc; - with the genmat() option, obs1, obs2, etc. as row and column names; - with the genfile() option, 1, 2, etc. But note that these numbers refer to the observations in the present order and become meaningless after a sort. Thus, the idvar() provides a more secure way of identifying the results.
The varprefix(), genmat(), and genfile() options are nonexclusive alternatives for obtaining the output; at least one of them must be used.
varprefix(varprefix) specifies that the results will be placed in a set of new variables, one for each observation. These variables will be named with a common prefix varprefix, and the remainder of the names are the values in idvar, or the observation numbers if idvar() is omitted. See the notes regarding acceptable content for idvar, above. Note that this option can generate a potentially very large set of variables - as many as there are observations (thus, constituting a square array of values), though that may set be reduced under the treated() option. (See remarks under treated() for more on that matter.) The default type for these variables is double.
genmat(genmat) specifies that the results will be placed in a matrix named genmat. The row and column names will be taken from the values in idvar, or will be obs1, obs2, etc., if idvar() is omitted. See the notes regarding acceptable content for idvar, above. If genmat already exists as a matrix, it will be overwritten. See additional remarks under treated() regarding which rows and columns will be included.
genmat() potentially creates a very large matrix. You may need to set matsize to a large value to enable this matrix to be created.
genfile(filename) specifies that the results will be placed in a separate dataset in long form. See reshape for an explanation of long form.
Under the genfile() option, the resulting file is a Stata dataset with these variables:
A primary and secondary id variable which refer to observations in the dataset from which the measures were derived.
A variable to hold the distance measure, measured between the observations identified in the primary and secondary id variables. The default name is _score, and its default type is double.
The types, content, and default names of the primary and secondary id variables depend on whether idvar() is specified:
If idvar() is specified, then these variables are of the same type as idvar, and contain values from idvar corresponding to the pertinant observations. Their default names are _refid and idvar.
If idvar() is omitted, then they are integer types, and contain the corresponding observation numbers. Their default names are _refobs and _obs.
Note that each of the three output options has two distinct entities that locate a distance measure value. We will identify one as primary and the other as secondary. The primary entities are...
for varprefix(), the variables generated; for genmat(), the rows of the matrix; for genfile(), the primary id variable.
The secondary entities are...
for varprefix(), the observations of the dataset (with values placed in the generated variables); for genmat(), the columns of the matrix; for genfile(), the secondary id variable.
Thus, the distance measure represents a difference measured from from the observation identified by the primary entity to the observation identified by the secondary entity; the distance in the other direction is the same. Consequently, the distinction between the primary and secondary entities often becomes immaterial, due to the symmetry of the situation. However, there is a situation where we choose to make a distinction, and the resulting set of values is asymmetric. In particular, this occurs under the treated() option, which will be described below.
Options for use with genfile
name1(name1) allows you to specify the name for the primary id variable. The default name depends on whether idvar() is specified, as explained above.
name2(name2) allows you to specify the name for the secondary id variable. The default name depends on whether idvar() is specified, as explained above.
scorevar(scorevar) allows you to specify the name of the distance measure variable. The default name is _score.
replace specifies that if the file already exists, it will be replaced.
invcovarmat(invcovarmat) specifies the name of a matrix to be used in the computation of the distance measure. It is presumably the inverse covariance matrix of varlist, but the only requirement is that it be a square p-by-p matrix, and both the row and column names must equal the names in varlist in the same order as in varlist.
invcovarmat(invcovarmat) is expected to be rarely used; it is provided in case the user wishes to supply an existing inverse covariance matrix, or one computed in some special way not provided for by the available options. Additionally, it might enable some efficiency advantage if repeated calls are made requiring the same inverse covariance matrix. For most usages, however, you probably want the compute_invcovarmat option
compute_invcovarmat specifies that you want the inverse covariance matrix to be computed, rather than passed in (via invcovarmat()). This computation is subject to weighting, as well as limitation by treatedvar if the treated() option is specified. (But see nocovtrlimitation.) Note that this will call covariancemat, which computes covariances limited to observations with all variables of varlist nonmissing. (I.e., it is potentially different from the pairwise computation of covariances.)
invcovarmat() and compute_invcovarmat are alternatives; one of them must be specified. If both are specified, then compute_invcovarmat takes precedence.
treated(treatedvar) specifies a numeric variable that distinguishes the "treated" observations, with values of 0 and non-zero signifying non-treated and treated, respectively. See mahapick for an explanation of the concept of the treated set. This option affects the action of the compute_invcovarmat in that the computation is limited to the set of observations for which treatedvar is non-zero, if treated() is specified. See nocovtrlimitation for how to control that limitation.
treated() also potentially limits the set of values that are output. In generic terms, the default action is that primary entities are associated with (limited to) the treated observations, and the secondary entities are associated with (limited to) the non-treated observations. (One exception: the secondary entities of the varprefix() option - the placing of values in the generated variables - are never limited in this way.) More specifically,
With varprefix(), only the variables corresponding to the treated observations will be generated.
With genmat(), only the rows corresponding to treated cases are generated; only the columns corresponding to non-treated cases are generated.
With genfile, only observations with primary id corresponding to treated cases, and with secondary id corresponding to non-treated cases are generated.
The rationale is that, with treated(), you would only be interested in distance measurments from a treated observation to a non-treated. (And these limitations save space as well.)
The all option lifts these limitations entirely; both the primary and secondary entities will range over all observations, yielding a square symmetric result.
The full option lifts the restriction on the secondary entities; all possible secondary id values or matrix columns are generated. (It has no effect on the varprefix() results, as the secondary entities for varprefix() are never limited by treated().)
In other words, the variables generated, the rows of the matrix, or the primary id variable, correspond to... the treated observations, if treated() is specified without all; all observations, otherwise. the colums of the matrix, or the secondary id variable correspond to... the non-treated observations, if treated() is specified without all or full; all observations, otherwise.
Note that all implies full; there is no provision for generating all primary id values (or matrix rows) without also getting all secondary id values (or matrix columns).
unsquared modifies the results to be the unsquared values, that is, the square roots of the default values.
euclidean takes effect only if compute_invcovarmat is also specified. This specifies that the normalized Euclidean measure is to be used, rather than the true Mahalanobis measure - meaning that the off-diagonal elements of the covariance matrix are replaced with zeroes prior to inverting. The result is a measure that accounts for the scale of measurement in each variable of varlist, but ignores correlation between the variables. This is probably not desirable, given the advantages of the true Mahalanobis measure, but is provided as an alternative and for comparison to (or emulation of) earlier releases of mahascore and mahapick. See mahascore for more details on this matter.
float specifies that the type for the variables generated by varprefix() or for the distance measure (or scorevar) generated by genfile() will be float, rather than double. This has no effect on genmat(), as matrices always contain doubles.
display(display_options) turns on the display of certain data structures used in the computation. If display_options contains covar, then the covariance matrix is listed; if it contains invcov, then the inverse covariance matrix is listed. Any other content is ignored.
verbose takes effect only if compute_invcovarmat is also specified. This causes each call to mahascore to be reported, along with information about what options were specified.
transpose specifies that the matrix (under genmat()) is to be transposed.
nocovtrlimitation specifies that the covariance computation (for compute_invcovarmat) not be limited to treated observations.
If the inverse covariance matrix is computed on a very small set of observations, it may not be valid and may yield strange results. It might fail to be positive semi-definite, and can yield negative measures. (It may also cause the unsquared option to have a real effect on comparisons and sortings of the results.)
Please see mahascore for more information on the computation of the Mahalanobis measure.
. mahascores income age numkids edlevel, idvar(persno) varprefix(d1_) treated(assisted) compute_invcov
. mahascores income age numkids edlevel, idvar(persno) genmat(m1) treated(assisted) compute_invcov
. mahascores income age numkids edlevel, idvar(persno) genfile(dist1) compute_invcov scorevar(d1)
Acknowledgement The author wishes to thank Heiko Giebler of Wissenschaftszentrum Berlin fur Sozialforschung GmbH, for suggestion leading to the development of this program.
Author David Kantor. Email firstname.lastname@example.org if you observe any problems.
Also See mahascore, mahascore2, mahapick, covariancemat, variancemat,