*! version 1.0.0 15nov2023

wgtdistrim -- Trim extreme sampling weights

Syntax

wgtdistrim
varname
[if] [in]
,
generate([type] newvar)
upper(#)
[ options ]

* generate([type] newvar) generate
newvar containing trimmed sampling weights

* upper(#) trim large sampling weights
with probability of occurence less than #

lower(#) trim small sampling weights
with probability of occurence less than #;
default is lower(0)

iterate(#) perform maximum of # iterations;
default is iterate(10)

tolerance(#) tolerance for trimmed sampling weights;
default is tolerance(0)

normalize normalize trimmed sampling weights
to sum to number of observations

* generate() and upper() are required.

Description

wgtdistrim trims extreme sampling weights
using the weight distribution approach suggested by Potter (1990).

The reciprocal of the sampling weights are assumed to follow a (scaled)
beta distribution.
The parameters of the beta distribution are estimated from the sampling weights
and the trimming levels (cut-offs) are computed for the specified percentiles.
Sampling weights that are more extreme than the specified percentiles
are trimmed to these percentiles and the excess is distributed equally
among the untrimmed sampling weights
so that the sum of the trimmed sampling weights
equals the sum of the untrimmed sampling weights.
This process is repeated a specified number of times
(or until the trimmed sampling weights do no longer change)
to otain the final trimmed sampling weights.

Options

generate([type] newvar)
specifies the name and, optionally, the type of the new variable
containing the trimmed sampling weights.
Option generate() is required.

upper(#)
specifies the probability of occurrence
of large sampling weights to be trimmed.
Large sampling weights with probability of occurence
less than #, where 0 < # < 1, are trimmed
to the 1-# percentile of the beta distribution.
Option upper() is required.

lower(#)
specifies the probability of occurrence
of small sampling weights to be trimmed.
Small sampling weights with probability of occurence
less than #, where 0 < # < 1, are trimmed
to the # percentile of the beta distribution.
Default is lower(0).

iterate(#)
specifies the maximum number of iterations,
i.e., how often the trimming levels are computed.
The default maximum number of iterations is 10.

tolerance(#)
specifies the tolerance for trimmed sampling weights.
When the relative difference in sampling weights
from one iteration to the next is less than or equal to #,
convergence is achieved.
The default tolerance() is 0.

normalize
specifies that the trimmed sampling weights be normalized
to sum to the number of observations
(with non-missing positive sampling weights).
Normalization is done once, after the trimming procedure.

Examples

Trim sampling weights only on the right tail of the beta distribution
if the probability of occurence is less than 0.01 (1 percent).

. wgtdistrim pweight , generate(double pweight_trimmed) upper(.01)

Trim sampling weights on the left and right tail of the beta distribution
if the probability of occurence is less than 0.01 (1 percent) on either tail.

. wgtdistrim pweight , generate(double pweight_trimmed) lower(.01) upper(.01)

Same as above, but also normalize the the trimmed sampling
weights to sum to the number of observations.

. wgtdistrim pweight , generate(double pweight_trimmed) lower(.01) upper(.01) normalize

References

Potter, F. J. 1990.A study of procedures to identify and trim extreme sampling weights.Proceedings of the Survey Research Methods Section of the American Statistical Association,225--230.

Chen, Q., Elliott, M. R., Haziza, D., Yang, Y., Ghosh, M., Little, R. J. A., Sedransk, J., & Thompson, M. 2017.Approaches to improving survey-weighted estimates.Statistical Science, 32(2), 227--248.

Suggested citation

Lang, S., & Klein, D. (2023). WGTDISTRIM: Stata module to trim extreme sampling weights. doi: 10.7802/2641

Available from https://raw.githubusercontent.com/se-lang/wgtdistrim/main

Support

contact@sebastianlang.eu

klein.daniel.81@gmail.com