help for als_norm

Title

asl_norm -- Bootstrap normality tests.

Syntax

asl_norm varname [if] [in]
[
,
dhansen
hzirkler
mkurtosis
mskewness
jbtest
sktest
skewness
kurtosis
swilk
sfrancia
all
fast
reps(#)
nodots
mcci(#)
saving(filename [, replace double every(#)])
]

Description

asl_norm computes bootstrap estimates of the Achieved Significance Levels (ASL) for a
number of Gaussianity (normality) tests. The idea behind a boostrap test is to take the logic of
a p-value literaly: In order to find the probability of observing data with a test statistic at
least as extreme as the one found in the data one can:

1) repeatedly draw a new variable from a Gaussian distribution with the mean and standard
deviation as observed in the data,

2) compute in each replication the test statistic for that simulated variable, and

3) compute the ASL as the number of times that that test statistic is as extreme or more than the one
found in the real data plus 1 divided by the number of replications plus 1.

This ASL can be interpreted as a more robust estimate of the p-value and/or as a simulation check of
whether the chosen statistic is working well in your sample, i.e. whether the ASL corresponds with
the asymptotic p-value.

A key aspect of ASLs is that they are computed with aid of a Monte Carlo experiment and are thus
uncertain. If one were to call asl_norm twice (and not set the seed), one
will find slightly different ASLs. asl_norm quantifies this uncertainty by displaying a
Monte Carlo confidence interval for the ASL. If you were to call asl_norm a 100 times you
would in 95 of these calls expect to find an ASL within this interval. If you are unhappy with
the width of this interval, than you can decrease it by specifying more replications in the
reps(). Typically, I find 19,999 a number that works well for this type of procedure; if
the "true" p-value is 0.05 than the ASL will on average be computed based on a 999 replications
where the simulated statistic is larger than the observed statistic.

An aside

The estimate of the ASL used in this program is discussed in Chapter 4 of
(Davison and Hinkley 1997) and deviates slightly from the estimate
discussed in Chapter 16 of (Efron and Tibshirani 1993) in that the
latter estimate the ASL using just the proportion of times that that test statistic is more extreme
than the one found in the real data. With a large number of replications this difference is not going
to matter, but striclty speaking the former is slightly preferable.

If we find an ASL of a, than the probability of drawing a dataset with an ASL less than or equal
to a if the null hypothesis is true should itself be a, and this should be true for all
possible values of a. So the sampling distribution of the ASL if the null hypothesis is true
should be a standard uniform distribution. In other words, if the null hypothesis is true all possible
values of a should be equally likely.

If we compute our ASL using B replications, than the number of possible values for a is limited
to only B+1 values: a value based on 0, 1, 2, ..., or B replications with a test statistic as extreme
or more than the one found in the real data. Based on the condition discussed above, each of these
possible values should have a probability of 1/(B+1) if the null hypothesis is true. This means that
if the null hypothesis is true, the probability of finding 0 replications with a test statistic as
extreme or more as the one found in the real data is 1/(B+1). Similarly, the probability of finding 1
or less replications with a test statistic at least as extreme as the one found in the real data is
2/(B+1). Generally, the probability of finding k or less replications with a test statistic as
extreme or more as the one found in the real data if the null hypothesis is true is (k+1)/(B+1).

Options

Statistics

dhansen specifies that thee Doornik-Hansen statistic as computed by mvtest_normality
should be used.

hzirkler specifies that the Henze-Zirkler statistic as computed by mvtest_normality
should be used.

mkurtosis specifies that Mardia's kurtosis test statistic as computed by
mvtest_normality should be used.

mskewness specifies that Mardia's skewness test statistic as computed by
mvtest_normality should be used.

jbtest specifies that the Jarque-Bera test statistic should be used.

sktest specifies that the skewness-kurtosis test statistic as computed by sktest
should be used. The default if N > 5000.

skewness specifies that the skewness test statistic as computed by sktest should
be used.

kurtosis specifies that the kurtosis test statistic as computed by sktest should
be used.

swilk specifies that the Shapiro-Wilk test statistic as computed by swilk should
be used. Only allowed if 4<=N<=2000. The default if N <= 2000.

sfrancia specifies that the Shapiro-Francia test statistic as computed by sfrancia
should be used. Only allowed if 5<=N<=5000. The default if 2000 < N <= 5000.

all specifies that all the above statistics (if allowed) should be used.

fast specifies that all the above statistics (if allowed), except he Henze-Zirkler statistic
and Mardia's skewness test statistic, which tend to be slow in moderate to large samples.

Other options

reps(#) specifies the number of replications used to compute the ASL. The default is reps(1000).

nodots suppresses display of the replication dots. By default, one dot character is displayed for
each successful replication. A red "x" is displayed if there was an error while computing one of the test
statistics or if one of the test statistics is missing.

mcci(#) specifies the confidence level, as a percentage, for the Monte Carlo confidence interval. The
default is mcci(95) or as set by set level.

saving(filename [, replace double every(#)]) creates a Stata data file containing for each replication
the test statistics. See: prefix_saving_option.

Examples

In a small sample most tests have at least some degree of trouble. In this case (74 observations) the
Doornik-Hansen test leads to a p-value that is a bit too large.

sysuse auto, clear
asl_norm trunk, dh reps(19999)

(click to run)

In larger samples the asymptotic p-values tend to be less problematic

sysuse nlsw88, clear
gen lnwage = ln(wage)
asl_norm lnwage if union < ., dh reps(19999)

(click to run)

You can use the saving() option to perform a simulation of how well the tests perform for the number
of observations in your dataset. This creates a dataset of test statistics on datasets where the null
hypothesis is true, so the distribution of these statistics should correspond to the theoretical sampling
distribution and the p-values should follow a continuous standard uniform distribution.

set seed 12345
sysuse auto, clear

tempfile res
asl_norm trunk, jb sktest reps(1999) saving("`res'", replace)
use "`res'"

// check sampling distribution of test statistic
qenvchi2 sktest, gen(lb ub) df(2) overall reps(5000)
tempname qplot
qplot sktest jbtest lb ub, trscale(invchi2(2,@)) ///
ms(o o none ..) c(. . l l) lc(gs10 ..) ///
scheme(s2color) ylab(,angle(horizontal)) ///
legend(order( 1 "sktest" ///
2 "Jarque-Bera" ///
3 "95% simultaneaous" ///
"Monte Carlo CI")) ///
name(`qplot')

// test sampling distribution of p-values
gen p_sk = chi2tail(2,sktest)
label var p_sk "sktest"
gen p_jb = chi2tail(2,jbtest)
label var p_jb "Jarque-Bera"
simpplot p_sk p_jb, overall reps(19999) ///
scheme(s2color) ylab(,angle(horizontal))

(click to run)

If you do find a significant difference between the variable and a Gaussian distribution, you will need to
follow this up with a graphical inspection of that variable in order to determine the source of the problem.
In this case, the problem is in all likelihood that wages were top-coded.

sysuse nlsw88, clear
gen lnwage = ln(wage)
asl_norm lnwage
qnorm lnwage

(click to run)

Better yet, you can combine the testing logic with the graphing logic by adding confidence bands using the
qenv package.

sysuse nlsw88, clear
gen lnwage = ln(wage)
qenvnormal lnwage, gen(lb ub) overall reps(19999)
sum lnwage
qplot lnwage lb ub, ms(oh none ..) c(. l l) lc(gs10 ..) ///
legend(off) ytitle("ln(wage)") xtitle(Normal quantiles) ///
trscale(`r(mean)' + `r(sd)' * invnormal(@))

(click to run)

Author

Maarten L. Buis

Wissenschaftszentrum Berlin für Sozialforschung, WZB

Research unit Skill Formation and Labor Markets

maarten.buis@wzb.eu

References

A.C. Davison & D.V. Hinkley (1997)
Bootstrap Methods and their Application. Cambridge: Cambridge University Press.

B. Efron & R.J. Tibshirani (1993)
An Introduction to the Bootstrap. Boca Raton: Chapman & Hall/CRC.

Also see

Online: qnorm, pnorm, sktest, mvtest normality, swilk

If installed: hangroot, qenvnormal, qplot