help file for hetop version 3.0

Title

hetop -- heteroskedastic ordered probit models

Syntax

hetop
grpvar levelvar
, numcats(#)
[ modtype(string)
identify(string)
setref(#)
phop(varname [ , mean ])
SETCuts(numlist)
pkvals(varname)
STARTFRom(namelist)
kappa(#)
save([ string , star prime raw se cov ])
ADDCONStraints(string)
minsize(#)
initvals
gaps
noisily
homop
CSD(#)
maximize_options ]

Required

numcats(#)
number of ordered categories in outcome variable

Optional

modtype(hetop/homop)
specify a heteroskedastic or homoskedastic model; allowable options are
hetop (the default) or homop

identify(sums/refgroup/cuts)
type of constraints used to identify the model; allowable options are sums
(the default) or refgroup or cuts

setref(#)
the ID number from grpvar of the group to be constrained as a reference
group

phop(varname [ , mean ])
constrain groups with varname=1 to have a single, common ln(SD) estimate.
Optionally typing phop(varname, mean) will constrain this single ln(SD) estimate
to be equal to the (unweighted) mean ln(SD) estimate for all groups with
varname=0

setcuts(numlist)
estimate the model with cut scores fixed to the values in numlist. Length
of numlist must be equal to numcats-1.

pkvals(varname)
specifies a variable containing the population proportions of each group. If not
specified, by default the sample proportions in the data will be used as the
population proportions of each group

STARTFRom(namelist)
start estimation from values specified in the matrix namelist.

kappa(1/2)
adjust the de-referencing equations. Recommended to leave this option
blank.

save([ string , star prime raw se cov ])
save estimated means and standard deviations in variables named
"mstarstring", "mstarstring_se", etc. Leaving string blank is
allowed. Estimates can be in the star, prime, and/or raw metric.
Optionally save estimated standard errors (se) and/or covariance
between mean and standard deviation estimate (cov).

ADDCONStraints(string)
index values of additional pre-defined constraints to pass to the
maximization.

minsize(#)
exclude all groups with sample sizes smaller than # from estimation.

initvals
calculate initial starting values for estimation.

gaps
display additional statistics.

noisily
print output during estimation.

homop
a shortcut equivalent to typing modtype(homop).

CSD(#)
constrain the single, common SD parameter to # when estimating a
HOMOP model; the default is 1.

maximize_options
options for the maximization process, see maximize for details.

Description

hetop uses maximum likelihood (ML) to estimate means and standard
deviations of underlying continuous,
normal distributions for each group in grpvar based on the frequency counts
of observations in each variable with the stem levelvar*. The estimates
are returned in a standardized metric to facilitate interpretation and
subsequent analyses.

The grpvar variable must uniquely identify the groups,
and it is recommended to number the groups consecutively 1:G. There should be
one row per group, with one column of frequency counts for each of the K
ordered categories. The count variables should be named as levelvarX where
X is a sequence of ordered integer values indicating the order of the categories.
While the order of these values must correspond to the ordering of the categories,
the magnitude of the values is arbitrary (so levelvar1, levelvar2,
levelvar3 is the same as levelvar0, levelvar4, levelvar5).
The starting value calculations are based on the integer codes; the best
starting values tend to be obtained when the values are sequential integers.

Earlier versions of hetop extended and relied on the oglm command
for Stata to carry out the estimation. Although the current version no longer
relies on oglm the estimation and approach are informed by oglm.

The data file should have the following layout:

+---------------------------------------------------------+
| grpvar levelvar0 levelvar1 levelvar2 levelvar3 |
|---------------------------------------------------------|
| 1 750 1428 741 66 |
| 2 600 1197 811 155 |
| 3 840 1653 1080 160 |
| ... ... ... ... ... |
| G 2204 3622 2507 570 |
|---------------------------------------------------------|

The hetop function can be used to estimate the following models:

HETOP: heteroskedastic ordered probit models with a unique
mean and standard deviation for each group.

PHOP: partially heteroskedastic ordered probit models
with unique means for all groups, unique standard deviations for one set of
groups and a single, pooled standard deviation estimate for remaining groups.

HOMOP: homoskedastic ordered probit model with a unique mean
estimated for each group but standard deviations of all groups constrained to
be equal.

hetop relies on a user-written ML function (hetop_lf.ado) to
carry out the ML estimation using Stata's ml features.

Note that when there are data in only two categories it is only possible to
estimate a HOMOP model. When there are more than 2 categories but there are
only non-zero counts
in 2 or fewer categories for some groups, finite ML estimates for the means and
standard deviations of those groups may not exist. hetop issues warnings
about the presence of such gruops but does not apply formal checks or other
precautions. In some cases, the estimation algorithm may
indicate convergence to a false solution in the presence of such groups. Slow
convergence or very large standard errors can be signs that the ML estimates
for some groups do not exist or are poorly identified.

Options

Required

numcats If numcats<3 only a HOMOP model can be fit.

Optional

modtype(string) This can be either homop for a HOMOP model or
hetop when fitting HETOP or PHOP models. Default is modtype(hetop).

identify(string) In order to identify the model and set the scale for the
estimates, constraints must be placed on the parameter estimates. There are
three possible types of constraints used in hetop.
The sums option constrains the
weighted sum of the log(SD) estimates and the weighted sum of the means to be 0,
where the weights for each group are the proportion of each group in the
population or the weights specified in pkvals(varname). This identification
method can only be used with G<=300 groups.
The refgroup option constraints the mean and log(SD) of a single group
to be 0. This group is referred to as a "reference" group. When
specifying identify(refgroup), an automatic reference group
will be selected unless a specific group is specified with setref(#).
The cuts option constrains the first two cut scores to -1 and 0, respectively.
If there is only 1 cut score (2 categories), the cut score will be constrained to
be 0; if a HOMOP model is fit a msingle cut score will be set to 0.
All 3 sets of constraints yield theoretically equivalent models; however,
in some cases the reference group approach may yield higher convergence rates for
problematic samples. In all cases, hetop produces "star" estimates that are
in a standardized metric as described in Reardon et al. (2017). These standardized
estimates should be identical regardless of which identification method is
used. Default is identify(sums).

setref(#) See above description of identify. Specifying setref
without specifying identify is the same as specifying
identify(refgroup).

phop(varname [ , mean ]) Specifies a 0/1 indicator variable. All groups with
varname=0 will receive a unique standard deviation estimate and all
groups with varname=1 will receive a single, pooled standard deviation
estimate. The mean option further constraints the single pooled ln(SD)
estimate to be equal to the (unweighted) mean of the ln(SD) estimates for all
groups with varname=0. Cannot be specified with the modtype(homop)
option. Note that the mean option will only work if there are 479 or
fewer unconstrained group SD estimates.

setcuts(numlist) Constrains the cut scores to be equal to the values in
numlist. The values in numlist must be ascending. This is a
shortcut to specifying these constraints with the addconstraints() option.

pkvals(varname) A variable specifying the proportion of each
group in the population.
These are used to identify the model for the sums constraints method and
are also used in the standardization and standard error calculations. No
checks made to verify that, e.g., these sum to 1.

STARTFRom(namelist) Specifies a matrix containing starting values
to use as initial values for the ML estimation. Names must match the
parameters names used in estimation (see e(b)). No checks are made to verify the
names or reasonableness of the values.

kappa(1/2) See Reardon et al. (2016) for details about this option.
Default is kappa(1).

save([ string , star prime raw se cov ]) Option to save mean and standard
deviation estimates as new variables. Including the star/prime/raw options
will include saved estimates in the star, prime, and/or raw metric.
The star metric estimates are on a standardized scale with a marginal mean of 0
and standard deviation of 1 across all groups. The prime metric estimates
are in a metric for which the weighted sum of the means and sum of the ln(SD)
estimates are 0. The raw metric estimates are relative to the model identification
used; i.e., relative to the sums, reference group, or cut score constraints. In
some cases the prime and raw metric estimates are identical. Specifying se
will also save estimated standard errors. Specifying cov will also save
the estimated sampling covariance between each group's mean and standard
deviation estimate. The new variables will have names of the form
m`metric'string, s`metric'string,
m`metric'string_se, s`metric'string_se, and z`metric'string_cov,
where `metric' is one of star/prime/raw. It is allowable to leave string
blank. Failing to specify at least one of star/prime/raw will not save
any variables.

ADDCONStraints(string) Many models fit by hetop specify constraints
automatically. This option can be used to place additional constraints on the
model. These constraints must be defined prior to running hetop.
Contraint numbers listed here will be added to the call to oglm.
WARNING: no checks are made to ensure that these constraints are compatible with
those that will automatically be applied within hetop. Use at own risk.

minsize(#) A convenience for excluding small groups from estimation.
Any group with fewer than minsize observations will be excluded from
estimation.

initvals Initial values are computed by treating the integer codes for
the ordinal categories as interval scores. A mean and standard deviation is
calculated for each group as an approximate starting value and rescaled
according to the model identification strategy. If not specified, default is
to start all mean and ln(SD) values at 0, with the cut scores set at the
standard normal quantiles of observed cumulative category proportions.

gaps If specified, an additional matrix of pairwise gaps will be returned.
This may slow down the runtime of the code.

noisily Displays output while running the estimation.

homop Equivalent to modtype(homop).

CSD(#) When estimating a HOMOP model, the common group standard deviation
value must be constrained to a fixed value. Specifying CSD(#) will constrain
it to #. By default it will be constrained to 1.

maximize_options See maximize for more details. It should not
generally be necessary to specify any of these options.

Remarks

For detailed explanation of the models fit see Reardon, Shear,
Castellano and Ho (2017).

Examples

use "hetop-example.dta" , clear

*Fit HETOP model with fewest possible options:
hetop id level , numcats(4)
ml display

*Which is equivalent to:
hetop id level , numcats(4) modtype(hetop) identify(refgroup)
ml display

*Fit HOMOP model with fewest possible options:
hetop id level , numcats(4) homop

*Which is equivalent to:
hetop id level , numcats(4) modtype(homop) identify(refgroup)

*Fit PHOP model with fewest possible options; all groups with n<=185 to have a single standard deviation estimate:
preserve
g phopvar = ng <= 185
hetop id level , numcats(4) phop(phopvar, mean) save(,star)

* show that this value is equal to the (unweighted) mean of unconstrained lns estimates:
g lnstar = ln(sstar)
table phopvar , c(m lnstar sd lnstar)
restore

*Fit HETOP model with reference group constraints; automatically selected reference group:
hetop id level , numcats(4) identify(refgroup)

*Fit HETOP model with reference group constraints; use group 1 as reference group:
hetop id level , numcats(4) setref(1)

*Use cut score identification
hetop id level , numcats(4) identify(cuts) noisily
mat lis e(craw)
mat lis e(cstar)

*Use sums constraints
hetop id level , numcats(4) identify(sums) noisily

*If integer codes are not sequential
preserve
rename level4 level6
hetop id level , numcats(4) initvals
restore

*Fix cut scores to pre-specified values
hetop id level , numcats(4) setcuts(-1 0 1)
ml display

*Choose reference group to be group 2
hetop id level , numcats(4) identify(refgroup) setref(2)
ml display

* Add a user-defined constraint
constraint free
local c1 = `r(free)'
constraint define `c1' [mean]:1.id=0.5
constraint free
local c2 = `r(free)'
constraint define `c2' [lnsigma]:1.id=0
hetop id level , numcats(4) identify(refgroup) setref(5) addconstraints(`c1' `c2')
constraint dir
constraint drop `c1' `c2'
constraint dir
ml display

Stored results

hetop stores the following in e():

Scalars

e(N) sum of counts across all groups

e(initvals) 0 if not used; 1 if used

e(mseRC) error code for mstar Std. Err. calculations

e(sseRC) error code for sstar Std. Err. calculations

e(cseRC) error code for cstar Std. Err. calculations

e(varmatsRC) error code for other Std. Err. calculations

e(icchat) estimated ICC

e(icchatratio) alternate form of estimated ICC

e(refgrp) reference group ID; 0 if reference group not used.

e(numgrps) number of groups.

e(varsigprime) estimated sampling variance of sigmaprime

e(sigmab) estimated between group variance in prime metric

e(sigmaw) estimated within group variance in prime metric

e(sigmaprime) estimated total standard deviation in prime metric

e(icchat_var) estimated sampling variance of ICC

Macros

e(grpid) name of group ID variable

e(levelvars) name of category count variables used

e(csd) value of CSD() used in estimation; "." if not used

e(phop) name of PHOP variable, "." if not used.

e(cmd) hetop

e(modtype) either "hetop" or "homop"

e(identify) either "sums" or "refgroup" or "cuts".

Matrices

e(b) parameter estimates returned by ml

e(V) variance-covariance matrix of the estimates from ml

e(Cns) constraints matrix

e(mstar) vector of estimated means in standardized metric

e(mstar_se) vector of standard errors of estimated means in standardized metric

e(sstar) vector of estimated standard deviations in standardized metric

e(sstar_se) vector of standard errors of estimated standard deviations in standardized metric

e(cstar) vector of estimated cut scores in standardized metric

e(cstar_se) vector of standard errors of estimated cut scores in standardized metric

e(Vstar) sampling variance-covariance matrix of mstar

e(Wstar) sampling variance-covariance matrix of sstar

e(Zstar) sampling variance-covariance matrix of mstar and sstar

e(Dstar) sampling variance-covariance matrix of sstar and cstar

e(Astar) sampling variance-covariance matrix of mstar and cstar

e(Bstar) sampling variance-covariance matrix of cstar

e(mprime) vector of estimated means in prime metric

e(mprime_se) vector of standard errors of estimated means in prime metric

e(sprime) vector of estimated standard deviations in prime metric

e(sprime_se) vector of standard errors of estimated standard deviations in prime metric

e(gprime) vector of estimated ln(SD) values in prime metric

e(gprime_se) vector of standard errors of estimated ln(SD) values in prime metric

e(cprime) vector of estimated cut scores in prime metric

e(cprime_se) vector of standard errors of estimated cut scores in prime metric

e(Vprime) sampling variance-covariance matrix of mprime

e(Wprime) sampling variance-covariance matrix of sprime

e(Zprime) sampling variance-covariance matrix of mprime and sprime

e(Dprime) sampling variance-covariance matrix of sprime and cprime

e(Aprime) sampling variance-covariance matrix of mprime and cprime

e(Bprime) sampling variance-covariance matrix of cprime

e(mraw) vector of estimated means in raw metric

e(mraw_se) vector of standard errors of estimated means in raw metric

e(sraw) vector of estimated standard deviations in raw metric

e(sraw_se) vector of standard errors of estimated standard deviations in raw metric

e(craw) vector of estimated cut scores in raw metric

e(craw_se) vector of standard errors of estimated cut scores in raw metric

e(Vraw) sampling variance-covariance matrix of mraw

e(Wraw) sampling variance-covariance matrix of sraw

e(Zraw) sampling variance-covariance matrix of mraw and sraw

e(Araw) sampling variance-covariance matrix of mraw and craw

e(Braw) sampling variance-covariance matrix of craw

e(initvalsmat) vector of initial starting values for all model parameters

e(refrank) vector of group ID values, sorted in order of (potentially) best to worst reference groups

e(PI) matrix used for standardization, based on e(pk) values

e(pk) vector of proportions used as population proportions for each group in computations

e(G) matrix of all pairwise standardized mean differences between groups, using an equally weighted pooled SD (if gaps option specified)

e(Gvar1) matrix of estimated sampling variances for each gap (if gaps option specified)

References

Reardon, S. F., Shear, B. R., Castellano, K. E., & Ho, A. D.
(2017). Using heteroskedastic ordered probit models to recover moments
of continuous test score distributions from coarsened data.
Journal of Educational and Behavioral Statistics, 42(1), 3–45.

DOI: 10.3102/1076998616666279

Pre-publication version available at:
http://cepa.stanford.edu/content/using-heteroskedastic-ordered-probit-models-recover-moments-continuous-test-score-distributions-coarsened-data

Acknowledgements

We gratefully acknowledge the help of Katherine Castellano, Andrew Ho,
Erin Fahle, Demetra Kalogrides, and JR Lockwood in developing and testing the HETOP
methods and code used here. hetop initially relied on and is informed by
the design of oglm, an excellent Stata program written by Richard Williams.
Richard Williams provided helpful input during development.

Development of hetop was partially funded by an Institute of Education Sciences
training grant (#R305B090016).

Citing hetop in published work

hetop is a free contribution to the research community. If you use
hetop in published work please cite it.

Suggested citation:

Shear, B. R., & Reardon, S. F. (2019).
HETOP: Stata module for estimating heteroskedastic ordered probit models with ordered frequency data.
Version 3.0.

If you have questions or suggestions for improvements, please do not hesitate
to contact the authors at the addresses below.

Authors

Benjamin R. Shear

University of Colorado Boulder

benjamin.shear@colorado.edu

sean f. reardon.

Stanford University

sean.reardon@stanford.edu