*! version 3.0 3 May 2021

help for ky_fitStephen P. Jenkins and Fernando Rios-Avila (May 2021)

Fitting mixture models of the Kapteyn-Ypma type to linked survey and administrative data

Syntax

ky_fit
r_var s_var [cl_var]
[if]
[in]
[pw fw aw iw]
[,
options]

Options

Required

r_var s_var [cl_var] Two variables are required for model
estimation. r_var refers to the measure of (log) earnings from the
administrative data. s_var refers to the measure of (log) earnings from the
survey data. cl_var is a binary variable that identifies the 'completely
labelled' group (observations for whom r_var and s_var are judged by the
analyst to be sufficiently close to each other and hence also latent 'true'
earnings to be counted as error-free)

If cl_var is not declared, a variable named __ll__ is created
identifying observations for which abs(r_var - s_var) <= delta

delta(#) Declares the value taken by variable __ll__ when cl_var
is not declared. Default is 0.

model(#) Selects the model that is fitted. Eight different models
are possible, corresponding to # = 1 through # = 8. The default value is # = 1
(Basic model). See the Description for further details.

Maximization options

from(init_specs) initial values for the coefficients

constraint(string) constraints by number to be applied

technique(algorithm_spec) maximization technique

search(srch opt) search options

robust reports robust standard errors

cluster(clvar) reports clustered standard errors

trace display current parameter vector in iteration log

difficult use a different stepping algorithm in nonconcave
regions

Display options

baselevels specifies that base levels be reported for factor
variables and for interactions with bases that cannot be inferred from
their component factor variables.

allbaselevels specifies that all base levels of factor
variables and interactions be reported.

Model specification options

For each model fitted, every parameter can be modelled as a
function of covariates using a varlist. This allows for richer
specifications in which there is (additional) heterogeneity related to
observed characteristics. Factor variable notation can be used to specify
varlist.

model(1) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

arho_s(varlist) and lpi_s(varlist)

model(2) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_w(varlist), ln_sig_w(varlist),

arho_s(varlist) and lpi_s(varlist)

model(3) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

arho_s(varlist), lpi_r(varlist) and lpi_s(varlist)

model(4) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

mu_w(varlist), ln_sig_w(varlist),

arho_s(varlist),

lpi_r(varlist), lpi_s(varlist) and lpi_w(varlist)

model(5) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

mu_w(varlist), ln_sig_w(varlist),

mu_v(varlist), ln_sig_v(varlist),

arho_r(varlist), arho_s(varlist),

lpi_r(varlist), lpi_s(varlist), lpi_w(varlist),

and lpi_v(varlist)

model(6) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

mu_v(varlist), ln_sig_v(varlist),

arho_r(varlist), arho_s(varlist),

lpi_r(varlist), lpi_s(varlist),

and lpi_v(varlist)

model(7) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

mu_w(varlist), ln_sig_w(varlist),

arho_s(varlist), arho_w(varlist),

lpi_r(varlist), lpi_s(varlist) and lpi_w(varlist)

model(8) The following parameters can be made functions of covariates:

mu_e(varlist), ln_sig_e(varlist),

mu_n(varlist), ln_sig_n(varlist),

mu_t(varlist), ln_sig_t(varlist),

mu_w(varlist), ln_sig_w(varlist),

mu_v(varlist), ln_sig_v(varlist),

arho_r(varlist), arho_s(varlist),

arho_w(varlist),

lpi_r(varlist), lpi_s(varlist), lpi_w(varlist),

and lpi_v(varlist)

aweights, fweights, iweights, and pweights are
allowed; see weight.

Description

ky_fit fits 8 finite mixture models of earnings and measurement
errors of various types using linked survey and administrative data on earnings
(or similar variables). The first four models were proposed by Kapteyn and
Ypma (2007); the second four models are generalisations of the Kapteyn-Ypma
models proposed by Jenkins and Rios-Avila (2021c). We refer here to the general
class of models as "KY" models. Other innovations relative to Kapteyn and Ypma
(2007) are: (i) we allow model parameters to be functions of covariates, and
(ii) we incorporate a potential non-zero correlation betweeen the latent true
earnings and contamination error.

The data comprise a set of observations ('workers', i = 1, ..., N),
for each of which there are 2 measures of (log) earnings: (a) from a
survey (denoted s_i) and (b) from an administrative record dataset (r_i). We
denote latent true (log) earnings by e_i. We refer to the mean of x as mu_x, and
the standard deviation (SD) of x as sig_x. The different combinations of
error-ridden and/or error-free observations characterize latent classes.
Latent class probabilities depend on the probabilities of the different
types of error.

Each of the 8 models is a finite mixture of up to nine bivariate normal
distributions. In ky_fit we label the KY models 1-8, where
model 1, the Basic Model, is the simplest, and model 8, the Extended Model, is the
most general. Next we set out out the structure of the Extended Model; the other
models are special cases of this. See Jenkins and Rios-Avila (2021b, c) for
further details.

The distribution of administrative earnings contains observations
that are correctly linked to survey records with probability pi_r,
as well as observations for which the linkage is incorrect ('mismatch') with
probability (1-pi_r). Even if observations are correctly linked, some
values of r_i may be subject to regression-to-the-mean (RTM) measurement error
with probability (1-pi_v). For mismatched individuals, observed
administrative earnings are a draw from the distribution of earnings in the
administrative data. In sum, there are three types of r_i observation:

R1: r_i = e_i, with probability pi_r * pi_v

R2: r_i = e_i + rho_r*(e_i-mu_e) + v_i , with probability pi_r * (1 - pi_v)

R3: r_i = t_i, with probability 1 - pi_r

The distribution of survey earnings contains three types of observation.
First, there are observations with earnings that are reported correctly
(i.e. without error), with probability pi_s. Second, there are observations with
earnings reported with measurement error with the error including a RTM
component, with probability (1-pi_s)*(1-pi_w). Third, there are
observations that contain additional 'contamination' error in addition to
RTM measurement error, with probability (1-pi_s)*pi_w. In sum, there are
three types of s_i observation:

S1: s_i = e_i, with probability pi_s

S2: s_i = e_i + rho_s*(e_i-mu_e) + n_i
, with probability (1-pi_s)*(1-pi_w)

S3: s_i = e_i + rho_s*(e_i-mu_e) + n_i + w_i
, with probability (1-pi_s)*pi_w

For model fitting, we follow KY and assume that errors are independently
and identically distributed normal, with the exception of e_i and w_i,
for which we assume a bivariate normal distribution with correlation rho_w:

(e_i, w_i) ~ BN([mu_e,mu_w], [(sig_e)^2,(sig_w)^2],rho_w*sig_e*sig_w)

n_i ~ N(mu_n, (sig_n)^2)

v_i ~ N(mu_v, (sig_v)^2)

t_i ~ N(mu_t, (sig_t)^2)

The Extended Model, ky_fit's model 8, has nine latent classes. It
is a mixture of nine bivariate distributions representing combinations of the
three types of administrative data observation and the three types of survey
data observation. Class 1 contains 'completely labeled' observations, i.e. those
for which survey and administrative data earnings measures are error-free and
hence also equal to latent true earnings.

Class 1: r_i ~ R1 and s_i ~ S1, with probability pi_r*pi_v*pi_s

Class 2: r_i ~ R1 and s_i ~ S2, with probability pi_r*pi_v*(1-pi_s)*(1-pi_w)

Class 3: r_i ~ R1 and s_i ~ S3, with probability pi_r*pi_v*(1-pi_s)*pi_w

Class 4: r_i ~ R2 and s_i ~ S1, with probability pi_r*(1-pi_v)*pi_s

Class 5: r_i ~ R2 and s_i ~ S2, with probability pi_r*(1-pi_v)*(1-pi_s)*(1-pi_w)

Class 6: r_i ~ R2 and s_i ~ S3, with probability pi_r*(1-pi_v)*(1-pi_s)*pi_w

Class 7: r_i ~ R3 and s_i ~ S1, with probability (1-pi_r)*pi_s

Class 8: r_i ~ R3 and s_i ~ S2, with probability (1-pi_r)*(1-pi_s)*(1-pi_w)

Class 9: r_i ~ R3 and s_i ~ S3, with probability (1-pi_r)*(1-pi_s)*pi_w

Models 1 and 2 assume that the administrative data contain no error (i.e.
no mismatch and no measurement error).

Models 3, 4, and 7 assume that the administrative data contain only
mismatch error.

Models 5, 6, and 8 assume that the administrative data contain mismatch
error and RTM error.

Models 1, 3, and 6 assume that the survey data contain only RTM
measurement error.

Models 2, 4, 5, 7 and 8 assume that the survey data contain RTM
measurement error plus contamination error.

Models 7 and 8 assume the survey contamination error is correlated with
the latent true earnings.

When fitting the models by maximum likelihood, we transform parameters
other than means to ensure their estimates lie within their theoretical ranges.
That is, we ensure that all standard deviations are strictly positive; the RTM
parameters and correlation between w_i and e_i lie between -1 and 1; and
error probabilities pi_s, pi_r, pi_w, and pi_v each lie between 0 and 1.

To report parameters in their natural metric, invert the transformations:

sig_x = exp(ln_sig_x) for x = e, n, w, t, v

rho_x = tanh(arho_x) for x = r, s, w

pi_x = logistic(lpi_x) for x = s, w, r, v

We provide post-estimation utilities ky_estat and ky_p to
enable users to derive parameters (and SEs) in their natural metric. See
Jenkins and Rios-Avila (2021b, 2021c) for details.

Users should experiment with multiple sets of initial values to check
that models converge to a global maximum rather than some local maximum. (The
risk of convergence to local maxima is greater for models with covariates.)
ky_fit fits models in a sequential fashion, beginning with simpler models
that provide starting values for more complex models. This reduces the risk of
convergence to local maxima but does not remove it altogether.

See Kapteyn and Ypma (2007) for details of models 1-4, and estimates
for a sample of Swedish workers aged 50+ years. For the same models, Jenkins
and Rios-Avila (2020) provide estimates for a sample of UK workers from
across the full range, also analyzing estimate sensitivity to the choice of the
'completely labelled' fraction (the size of class 1). Meijer, Rohwedder, and
Wansbeek (2012) derive hybrid earnings predictors of latent true earnings
combining information from administrative and survey data and model estimates,
and measures of reliability. They illustrate their methods using Kapteyn and
Ypma's (2007) Full model (what we label model 4). Jenkins and Rios-Avila (2021a)
replicate Meijer, Rohwedder, and Wansbeek's analysis, and apply their methods
to estimates of model 4 derived from UK data.

See Jenkins and Rios-Avila (2021c) for discussion of models 5-8, and
estimates for models with and without parameters expressed as functions of
covariates. They use UK data. Jenkins and Rios-Avila (2021b) discuss in greater
detail model fitting using ky_fit and post-estimation methods,
including the methods of Meijer, Rohwedder, and Wansbeek (2012) for models
1-8.

Examples

For detailed example, please see do-file "ky_example.do".

Stored results

In addition to the results stored from ml (see maximize),
ky_fit stores the following in e():

Scalars

e(ic) Number of iterations of last model.
Does not include iterations of intermediate models, if any.

e(method_c) Code for estimated model. See model specification.

Macros

e(predict) ky_p: program used to implement predict.

e(estat_cmd) ky_estat: program used to implement post-estimation statistics estat.

e(method) Description of model specification.

e(cmd) ky_fit

e(cmdline) command as typed

e(depvar) names of dependent variables, i.e. the administrative
and survey log earnings variables, and completely-labeled group identifier

e(user) name of likelihood-evaluator program. ky_ll_# with # =
1, 2, ..., 8.

References

Jenkins, S. P. and Rios-Avila, F. (2020).
Measurement errors in survey and administrative data on employment earnings:
sensitivity to the fraction assumed to have error-free earnings’,
Economics Letters, 192: 109253. https://doi.org/10.1016/j.econlet.2020.109253

Jenkins, S. P. and Rios-Avila, F. (2021a).
Measurement error in earnings data: replication of
Meijer, Rohwedder, and Wansbeek’s mixture model
approach to combining survey and register data,
Journal of Applied Econometrics, online first.
https://doi.org/10.1002/jae.2811

Jenkins, S. P. and Rios-Avila, F. (2021b).
Finite mixture models for linked survey and administrative data: estimation and
post-estimation. IZA Discussion Paper, forthcoming.
https://www.iza.org/publications/dp
For submission to The Stata Journal.

Jenkins, S. P. and Rios-Avila, F. (2021c).
Reconciling reports: modelling employment earnings and measurement errors
using linked survey and administrative data.
IZA Discussion Paper, forthcoming. https://www.iza.org/publications/dp

Kapteyn, A. and Ypma, Y. A. (2007). Measurement error and misclassification: a
comparison of survey and administrative data.
Journal of Labor Economics 25 (3): 513-551.
https://www.journals.uchicago.edu/doi/abs/10.1086/513298

Meijer, E., Rohwedder, S. and Wansbeek T. (2012). Measurement error in
earnings data: using a mixture model approach to combine survey and register data.
Journal of Business & Economic Statistics 30 (2): 191-201.
https://www.tandfonline.com/doi/abs/10.1198/jbes.2011.08166

Authors

Stephen P. Jenkins

Department of Social Policy

London School of Economics and Political Science

Houghton Street, London WC2A 2AE, UK

Email: s.jenkins@lse.ac.uk

Fernando Rios-Avila

Levy Economics Institute of Bard College

Annandale-on-Hudson, NY 12504-5000, USA

Email: friosavi@levy.org

Also see

ky_estat if installed; ky_sim if installed.