Title

mipolate -- Interpolate (extrapolate) values

Syntax

mipolate
yvar
xvar
[if] [in]
,
generate(newvar)
[
linear
cubic
spline
pchip
idw[(power)]
forward
backward
nearest
groupwise
ties(ties_rule)
epolate
]

by is allowed; see [D] by.

Description

mipolate creates in newvar an interpolation of yvar on
xvar for missing values of yvar.

mipolate uses one of the following methods: linear, cubic, cubic
spline, pchip (piecewise cubic Hermite interpolation), idw (inverse
distance weighted), forward, backward,
nearest neighbour, groupwise. The default method is linear.

Interpolation requires that yvar be a function of xvar, so
yvar is also interpolated for tied values of xvar. When
yvar is not missing and xvar is neither missing nor repeated,
the value of newvar is just yvar.

Remarks

mipolate does not require tsset or xtset data and makes no
check for, or use of, any such settings. With panel data, it will be essential
to specify a panel identifier to by:.

mipolate pays no special attention to extended missing values .a
to .z.

Options

generate(newvar) is required and specifies the name of the
new variable to be created.

linear specifies linear interpolation using the known
values of yvar before and after any missing values. This is the
default method.

cubic specifies cubic interpolation, using exact fitting of
a cubic curve to two data points before and two data points after each
observation for which yvar is missing. Missing values are thus
produced whenever fewer than two data points are present on either side.
Note that this is not a spline method.

spline specifies natural cubic spline interpolation. The
method uses Mata functions spline3() and spline3eval(). If
desired see help and in turn Mata source code here.
That code is a translation of code originally given by Herriot and
Reinsch (1973).

pchip specifies piecewise cubic Hermite interpolation. For
a lucid account, see Moler (2004, Ch.3). This method uses piecewise
cubics that join smoothly, so that both the interpolated function and
its first derivative are continuous. In addition, the interpolant is
shape-preserving in the sense that it cannot overshoot locally; sections
in which observed yvar is increasing, decreasing or constant with
xvar remain so after interpolation, and local extremes (maxima,
maxima) also remain so. This interpolation method also extrapolates.

idw[(power)] specifies inverse distance
weighted interpolation. This method uses a weighted average of
non-missing values, the weights being reciprocals of the powered
distance between values, the power being zero or positive. The default
power is 2; any other choice must be specified. Thus with power 2,
values at distance 1 from a point with unknown values have weight 1,
values at distance 2 from a point have weight 1/4, distance 3 weight
1/9, and so forth. If the power is 0, all known points have equal
weight and the interpolant reduces to the average of all values. As the
power becomes large, only those values that are nearest have appreciable
weight. This interpolation method also extrapolates.

forward specifies forward interpolation, so that any known
value just before one or more missing values is copied in cascade to
provide interpolated values, constant within any such block.

backward specifies backward interpolation, so that any known
value just after one or more missing values is copied in cascade to
provide interpolated values, constant within any such block.

nearest specifies nearest neighbour interpolation, which
means using known values of yvar either before or after missing
values, depending on which is nearer in terms of xvar.
When values before and after are equally distant from a known value,
there is a choice of rules that may be applied. The default rule uses
the mean of the two values. The ties() option provides
alternative rules. This method also extrapolates, as unknown values
before the first known value and unknown values after the last known
value are replaced by those respective known values.

groupwise specifies that non-missing values be copied to
missing values if, and only if, just one distinct non-missing value
occurs in each group. Thus a group of values ., 42, ., . qualifies as 42
is not missing and is the only non-missing value in the group. Hence the
missing values in the group will be replaced with 42 in the new
variable. By the same rules 42, ., 42, . qualifies but 42, ., 43, .
does not. Normally, but not necessarily, this option is used in
conjunction with by:, which is how groups are specified; otherwise
the (single) group is the entire set of observations being used.

Note that xvar is strictly irrelevant for this method, as
order of values is immaterial. To keep syntax consistent, it should be
specified any way.

ties() specifies an alternative to the default rule for the
nearest option, whereby previous and next values equally distant
from a given point are averaged. You may choose one of after
(value after is used), before (value before is used),
minimum (smaller value is used), or maximum (larger
value is used). As indicated, any unambiguous abbreviation is allowed.

epolate specifies that values be both interpolated and
linearly extrapolated. Interpolation only is the default with
linear, cubic and spline. This option is ignored
with pchip, forward, backward, nearest and
groupwise, which apply their own kinds of extrapolation.

Examples

Setup

. webuse ipolxmpl1

List the data

. list, sep(0)

Create ly1, cy1, sy1, py1, ny1
containing interpolations of y on x for missing values of
y

. mipolate y x, gen(ly1)

. mipolate y x, cubic gen(cy1)

. mipolate y x, spline gen(sy1)

. mipolate y x, pchip gen(py1)

. mipolate y x, nearest gen(ny1)

Use alternative rules for handling ties:

. foreach r in after before max min {

. mipolate y x, nearest ties(`r') gen(ny`r')

. }

List the results

. list, sep(0)

Setup

. webuse ipolxmpl2

Show years for which the circulation data are missing

. tabulate circ year if circ == ., missing

Create pchipcirc containing a pchip interpolation of circ on
year for missing values of circ and perform this calculation
separately for each magazine

. by magazine: mipolate circ year, pchip gen(pchipcirc)

Moler's example

. clear

. set obs 6

. matrix y = (16, 18, 21, 17, 15, 12)'

. gen y = y[_n, 1]

. gen x = _n

. set obs 61

. replace x = (_n + 1)/10 in 7/L

. mipolate y x, pchip gen(pchip)

. line pchip x, sort || scatter y x

Sandbox for groupwise:

. clear

. set obs 10

. gen x = _n

. gen group = 1 in 1

. replace group = 2 in 2/3

. replace group = 3 in 4/6

. replace group = 4 in 7/10

. gen y = .

. replace y = 2 in 2

. replace y = 5 in 5

. replace y = 10 in 10

. bysort group: mipolate y x, gen(y1) groupwise

. list, sepby(group)

. replace y = 9 in 9

. * should fail:

. bysort group: mipolate y x, gen(y2) groupwise

Author

Nicholas J. Cox, Durham University, U.K.

n.j.cox@durham.ac.uk

Acknowledgments

Most of the Mata code for the pchip option is a translation
of MATLAB code given by Moler (2004).

References

Hamming, R.W. 1973. Numerical methods for scientists and engineers.
New York: McGraw-Hill.

Herriot, J.G. and C.H. Reinsch. 1973.
Algorithm 472: procedures for natural spline interpolation.
Communications of the Association for Computing Machinery
16: 763-768.

Lancaster, P. and K. Salkauskas. 1986.
Curve and surface fitting: an introduction.
London: Academic Press.
[capital S of Salkauskas should bear caron or wedge diacritic]

Moler, C. 2004. Numerical Computing with MATLAB.
Philadelphia: SIAM. Chapter 3.
(also available in slightly different form at
http://www.mathworks.com/moler/interp.pdf)

Morton, B.R. 1964. Numerical approximation.
London: Routledge and Kegan Paul.

Press, W.H., S.A. Teukolsky, W.T. Vetterling, B.P. Flannery. 2007.
Numerical recipes: the art of scientific computing.
Cambridge: Cambridge University Press.