*! version 1.0 09sep2013

help krls

Title

krls -- Kernel-based Regularized Least Squares

Syntax

krls [depvar] covar
[if] [in]
[,
options]

Main
deriv(string) save derivatives in current dataset with the specified prefix

sderiv(filename) save derivatives in a named dataset

graph generate histograms of the derivatives

vcov save fitted variance-covariance matrix to memory

svcov(filename) save fitted variance-covariance matrix in a named dataset

keep(filename) saves the output table in a named dataset

Advanced
lambda(real) manually set lambda; selected via optimization routine by default

ltolerance(real) manually set tolerance for lambda optimization

lowerbound(real) manually set lower bound for lambda search window

sigma(real) manually set sigma; defaults to the number of variables specified in covar

suppress suppresses the calculation of derivatives and the output table

quantile(numlist) allows the user to specify the quantiles displayed within the results table

covar is a varlist that may include factor variables, see fvvarlist.

Description

krls implements Kernel-based Regularized Least Squares (KRLS), a machine learning method to fit multidimensional functions y = f(x)
for regression and classification problems without relying on linearity or additivity assumptions.
KRLS finds the best fitting function by minimizing the squared loss of a Tikhonov regularization problem,
using Gaussian kernels as radial basis functions. For further details see Hainmueller and Hazlett (2013).

Required

depvar varname that specifies the dependent variable (continuous or binary).

covar varlist that specifies the covariates (continuous or binary). At least one variable should be specified.

Note

The krls routine constructs a n x n matrix. Please ensure that your system has sufficient memory before using the command on large datasets.

Options

Main

deriv(string) saves a n x k matrix of pointwise derivatives from the Gaussian kernel into the current dataset. These derivatives describe the marginal effect of covariates at each datapoint. Appended with the prefix specified by the user.

sderiv(filename) stores the matrix of pointwise derivatives in a new dataset specified by filename.dta

graph generates histograms for the pointwise derivatives.

vcov stores the n x n fitted variance-covariance matrix for fitted values in e(Vcov_y). By default this matrix is not stored in memory.

svcov(filename) stores the fitted variance-covariance matrix in a new dataset specified by filename.dta

keep(filename) stores the output table in a new dataset specified by filename.dta.

Advanced

lambda(real) allows the user to manually provide a value for lambda, which governs the tradeoff between model fit and complexity.
By default, krls selects lambda by minimizing the sum of the squared leave-one-out (Loo) errors.

ltolerance(real) overrides the default tolerance used in the optimization routine, allowing the user to increase or decrease the
sensitivity of the lambda optimization. By default, convergence is achieved when the difference in the sum of squared leave-one-out (Loo)
errors between the i and the i+1 iteration is less than n * 10^-3

lowerbound(real) can be used to manually specify a lower bound for the labmda search window. By default, the lower bound is dynamically
set according to eigenvalue size. Set this value to 0 for maximum sensitivity.

suppress instructs the routine to avoid calculating derivatives and suppress all output. This offers a speed increase when the user
is primarily interested in attaining predicted values.

quantile(numlist) allows the user to manually specify the values displayed within the results table. By default, the 25th, 50th, and 75th percentiles of the pointwise derivatives are displayed. Users may input a minimum of 1 and a maximum of 3 quantiles.

Examples

Load example data

. use growthdata.dta

Basic syntax

. krls growth yearsschool assassinations

. krls growth yearsschool assassinations, deriv(myprefix)

. krls growth yearsschool assassinations, graph

The command returns the average of the point-wise derivatives for continuous variables.

. gen yearsschool3 = (yearsschool>3)

. krls growth rgdp60 tradeshare yearsschool3 assassinations

As well as first differences for binary variables (marked with an asterix).

Advanced syntax

. krls growth yearsschool assassinations, lambda(.8) sigma(3)

. krls growth yearsschool assassinations, ltolerance(.01)

. krls growth yearsschool assassinations, ltolerance(.001) lowerbound(0)

If you wish to adjust the sensitivity of the optimization routine or replicate an existing analysis,
use the advanced syntax to override or modify the lambda optimization routine.

Viewing the Variance-Covariance Matrix

. krls growth yearsschool assassinations, vcov

. matrix list e(Vcov_y)

In order to conserve memory, the fitted variance-covariance matrix is not saved by default.

Adjusting the Results Table

. krls growth yearsschool assassinations, quantile(.1 .5 .9)

Viewing fitted values, standard errors, etc using predict

. krls growth yearsschool assassinations

. predict myname_fitted, fitted

. predict myname_se, se

Out of sample Prediction

. set seed 1

. gen double u = runiform()

. sort u

. gen insample = 1

. replace insample = 0 in 1/5

. krls growth rgdp60 tradeshare yearsschool assassinations if insample==1

. predict myname2_fitted, fitted

. predict myname2_residuals, resid

Saved results

By default, krls ereturns the following results, which can be displayed by typing ereturn list after
krls is finished (also see ereturn).

Scalars

e(Looloss) the sum of squared leave-out-one (LOO) error

e(lambda) the lambda value used (manual or selected via optimization)

e(R2) the R squared of the final model

e(sigma) the sigma value used (manual or selected via optimization)

e(Effective_df) the effective degrees of freedom

e(tolerance) the tolerance (manual or selected via optimization)

e(lowerbound) the lowerbound (manual or selected via optimization)

Macros

e(cmd) krls

e(depvar) the dependent variable

e(indvar) the independent variables

Matrices

e(b) pointwise derivatives or first differences

e(Output) the full output table in matrix form

e(Coeffs) krls coefficients; used in predict

e(Vcov_c) the krls coefficient variance covariance matrix; used in predict

e(Vcov_y) the variance covariance matrix for fitted values.

References

Hainmueller, J & Hazlett, C, 2014. "Kernel Regularized Least Squares: Reducing Misspecification Bias with a Flexible and Interpretable Machine Learning Approach.” Political Analysis (Spring 2014) 22(2): 143-168.

Authors

Jeremy Ferwerda, ferwerda@mit.edu
MIT

Jens Hainmueller, jhain@stanford.edu
Stanford

Chad Hazlett, chazlett@mit.edu
UCLA