+-------+ ----+ Title +------------------------------------------------------------
ridgereg: OLS Ridge Regression Models"
+-------------------+ ----+ Table of Contents +------------------------------------------------
Syntax Options Description Saved Results References
*** Examples
Author
+--------+ ----+ Syntax +-----------------------------------------------------------
ridgereg depvar indepvars [if] [in] [weight] , model(orr|grr1|grr2|grr3) [ kr(numlist) mfx(lin|log) lmcl diag dn weights(type) wvar(varname) tolog predict(new_var) resid(new_var) tolerance(#) iter(#) ]
+---------+ ----+ Options +----------------------------------------------------------
options Description -------------------------------------------------------------------------
kr(#) Ridge k value, must be in the range (0 < k < 1).
model(orr, grr1, grr2, grr3).
ridgereg can estimate 4 types of Ridge Regressions family models.
model(orr) : Ordinary Ridge Regression [Judge, et al(1988, p.878) eq.21.4.2]. model(grr1): Generalized Ridge Regression [Judge, et al(1988, p.881) eq.21.4.12]. model(grr2): Iterative Generalized Ridge Regression [Judge, et al(1988, p.881) eq.21.4.12]. model(grr3): Adaptive Generalized Ridge Regression [Strawderman(1978)].
model(grr1, grr2, grr3) and kr(#) must be not combined.
IF kr(0) in model(orr, grr1, grr2, grr3), this means that a value of (kr = 0), so the model will be an OLS regression.
IF you dont set model, then OLS Regression will be estiamted.
model(orr) allows to set kr value that should be between zero and one (0 < kr < 1) to convert OLS regression to RIDGE regression, the diagonal elements of the X'X matrix are augmented by kr.
wvar(varname) Weighted Variable Name
weights(varname) Weighted Variable Type Options
Type Options Description
weights(yh) Yh - Predicted Value weights(yh2) Yh^2 - Predicted Value Squared weights(abse) abs(E) - Absolute Value of Residual weights(e2) E^2 - Residual Squared weights(le2) log(E^2) - Log Residual Squared weights(x) (x) Variable weights(xi) (1/x) Inverse Variable weights(x2) (x^2) Squared Variable weights(xi2) (1/x^2) Inverse Squared Variable
tolog Convert dependent and independent variables to LOG Form in the memory for Log-Log regression. tolog Transforms depvar and indepvars to Log Form without lost the original data variables
iter(#) number of iterations; Default is iter(50)
tolerance(#) tolerance for coefficient vector; Default is tol(0.00001)
mfx(lin, log) type of functional form, either Linear model (lin), or Log-Log model (log), to compute Marginal Effects and Elasticities.
- In Linear Model Marginal Effects = Coefficients (Bm), and Elasticities =(Bm X/Y).
- In Log-Log Model: Elasticities = Coefficients (Es), and Marginal Effects =(Es Y/X).
lmcl Multicollinearity Diagnostic Tests and Criteria
* Correlation matrix * (VIF): Variance Inflation Factor * (1/VIF): Tolerance * Eigenvalues (computed from correlation matrix) * (C_Number): Condition Number * (C_Index): Condition Index * (R2_xi,X): R2-squared between each Xi and other independent variabl > es
Display Farrar-Glauber Multicollinearity tests: * Farrar-Glauber Multicollinearity Chi2-Test * Farrar-Glauber Multicollinearity F-Test * Farrar-Glauber Multicollinearity t-Test
Display Multicollinearity Ranges: * Determinant of |X'X|
* Theil R2 Multicollinearity Effect [Theil(1971, p.179)], [Judge, et > al(1988, p.870)]
* Gleason-Staelin Q0 Multicollinearity Range * Heo Multicollinearity Range Q1 * Heo Multicollinearity Range Q2 * Heo Multicollinearity Range Q3 * Heo Multicollinearity Range Q4 * Heo Multicollinearity Range Q5 * Heo Multicollinearity Range Q6
+-------------+ ----+ Description +------------------------------------------------------
Ridge regression is considered as a multicollinearity remediation method. General form of Ridge Coefficients and Covariance Matrix are:
Br = inv[X'X + kI] X'Y
Cov=Sig^2 * inv[X'X + kI] (X'X) inv[X'X + kI]
where: Br = Ridge Coefficients Vector (k x 1). Cov = Ridge Covariance Matrix (k x k). Y = Dependent Variable Vector (N x 1). X = Independent Variables Matrix (N x k). k = Ridge Value (0 < kr < 1). I = Diagonal Matrix of Cross Product Matrix (Xs'Xs). Xs = Standardized Variables Matrix in Deviation from Mean. Sig2 = (Y-X*Br)'(Y-X*Br)/DF
- Multicollinearity Detection: 1. A high F statistic or R2 leads to reject the joint hypothesis that all of the coefficients are zero, but individual t-statistics are low. 2. High simple correlation coefficients are sufficient but not necessary for multicollinearity. 3. One can compute condition number. That is, the ratio of largest to smallest root of the matrix x'x. This may not always be useful as the standard errors of the estimates depend on the ratios of elements of characteristic vectors to the roots.
- Multicollinearity Remediation: 1. Use prior information or restrictions on the coefficients. One clever way to do this was developed by Theil and Goldberger. See tgmixed, and Theil(1971, pp 347-352). 2. Use additional data sources. This does not mean more of the same. It means pooling cross section and time series. 3. Transform the data. For example, inversion or differencing. 4. Use a principal components estimator. This involves using a weighted average of the regressors, rather than all of the regressors. 5. Another alternative regression technique is ridge regression. This involves putting extra weight on the main diagonal of X'X. 6. Dropping troublesome RHS variables. This begs the question of specification error.
+---------------+ ----+ Saved Results +----------------------------------------------------
ridgereg saves the following in e():
e(N) Number of obs e(r2c) R2 e(r2c_a) adj R2 e(r2cc) Corrected R2 e(r2cc_a) adj Corrected R2 e(r2u) Raw Moment R2 e(r2u_a) Adj Raw Moment R2 e(sig) Sigma (MSE) e(wald) Wald Test e(waldp) Wald Test P-Value e(llf) Log Likelihood Function e(aic) AKAIKE Final Prediction Error e(sc) Schwartz Criterion e(laic) Akaike Information Criterion ln AIC e(lsc) Schwarz Criterion Log SC e(fpe) Amemiya Prediction Criterion e(hq) Hannan-Quinn Criterion e(rice) Rice Criterion e(shibata) Shibata Criterion e(gcv) Craven-Wahba Generalized Cross Validation-GCV e(f) F Test e(fp) F Test P-Value
+------------+ ----+ References +-------------------------------------------------------
D. Belsley (1991) "Conditioning Diagnostics, Collinearity and Weak Data in Regression", John Wiley & Sons, Inc., New York, USA.
D. Belsley, E. Kuh, and R. Welsch (1980) "Regression Diagnostics: Identifying Influential Data and Sources of Collinearity", John Wiley & Sons, Inc., New York, USA.
Evagelia, Mitsaki (2011) "Ridge Regression Analysis of Collinear Data", http://www.stat-athens.aueb.gr/~jpan/diatrives/Mitsaki/chapter2.pdf
Farrar, D. and Glauber, R. (1976) "Multicollinearity in Regression Analysis: the Problem Revisited", Review of Economics and Statistics, 49; 92-107.
Hoerl A. E. (1962) "Application of Ridge Analysis to Regression Problems", Chemical Engineering Progress, 58; 54-59.
Hoerl, A. E. and R. W. Kennard (1970a) "Ridge Regression: Biased Estimation for Non-Orthogonal Problems", Technometrics, 12; 55-67.
Hoerl, A. E. and R. W. Kennard (1970b) "Ridge Regression: Applications to Non-Orthogonal Problems", Technometrics, 12; 69-82.
Hoerl, A. E. ,R. W. Kennard, & K. Baldwin (1975) "Ridge Regression: Some Simulations", Communications in Statistics, A, 4; 105-123.
Hoerl, A. E. and R. W. Kennard (1976) "Ridge Regression: Iterative Estimation of the Biasing Parameter", Communications in Statistics, A, 5; 77-88.
Judge, Georege, William. E. Griffiths, R. Carter Hill, Helmut Lutkepohl, & Tsoung-Chao Lee (1985) "The Theory and Practice of Econometrics", 2nd ed., John Wiley & Sons, Inc., New York, USA; 914-916.
Judge, Georege, R. Carter Hill, William . E. Griffiths, Helmut Lutkepohl, & Tsoung-Chao Lee (1988) "Introduction To The Theory And Practice Of Econometrics", 2nd ed., John Wiley & Sons, Inc., New York, USA; 878-882.
Marquardt D.W. (1970) "Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation", Technometrics, 12; 591-612.
Marquardt D.W. & R. Snee (1975) "Ridge Regression in Practice", The Anerican Statistician, 29; 3-19.
Pidot, George (1969) "A Principal Components Analysis of the Determinants of Local Government Fiscal Patterns", Review of Economics and Statistics, Vol. 51; 176-188.
Rencher, Alvin C. (1998) "Multivariate Statistical Inference and Applications", John Wiley & Sons, Inc., New York, USA; 21-22.
Strawderman, W. E. (1978) "Minimax Adaptive Generalized Ridge Regression Estimators", Journal American Statistical Association, 73; 623-627.
Theil, Henri (1971) "Principles of Econometrics", John Wiley & Sons, Inc., New York, USA.
William E. Griffiths, R. Carter Hill and George G. Judge (1993) "Learning and Practicing Econometrics", John Wiley & Sons, Inc., New York, USA.
+----------+ ----+ Examples +---------------------------------------------------------
(1) Example of Ridge regression models, is decribed in: [Judge, et al(1988, p.882)], and also Theil R2 Multicollinearity Effect in: [Judge, et al(1988, p.872)], for Klein-Goldberger data.
clear all
sysuse ridgereg1.dta, clear
ridgereg y x1 x2 x3 , lmcl diag
ridgereg y x1 x2 x3 , model(orr) kr(0.5) lmcl diag
ridgereg y x1 x2 x3 , model(grr1)
ridgereg y x1 x2 x3 , model(grr2)
ridgereg y x1 x2 x3 , model(grr3)
ridgereg y x1 x2 x3 , model(grr2) mfx(lin)
(2) Example of Gleason-Staelin, and Heo Multicollinearity Ranges, is decribed in: [Rencher(1998, pp. 20-22)].
clear all
sysuse ridgereg2.dta, clear
ridgereg y x1 x2 x3 x4 x5 , lmcl
(3) Example of Farrar-Glauber Multicollinearity Chi2, F, t Tests is decribed in:[Evagelia(2011, chap.2, p.23)].
clear all
sysuse ridgereg3.dta, clear
ridgereg y x1 x2 x3 x4 x5 x6 , lmcl
+--------+ ----+ Author +-----------------------------------------------------------
Emad Abd Elmessih Shehata Assistant Professor Agricultural Research Center - Agricultural Economics Research Institute - Eg > ypt Email: emadstat@hotmail.com WebPage: http://emadstat.110mb.com/stata.htm WebPage at IDEAS: http://ideas.repec.org/f/psh494.html WebPage at EconPapers: http://econpapers.repec.org/RAS/psh494.htm
+-------------------+ ----+ ridgereg Citation +------------------------------------------------
Shehata, Emad Abd Elmessih (2011) RIDGEREG: "Stata Module to Estimate OLS Ridge Regression Models"
http://ideas.repec.org/c/boc/bocode/s457347.html
http://econpapers.repec.org/software/bocbocode/s457347.htm
Also see
Online: ridgereg, tgmixed, ridge2sls, ridgeliml, ridgegmm, ridgemelo, (if installed).