Addressing Linear Regression Models with Correlated Regressors: Some Package Development in R

Abstract

Regression analysis is widely used in many elds such as economics, nance, technology and social sciences. A linear regression model describes the relationship between dependent variable and one or more regressors. A problem named multicollinearity is the existence/presence of nearly linear dependency among regressors. The existence of severe multicollinearity questions the accuracy of the parameters estimate in a linear regression model, because the variance of the ordinary least square estimator (OLSE) would be large enough. Therefore, detection of multicollinearity can be considered as rst step for giving solution of this problem. There are several indicators (diagnostic measures) for the quanti cation of collinearity among regressors available in the literature. Widely used and the most suggested diagnostics are values of pair-wise correlations among regressors, overall R2, VIF and tolerance limit, eigenvalues, condition number and condition indices etc. However, there is no unique method that can detect and measure the existence and strength of multicollinearity in data. For remedy or reduction of collinearity among regressors, biased estimators are suggested such as the ridge and Liu estimators in literature and are alternative to the OLSE. These alternative methods also improve the accuracy of the parameter estimates of linear regression models. The biased methods resemble each other and are based on similar principals. In this study, we propose two new detection methods for indication of collinearity vi existence and developed 3 packages in R language namely mctest, lmridge and liureg. The package mctest computes already existing collinearity diagnostic measures and our proposed measures. We also compare existing and our proposed collinearity diagnostics measures for the detection of collinearity existence among regressors. Our proposed collinearity diagnostics not only perform well for simulated but also for real collinear data sets. For computation and detection of existence of collinearity mctest package can be used. The package contains functions omcdiag for computation of overall and imcdiag for individual collinearity diagnostics as described in Chapter 3, Section 3.5. For estimation and testing of ridge coe cients, the ridge package lmridge can be used to compute ridge coe cients, di erent existing biasing parameters available in literature and testing of ridge coe cients with 15 ridge related statistics such as R2, adjusted-R2, mean square error and e ective degrees of freedom etc. Similarly, estimation and testing of Liu coe cient can be done using package liureg package. It computes biasing parameters from (Liu, 1993) and the Liu related statistics. In addition, numerical comparisons between existing estimator of ridge and Liu are also done by using existing collinear data set from Hald (1952), Longley (1967) and Malinvaud (1968).