Enhancing Regression Models with Regularization and Hybrid Techniques to Address Multicollinearity

Bassuny, Abdel-reheem Awad; Salem, Hanaa Abdel-Reheem Ibrahim

doi:10.21608/caf.2025.455702

Enhancing Regression Models with Regularization and Hybrid Techniques to Address Multicollinearity

المؤلفون

¹ Lecturer at the Higher Institute of Management in EL Mahalla El-Kubra PhD Statistics, Faculty of Commerce, Tanta University, Egypt

² Assistant professor in Statistics, Faculty of Commerce, Tanta University, Egypt.

10.21608/caf.2025.455702

المستخلص

This study investigates the application of regularization techniques—Ridge, Lasso, Elastic Net, and their ensembles (Ridge-Elastic Net and Lasso-Elastic Net)—to correct multicollinearity in regression models of forecasting internal migration rates. With a sample dataset of 250 regions (2020–2024) and 12 highly correlated predictors, such as income, unemployment, and healthcare quality, we compare these techniques with ordinary least squares (OLS). Multicollinearity is confirmed with high Variance Inflation Factors (VIF > 10), high correlations (e.g., r = 0.85 between income and cost of living), and eigenvalue. Results show that Elastic Net and Ridge-Elastic Net are superior, with the lowest MAE (0.17), MSE (0.26), and the highest R² (0.84), while exhibiting moderate variable selection (excluding population density). Lasso-Elastic Net and Lasso simplify models to the exclusion of transportation and population density but also yield slightly poor performance (MAE = 0.18, MSE = 0.27, R² = 0.83). Ridge attains Elastic Net's prediction performance but retains all variables, while OLS is poor (MAE = 0.20, MSE = 0.30, R² = 0.80). Elastic Net and Ridge-Elastic Net are the best picks for most accuracy, while Lasso-Elastic Net is preferred in scenarios that appreciate model simplicity. The findings highlight the strength of regularization in enhancing model stability and predictive accuracy in the presence of multicollinearity.

الكلمات الرئيسية