A Monte Carlo Comparison of the Statistical Performance of Selected Robust Linear Regression Estimators under Varying Sample Sizes and Outlier Contamination Proportions

• Mohammed Abdulmoula H. Raheel

  General Department, Faculty of Science, University of Benghazi, Benghazi, Libya

  Author

• Zadm Rafall Salih

  Department of Mathematics, College of Education, University of Benghazi, Benghazi, Libya

  Author

This study compares the statistical performance of Ordinary Least Squares (OLS), Huber M-estimation, Least Trimmed Squares (LTS), and MM-estimation under varying sample sizes and outlier contamination proportions using Monte Carlo simulation. The data-generating model contains two independent standard-normal explanatory variables, with true parameters β₀ = 2, β₁ = 3, and β₂ = 2. Four sample sizes (n = 40, 60, 100, and 200) and four contamination proportions (π = 0%, 5%, 10%, and 20%) were considered, producing sixteen simulation conditions. Vertical outliers were introduced through the regression error term, and each condition was replicated 1,000 times. Estimator performance was evaluated using bias, variance, mean squared error (MSE), root mean squared error (RMSE), and MSE-based relative efficiency. OLS produced the lowest MSE and RMSE under uncontaminated conditions, but its performance deteriorated markedly as contamination increased. The robust estimators remained more stable under contamination; Huber M-estimation was competitive at lower contamination levels, LTS became increasingly competitive as contamination increased, and MM-estimation generally produced the lowest MSE and RMSE and the highest relative efficiency under contaminated conditions. Increasing sample size improved estimation precision for all estimators but did not remove the sensitivity of OLS to outliers. These findings indicate that estimator selection should be guided by sample characteristics and the expected presence and degree of contamination.

1. Mohammed Abdulmoula H. Raheel, General Department, Faculty of Science, University of Benghazi, Benghazi, Libya

   General Department, Faculty of Science, University of Benghazi, Benghazi, Libya

2. Zadm Rafall Salih, Department of Mathematics, College of Education, University of Benghazi, Benghazi, Libya

   Department of Mathematics, College of Education, University of Benghazi, Benghazi, Libya

Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust statistics: The approach based on influence functions. John Wiley & Sons.

Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101. doi:10.1214/aoms/1177703732 DOI: https://doi.org/10.1214/aoms/1177703732

Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. The Annals of Statistics, 1(5), 799–821. doi:10.1214/aos/1176342503 DOI: https://doi.org/10.1214/aos/1176342503

Huber, P. J., & Ronchetti, E. M. (2009). Robust statistics (2nd ed.). John Wiley & Sons. doi:10.1002/9780470434697 DOI: https://doi.org/10.1002/9780470434697

Maronna, R. A., Martin, R. D., Yohai, V. J., & Salibián-Barrera, M. (2019). Robust statistics: Theory and methods (with R) (2nd ed.). John Wiley & Sons. doi:10.1002/9781119214656 DOI: https://doi.org/10.1002/9781119214656

Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102. doi:10.1002/sim.8086 DOI: https://doi.org/10.1002/sim.8086

Rousseeuw, P. J., Daniels, B., & Leroy, A. (1984). Applying robust regression to insurance. Insurance: Mathematics and Economics, 3(1), 67–72. doi:10.1016/0167-6687(84)90020-9 DOI: https://doi.org/10.1016/0167-6687(84)90020-9

Rousseeuw, P. J. (1984). Least median of squares regression. Journal of the American Statistical Association, 79(388), 871–880. doi:10.1080/01621459.1984.10477105 DOI: https://doi.org/10.1080/01621459.1984.10477105

Rousseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier detection. John Wiley & Sons. doi:10.1002/0471725382 DOI: https://doi.org/10.1002/0471725382

Rousseeuw, P. J., & Van Driessen, K. (2006). Computing LTS regression for large data sets. Data Mining and Knowledge Discovery, 12, 29–45. doi:10.1007/s10618-005-0024-4 DOI: https://doi.org/10.1007/s10618-005-0024-4

Rousseeuw, P. J., & Yohai, V. J. (1984). Robust regression by means of S-estimators. In J. Franke, W. Härdle, & D. Martin (Eds.), Robust and nonlinear time series analysis (Lecture Notes in Statistics, Vol. 26, pp. 256–272). Springer. doi:10.1007/978-1-4615-7821-5_15 DOI: https://doi.org/10.1007/978-1-4615-7821-5_15

Salibian-Barrera, M., & Yohai, V. J. (2006). A fast algorithm for S-regression estimates. Journal of Computational and Graphical Statistics, 15(2), 414–427. doi:10.1198/106186006X113629 DOI: https://doi.org/10.1198/106186006X113629

Verardi, V., & Croux, C. (2009). Robust regression in Stata. The Stata Journal, 9(3), 439–453. doi:10.1177/1536867X0900900306 DOI: https://doi.org/10.1177/1536867X0900900306

Yohai, V. J. (1987). High breakdown-point and high efficiency robust estimates for regression. The Annals of Statistics, 15(2), 642–656. doi:10.1214/aos/1176350366 DOI: https://doi.org/10.1214/aos/1176350366

Yu, C., & Yao, W. (2017). Robust linear regression: A review and comparison. Communications in Statistics - Simulation and Computation, 46(8), 6261–6282. doi:10.1080/03610918.2016.1202271 DOI: https://doi.org/10.1080/03610918.2016.1202271

• PDF

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.