Best subset selection for eliminating multicollinearity. (English) Zbl 1382.90068

Tamura, Ryuta; Kobayashi, Ken; Takano, Yuichi; Miyashiro, Ryuhei; Nakata, Kazuhide; Matsui, Tomomi

J. Oper. Res. Soc. Japan 60, No. 3, 321-336 (2017).

Summary: This paper proposes a method for eliminating multicollinearity from linear regression models. Specically, we select the best subset of explanatory variables subject to the upper bound on the condition number of the correlation matrix of selected variables. We first develop a cutting plane algorithm that, to approximate the condition number constraint, iteratively appends valid inequalities to the mixed integer quadratic optimization problem. We also devise a mixed integer semidefinite optimization formulation for best subset selection under the condition number constraint. Computational results demonstrate that our cutting plane algorithm frequently provides solutions of better quality than those obtained using local search algorithms for subset selection. Additionally, subset selection by means of our optimization formulation succeeds when the number of candidate explanatory variables is small.

MSC:

90C11	Mixed integer programming
90C05	Linear programming
90C22	Semidefinite programming

Keywords:

optimization; statistics; subset selection; multicollinearity; linear regression; mixed integer semidefinite optimization