×

zbMATH — the first resource for mathematics

Gene selection and prediction for cancer classification using support vector machines with a reject option. (English) Zbl 1328.62586
Summary: In cancer classification based on gene expression data, it would be desirable to defer a decision for observations that are difficult to classify. For instance, an observation for which the conditional probability of being cancer is around 1/2 would preferably require more advanced tests rather than an immediate decision. This motivates the use of a classifier with a reject option that reports a warning in cases of observations that are difficult to classify. In this paper, we consider a problem of gene selection with a reject option. Typically, gene expression data comprise of expression levels of several thousands of candidate genes. In such cases, an effective gene selection procedure is necessary to provide a better understanding of the underlying biological system that generates data and to improve prediction performance. We propose a machine learning approach in which we apply the \(l_{1}\) penalty to the SVM with a reject option. This method is referred to as the \(l_{1}\) SVM with a reject option. We develop a novel optimization algorithm for this SVM, which is sufficiently fast and stable to analyze gene expression data. The proposed algorithm realizes an entire solution path with respect to the regularization parameter. Results of numerical studies show that, in comparison with the standard \(l_{1}\) SVM, the proposed method efficiently reduces prediction errors without hampering gene selectivity.

MSC:
62P10 Applications of statistics to biology and medical sciences; meta analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62J07 Ridge regression; shrinkage estimators (Lasso)
62-07 Data analysis (statistics) (MSC2010)
Software:
SparseLOGREG; LASSO
PDF BibTeX Cite
Full Text: DOI
References:
[1] Ambroise, C.; McLachlan, G.J., Selection bias in gene extraction on the basis of microarray gene expression data, Proceedings of the national Academy of sciences USA, 99, 6562-6566, (2002) · Zbl 1034.92013
[2] Balakrishnan, S.; Madigan, D., Algorithms for sparse linear classifiers in the massive data, Journal of machine learning research, 9, 313-337, (2008) · Zbl 1225.68148
[3] Bartlett, P.; Wegkamp, M.H., Classification with a reject option using a hinge loss, Journal of machine learning research, 9, 1823-1840, (2008) · Zbl 1225.62080
[4] Bertsekas, D.P., Nonlinear programming, (2003), Athena Scientific · Zbl 0935.90037
[5] Bhattacharjee, A.; Richards, W.G.; Staunton, J.; Li, C.; Monti, S.; Vasa, P.; Ladd, C.; Beheshti, J.; Bueno, R.; Gillette, M.; Loda, M.; Weber, G.; Mark, E.J.; Lander, E.S.; Wong, W.; Johnson, B.E.; Golub, T.R.; Sugarbaker, D.J.; Meyerson, M., Classification of human lung carcinomas by MRNA expression profiling reveals distinct adenocarcinoma subclasses, Proceedings of the national Academy of sciences USA, 98, 13790-13795, (2001)
[6] Chow, C.K., On optimum recognition error and reject tradeoff, IEEE transactions on information theory, 16, 41-46, (1970) · Zbl 0185.47804
[7] Efron, B.; Hastie, T.; Johnstone, I.; Tibshirani, R., Least angle regression, The annals of statistics, 32, 407-499, (2004) · Zbl 1091.62054
[8] Genkin, A.; Lewis, D.D.; Madigan, D., Large-scale Bayesian logisitic regression for text categorization, Technometrics, 49, 291-304, (2007)
[9] Greenstein, E., Best subset selection, persistence in high-dimensional statistical learning and optimization under L1 constraint, The annals of statistics, 34, 2367-2386, (2006) · Zbl 1106.62022
[10] Guyon, I.; Weston, J.; Barnhill, S.; Vapnik, V., Gene selection for cancer classification using support vector machines, Machine learning, 46, 389-422, (2002) · Zbl 0998.68111
[11] Hale, E.; Yin, W.; Zhang, Y., Fixed-point continuation for L1-minimization: methodology and convergence, SIAM journal on optimization, 19, 1107-1130, (2008) · Zbl 1180.65076
[12] Hall, P.; Marron, J.S.; Neeman, A., Geometric representation of high dimension low sample size data, Journal of the royal statistical society. series B, 67, 427-444, (2005) · Zbl 1069.62097
[13] Hastie, T.; Rosset, S.; Tibshirani, R.; Zhu, J., The entire regularization path for the support vector machine, Journal of machine learning research, 5, 1391-1415, (2004) · Zbl 1222.68213
[14] Hastie, T.; Tibshirani, R.; Friedman, J., The elements of statistical learning, (2001), Springer-Verlag New York
[15] Herbei, R.; Wegkamp, M.H., Classification with reject option, The Canadian journal of statistics, 34, 709-721, (2006) · Zbl 1151.62302
[16] Hong, P.; Liu, S.; Zhou, Q.; Lu, X.; Liu, S.; Wong, H., A boosting approach for motif modeling using CHIP-chip data, Bioinformatics, 21, 2636-2643, (2005)
[17] Iwao, K.; Matoba, R.; Ueno, N.; Ando, A.; Miyoshi, Y.; Matsubara, K.; Noguchi, S.; Kato, K., Molecular classification of primary breast tumors possessing distinct prognostic properties, Human molecular genetics, 11, 199-206, (2002)
[18] Kim, J.; Kim, Y.; Kim, Y., A gradient-based optimization algorithm for lasso, Journal of computational and graphical statistics, 17, 994-1009, (2008)
[19] Koh, K.; Kim, S.J.; Boyd, S., An interior-point method for large-scale L1-regularized logistic regression, Journal of machine learning research, 8, 1519-1555, (2007) · Zbl 1222.62092
[20] Lendgrebe, C.W.; Tax, M.J.; Paclik, P.; Duin, P.W., The interaction between classification and reject performance for distance-based reject-option classifiers, Pattern recognition letters, 27, 908-917, (2006)
[21] Liao, J.G.; Chin, K.V., Logistic regression for disease classification using microarray data: model selection in a large \(p\) and small \(n\) case, Bioinformatics, 23, 1945-1951, (2007)
[22] McLachlan, G.J., Discriminant analysis and statistical pattern recognition, (1992), Wiley New York
[23] Meier, L.; van de Geer, S.; Buhlmann, P., The group lasso for logistic regression, Journal of the royal statistical society. series B, 70, 53-71, (2008) · Zbl 1400.62276
[24] Park, M.Y.; Hastie, T., L1-regularization path algorithm for generalized linear models, Journal of the royal statistical society. series B, 69, 659-677, (2007)
[25] Petricoin, E.F.; Ardekani, A.M.; Hitt, B.A.; Levine, P.J.; Fusaro, V.A.; Steinberg, S.A.; Mills, G.B.; Simone, C.; Fishman, D.A.; Kohn, E.C.; Liotta, L.A., Use of proteomic patterns in serum to identify Ovarian cancer, Lancet, 359, 572-577, (2002)
[26] Rosset, S.; Zhu, J., Piecewise linear regularized solution paths, The annals of statistics, 35, 1012-1030, (2007) · Zbl 1194.62094
[27] Schwarz, G., Estimating the dimension of a model, The annals of statistics, 6, 461-464, (1978) · Zbl 0379.62005
[28] Shen, X.; Tseng, G.C.; Zhang, X.; Wong, W.H., On \(\psi\)-learning, Journal of the American statistical association, 98, 724-734, (2003) · Zbl 1052.62095
[29] Shevade, S.K.; Keerthi, S.S., A simple and efficient algorithm for gene selection using sparse logistic regression, Bioinformatics, 19, 2246-2253, (2003)
[30] Singh, D.; Febbo, P.G.; Ross, K.; Jackson, D.G.; Manola, J.; Ladd, C.; Tamayo, P.; Renshaw, A.A.; D’Amico, A.V.; Richie, J.P.; Lander, E.S.; Loda, M.; Kantoff, P.W.; Golub, T.R.; Sellers, W.R., Gene expression correlates of clinical prostate cancer behavior, Cancer cell, 1, 203-209, (2002)
[31] Terrence, S.; Cristianini, N.; Duffy, N.; Bednarski, D.W.; Schummer, M.; Haussler, D., Support vector machine classification and validation of cancer tissue samples using microarray expression data, Bioinformatics, 16, 906-914, (2000)
[32] Tibshirani, R., Regression shrinkage and selection via the lasso, Journal of the royal statistical society. series B, 58, 267-288, (1996) · Zbl 0850.62538
[33] Tortorella, F., An optimal reject rule for binary classifiers, Lecture notes in computer science, 876, 611-620, (2000) · Zbl 0996.68769
[34] Wang, L.; Zhu, J.; Zou, H., Hybrid huberized support vector machines for microarray classification and gene selection, Bioinformatics, 24, 412-419, (2008)
[35] Wegkamp, M.H., Lasso type classifiers with a reject option, Electronic journal of statistics, 1, 155-168, (2007) · Zbl 1320.62153
[36] Wu, T.T.; Lange, K., Coordinate descent algorithm for lasso penalized regression, The annals of applied statistics, 2, 224-244, (2008) · Zbl 1137.62045
[37] Yukinawa, N.; Oba, S.; Kato, K.; Taniguchi, K.; Iwao-Koizumi, K.; Tamaki, Y.; Noguchi, S.; Ishii, S., A multi-class predictor based on a probabilistic model: application to gene expression profiling-based diagnosis of thyroid tumors, BMC bioinformatics, 7, 1471-2164, (2006)
[38] Zhang, H.H.; Ahn, J.; Lin, X.; Park, C., Gene selection using support vector machines with nonconvex penalty, Bioinformatics, 22, 88-95, (2006)
[39] Zhang, H.H.; Wahba, G.; Lin, Y.; Voelker, M.; Ferris, M.; Klein, R.; Klein, B., Variable selection and model building via likelihood basis pursuit, Journal of the American statistical association, 99, 659-672, (2004) · Zbl 1117.62459
[40] Zhu, J.; Rosset, S.; Hastie, T.; Tibshirani, R., 1-norm support vector machines, ()
[41] Zou, H.; Hastie, T.; Tibshirani, R., On the degrees of freedom of the lasso, The annals of statistics, 35, 2173-2192, (2007) · Zbl 1126.62061
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.