×

Retrieval of brain tumors with region-specific bag-of-visual-words representations in contrast-enhanced MRI images. (English) Zbl 1254.92043

Summary: A content-based image retrieval (CBIR) system is proposed for the retrieval of T1-weighted contrast-enhanced MRI (CE-MRI) images of brain tumors. In this CBIR system, spatial information in the bag-of-visual-words model and domain knowledge on the brain tumor images are considered for the representation of brain tumor images. A similarity metric is learned through a distance metric learning algorithm to reduce the gap between the visual features and the semantic concepts in an image. The learned similarity metric is then used to measure the similarity between two images and then retrieve the most similar images in the data set when a query image is submitted to the CBIR system. The retrieval performance of the proposed method is evaluated on a brain CE-MRI data set with three types of brain tumors (i.e., meningioma, glioma, and pituitary tumor). The experimental results demonstrate that the mean average precision values of the proposed method range from 90.4% to 91.5% for different views (transverse, coronal, and sagittal) with an average value of 91.0%.

MSC:

92C50 Medical applications (general)
92C55 Biomedical imaging and signal processing
92C20 Neural biology
68T05 Learning and adaptive systems in artificial intelligence

Software:

LMNN; STL-10 dataset
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] H. Müller, N. Michoux, D. Bandon, and A. Geissbuhler, “A review of content-based image retrieval systems in medical applications-clinical benefits and future directions,” International Journal of Medical Informatics, vol. 73, no. 1, pp. 1-23, 2004. · doi:10.1016/j.ijmedinf.2003.11.024
[2] M. M. Rahman, B. C. Desai, and P. Bhattacharya, “Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion,” Computerized Medical Imaging and Graphics, vol. 32, no. 2, pp. 95-108, 2008. · Zbl 05328001 · doi:10.1016/j.compmedimag.2007.10.001
[3] D. K. Iakovidis, N. Pelekis, E. E. Kotsifakos, I. Kopanakis, H. Karanikas, and Y. Theodoridis, “A pattern similarity scheme for medical image retrieval,” IEEE Transactions on Information Technology in Biomedicine, vol. 13, no. 4, pp. 442-450, 2009. · Zbl 05740722 · doi:10.1109/TITB.2008.923144
[4] H. Greenspan and A. T. Pinhas, “Medical image categorization and retrieval for PACS using the GMM-KL framework,” IEEE Transactions on Information Technology in Biomedicine, vol. 11, no. 2, pp. 190-202, 2007. · Zbl 05456289 · doi:10.1109/TITB.2006.874191
[5] J. E. E. de Oliveira, A. M. C. Machado, G. C. Chavez, A. P. B. Lopes, T. M. Deserno, and A. D. A. Araújo, “MammoSys: a content-based image retrieval system using breast density patterns,” Computer Methods and Programs in Biomedicine, vol. 99, no. 3, pp. 289-297, 2010. · doi:10.1016/j.cmpb.2010.01.005
[6] I. El-Naqa, Y. Yang, N. P. Galatsanos, R. M. Nishikawa, and M. N. Wernick, “A similarity learning approach to content-based image retrieval: application to digital mammography,” IEEE Transactions on Medical Imaging, vol. 23, no. 10, pp. 1233-1244, 2004. · doi:10.1109/TMI.2004.834601
[7] Y. L. Huang, S. J. Kuo, C. S. Chang, Y. K. Liu, W. K. Moon, and D. R. Chen, “Image retrieval with principal component analysis for breast cancer diagnosis on various ultrasonic systems,” Ultrasound in Obstetrics and Gynecology, vol. 26, no. 5, pp. 558-566, 2005. · doi:10.1002/uog.1951
[8] J. G. Dy, C. E. Brodley, A. Kak, L. S. Broderick, and A. M. Aisen, “Unsupervised feature selection applied to content-based retrieval of lung images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 3, pp. 373-378, 2003. · Zbl 05110852 · doi:10.1109/TPAMI.2003.1182100
[9] L. M. DeAngelis, “Brain tumors,” The New England Journal of Medicine, vol. 344, no. 2, pp. 114-123, 2001. · doi:10.1056/NEJM200101113440207
[10] K. Q. Weinberger and L. K. Saul, “Distance metric learning for large margin nearest neighbor classification,” Journal of Machine Learning Research, vol. 10, pp. 207-244, 2009. · Zbl 1235.68204
[11] M. Sugiyama, “Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis,” Journal of Machine Learning Research, vol. 8, pp. 1027-1061, 2007. · Zbl 1222.68312
[12] B. Alipanahi, M. Biggs, and A. Ghodsi, “Distance metric learning VS. fisher discriminant analysis,” in Proceedings of the 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, vol. 48, pp. 598-603, Chicago, Ill, USA, July 2008.
[13] J. Levesque, Efficient Query Support for a Brain Tumour Database, University of Alberta, 2007.
[14] H. K. Huang, J. F. Nielsen, M. D. Nelson, and L. Liu, “Image-matching as a medical diagnostic support tool (DST) for brain diseases in children,” Computerized Medical Imaging and Graphics, vol. 29, no. 2-3, pp. 195-202, 2005. · doi:10.1016/j.compmedimag.2004.09.008
[15] S. Dube, S. El-Saden, T. F. Cloughesy, and U. Sinha, “Content based image retrieval for MR image studies of brain tumors,” in Proceedings of the 28th IEEE Annual International Conference on Engineering in Medicine and Biology Society, vol. 1, pp. 3337-3340, New York, NY, USA, September 2006.
[16] G. P. Robinson, H. D. Tagare, J. S. Duncan, and C. C. Jaffe, “Medical image collection indexing: shape-based retrieval using KD-trees,” Computerized Medical Imaging and Graphics, vol. 20, no. 4, pp. 209-217, 1996. · Zbl 05327443 · doi:10.1016/S0895-6111(96)00014-6
[17] E. G. M. Petrakis and C. Faloutsos, “Similarity searching in medical image databases,” IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 3, pp. 435-447, 1997.
[18] Y. Zhang, Z. Dong, L. Wu, and S. Wang, “A hybrid method for MRI brain image classification,” Expert Systems with Applications, vol. 38, no. 8, pp. 10049-10053, 2011. · doi:10.1016/j.eswa.2011.02.012
[19] M. R. Nazari and E. Fatemizadeh, “A CBIR system for human brain magnetic resonance image indexing,” International Journal of Computer Applications, vol. 7, no. 14, pp. 33-37, 2010.
[20] S. Lahmiri and M. Boukadoum, “Brain MRI classification using an ensemble system and LH and HL wavelet sub-bands features,” in Proceedings of the IEEE 3rd International Workshop on Computational Intelligence in Medical Imaging, pp. 1-7, Paris, France, April 2011.
[21] S. Klöppel, A. Abdulkadir, S. Hadjidemetriou et al., “A comparison of different automated methods for the detection of white matter lesions in MRI data,” NeuroImage, vol. 57, no. 2, pp. 416-422, 2011. · doi:10.1016/j.neuroimage.2011.04.053
[22] S. Chaplot, L. M. Patnaik, and N. R. Jagannathan, “Classification of magnetic resonance brain images using wavelets as input to support vector machine and neural network,” Biomedical Signal Processing and Control, vol. 1, no. 1, pp. 86-92, 2006. · doi:10.1016/j.bspc.2006.05.002
[23] S. B. Antel, D. L. Collins, N. Bernasconi et al., “Automated detection of focal cortical dysplasia lesions using computational models of their MRI characteristics and texture analysis,” NeuroImage, vol. 19, no. 4, pp. 1748-1759, 2003. · doi:10.1016/S1053-8119(03)00226-X
[24] B. S. Manjunath and W. Y. Ma, “Texture features for browsing and retrieval of image data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837-842, 1996.
[25] F. F. Li and P. Perona, “A bayesian hierarchical model for learning natural scene categories,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’05), vol. 2, pp. 524-531, June 2005.
[26] H. Jégou, M. Douze, and C. Schmid, “Improving bag-of-features for large scale image search,” International Journal of Computer Vision, vol. 87, no. 3, pp. 316-336, 2010. · Zbl 06023081 · doi:10.1007/s11263-009-0285-2
[27] A. Bosch, X. Muñoz, A. Oliver, and J. Martí, “Modeling and classifying breast tissue density in mammograms,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’06), vol. 2, pp. 1552-1558, June 2006. · doi:10.1109/CVPR.2006.188
[28] S. Savarese, J. Winn, and A. Criminisi, “Discriminative object class models of appearance and shape by correlatons,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’06), vol. 2, pp. 2033-2040, June 2006. · doi:10.1109/CVPR.2006.102
[29] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: spatial pyramid matching for recognizing natural scene categories,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’06), pp. 2169-2178, June 2006. · doi:10.1109/CVPR.2006.68
[30] U. Avni, H. Greenspan, E. Konen, M. Sharon, and J. Goldberger, “X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words,” IEEE Transactions on Medical Imaging, vol. 30, no. 3, pp. 733-746, 2011. · doi:10.1109/TMI.2010.2095026
[31] B. André, T. Vercauteren, A. Perchant, A. M. Buchner, M. B. Wallace, and N. Ayache, “Introducing space and time in local feature-based endomicroscopic image retrieval,” Medical Content-Based Retrieval for Clinical Decision Support, vol. 5853, pp. 18-30, 2010. · Zbl 05672514 · doi:10.1007/978-3-642-11769-5_2
[32] L. G. Nyul, J. K. Udupa, and X. Zhang, “New variants of a method of MRI scale standardization,” IEEE Transactions on Medical Imaging, vol. 19, no. 2, pp. 143-150, 2000. · doi:10.1109/42.836373
[33] D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV ’99), vol. 2, pp. 1150-1157, Corfu, Greece, September 1999.
[34] M. Varma and A. Zisserman, “A statistical approach to material classification using image patch exemplars,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 2032-2047, 2009. · doi:10.1109/TPAMI.2008.182
[35] K. Mikolajczyk and C. Schmid, “An affine invariant interest point detector,” in Proceedings of the 7th European Conference on Computer Vision (ECCV ’02), vol. 2350 of Lecture Notes in Computer Science, pp. 128-142, 2002. · Zbl 1034.68642
[36] J. Zhang, M. Marszałek, S. Lazebnik, and C. Schmid, “Local features and kernels for classification of texture and object categories: a comprehensive study,” International Journal of Computer Vision, vol. 73, no. 2, pp. 213-238, 2007. · Zbl 05146333 · doi:10.1007/s11263-006-9794-4
[37] A. Coates, H. Lee, and A. Y. Ng, “An analysis of single-layer networks in unsupervised feature learning,” in Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, pp. 1-9, Ft. Lauderdale, Fla, USA, April 2011.
[38] A. Hyvärinen and E. Oja, “Independent component analysis: algorithms and applications,” Neural Networks, vol. 13, no. 4-5, pp. 411-430, 2000. · doi:10.1016/S0893-6080(00)00026-5
[39] A. Frome, Y. Singer, and J. Malik, “Image retrieval and classification using local distance functions,” in Proceedings of the Conference on Neural Information Processing Systems, vol. 19, pp. 417-424, 2006.
[40] H. Chang and D. Y. Yeung, “Kernel-based distance metric learning for content-based image retrieval,” Image and Vision Computing, vol. 25, no. 5, pp. 695-703, 2007. · doi:10.1016/j.imavis.2006.05.013
[41] C. Liu, J. Yuen, and A. Torralba, “SIFT flow: dense correspondence across scenes and its applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 978-994, 2011. · doi:10.1109/TPAMI.2010.147
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.