×

A hybrid autoencoder network for unsupervised image clustering. (English) Zbl 1461.94005

Summary: Image clustering involves the process of mapping an archive image into a cluster such that the set of clusters has the same information. It is an important field of machine learning and computer vision. While traditional clustering methods, such as \(k\)-means or the agglomerative clustering method, have been widely used for the task of clustering, it is difficult for them to handle image data due to having no predefined distance metrics and high dimensionality. Recently, deep unsupervised feature learning methods, such as the autoencoder (AE), have been employed for image clustering with great success. However, each model has its specialty and advantages for image clustering. Hence, we combine three AE-based models – the convolutional autoencoder (CAE), adversarial autoencoder (AAE), and stacked autoencoder (SAE) – to form a hybrid autoencoder (BAE) model for image clustering. The MNIST and CIFAR-10 datasets are used to test the result of the proposed models and compare the results with others. The results of the clustering criteria indicate that the proposed models outperform others in the numerical experiment.

MSC:

94A08 Image processing (compression, reconstruction, etc.) in information and communication theory
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H35 Image analysis in multivariate analysis
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Fayyad, U.; Piatetsky-Shapiro, G.; Smyth, P.; The KDD process for extracting useful knowledge from volumes of data; Commun. ACM: 1996; Volume 39 ,27-34.
[2] Chang, J.; Wang, L.; Meng, G.; Xiang, S.; Pan, C.; Deep adaptive image clustering; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: ; ,5879-5887.
[3] Guo, X.; Liu, X.; Zhu, E.; Yin, J.; Deep clustering with convolutional autoencoders; Proceedings of the 24th International Conference on Neural Information Processing: ; ,373-382.
[4] Alireza, M.; Shlens, J.; Jaitly, N.; Goodfellow, I.; Frey, B.; Adversarial autoencoders; arXiv: 2015; .
[5] Hinton, G.E.; Salakhutdinov, R.R.; Reducing the Dimensionality of Data with Neural Networks; Science: 2006; Volume 313 ,504-507. · Zbl 1226.68083
[6] Kingma, D.P.; Welling, M.; Auto-encoding variational bayes; arXiv: 2013; .
[7] Rezende, D.J.; Mohamed, S.; Wierstra, D.; Stochastic backpropagation and approximate inference in deep generative models; Int. Conf. Mach. Learn.: 2014; Volume 32 ,1278-1286.
[8] Xie, J.; Girshick, R.; Farhadi, A.; Unsupervised deep embedding for clustering analysis; Proceedings of the International Conference on Machine Learning: ; ,478-487.
[9] Dilokthanakul, N.; Mediano, P.A.M.; Garnelo, M.; Lee, M.C.H.; Salimbeni, H.; Arulkumaran, K.; Shanahan, M.; Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders; arXiv: 2016; .
[10] Liu, H.; Shao, M.; Li, S.; Fu, Y.; Infinite ensemble for image clustering; Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: ; ,1745-1754.
[11] Miguel, N.; McDermott, J.; A hybrid autoencoder and density estimation model for anomaly detection; International Conference on Parallel Problem Solving from Nature: Cham, Switzerland 2016; ,717-726.
[12] Wang, C.; Pan, S.; Long, G.; Zhu, X.; Jiang, J.; Mgae: Marginalized graph autoencoder for graph clustering; Proceedings of the 2017 ACM on Conference on Information and Knowledge Management: ; ,889-898.
[13] Sakurada, M.; Yairi, T.; Anomaly detection using autoencoders with nonlinear dimensionality reduction; Proceedings of the 2nd MLSDA Workshop on Machine Learning for Sensory Data Analysis: ; ,4.
[14] Wang, Y.; Yao, H.; Zhao, S.; Auto-encoder based dimensionality reduction; Neurocomputing: 2016; Volume 184 ,232-242.
[15] Sun, W.; Shao, S.; Zhao, R.; Yan, R.; Zhang, X.; Chen, X.; A sparse auto-encoder-based deep neural network approach for induction motor faults classification; Measurement: 2016; Volume 89 ,171-178.
[16] Bose, T.; Majumdar, A.; Chattopadhyay, T.; Machine Load Estimation Via Stacked Autoencoder Regression; Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): ; ,2126-2130.
[17] Dalal, N.; Triggs, B.; Histograms of oriented gradients for human detection; Computer Vision and Pattern Recognition 2005, Proceedings of the IEEE Computer Society Conference (CVPR 2005), San Diego, CA, USA, 20-25 June 2005: New York, NY, USA 2005; Volume Volume 1 ,886-893.
[18] Lowe, D.G.; Distinctive Image Features from Scale-Invariant Keypoints; Int. J. Comput. Vis.: 2004; Volume 60 ,91-110.
[19] Roy, M.; Bose, S.K.; Kar, B.; Gopalakrishnan, P.K.; Basu, A.; A Stacked Autoencoder Neural Network based Automated Feature Extraction Method for Anomaly detection in On-line Condition Monitoring; Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI): ; ,1501-1507.
[20] Li, W.; Fu, H.; Yu, L.; Gong, P.; Feng, D.; Li, C.; Clinton, N.; Stacked Autoencoder-based deep learning for remote-sensing image classification: a case study of African land-cover mapping; Int. J. Remote. Sens.: 2016; Volume 37 ,5632-5646.
[21] Ian, G.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y.; Generative adversarial nets; Advances in Neural Information Processing Systems 27, Proceedings of the Neural Information Processing Systems Conference (NIPS 2014), Montreal, QC, Canada, 8-13 December 2014: Red Hook, NY, USA 2014; ,2672-2680.
[22] Mukherjee, S.; Asnani, H.; Lin, E.; Kannan, S.; ClusterGAN: Latent Space Clustering in Generative Adversarial Networks; arXiv: 2018; .
[23] Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever, I.; Abbeel, P.; InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets; Advances in Neural Information Processing Systems 29, Proceedings of the 2016 Conference on Neural Information Processing Systems, Barcelona, Spain, 5-10 December 2016: Red Hook, NY, USA 2016; ,2172-2180.
[24] Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.A.; Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion; J. Mach. Learn. Res.: 2010; Volume 11 ,3371-3408. · Zbl 1242.68256
[25] Ralf, S.; Kurths, J.; Daub, C.O.; Weise, J.; Selbig, J.; The mutual information: detecting and evaluating dependencies between variables; Bioinformatics: 2002; Volume 18 ,S231-S240.
[26] Peng, X.; Zhou, J.T.; Zhu, H.; k-meansNet: When k-means Meets Differentiable Programming; arXiv: 2018; .
[27] Bezdek, J.C.; ; Pattern Recognition with Fuzzy Objective Function Algorithms: Berlin, Germany 2013; .
[28] Nie, F.; Zeng, Z.; Tsang, I.W.; Xu, D.; Zhang, C.; Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering; IEEE Trans. Neural Netw.: 2011; Volume 22 ,1796-1808.
[29] Liu, G.; Lin, Z.; Yan, S.; Sun, J.; Yu, Y.; Ma, Y.; Robust recovery of subspace structures by low-rank representation; IEEE Trans. Pattern Anal. Mach. Intell.: 2012; Volume 35 ,171-184.
[30] Lu, C.Y.; Min, H.; Zhao, Z.Q.; Zhu, L.; Huang, D.S.; Yan, S.; Robust and efficient subspace segmentation via least squares regression; Computer Vision—ECCV 2012, Proceedings of the European Conference on Computer Vision, Florence, Italy, 7-13 October 2012: Berlin/Heidelberg, Germany 2012; ,347-360.
[31] You, C.; Li, C.-G.; Robinson, D.P.; Vidal, R.; Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering; Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): ; ,3928-3937.
[32] Cai, D.; Chen, X.; Large scale spectral clustering via landmark-based sparse representation; IEEE Trans. Cybern.: 2014; Volume 45 ,1669-1680.
[33] Cai, D.; He, X.; Han, J.; Locally consistent concept factorization for document clustering; IEEE Trans. Knowl. Data Eng.: 2010; Volume 23 ,902-913.
[34] Zhao, D.; Tang, X.; Cyclizing clusters via zeta function of a graph; Advances in Neural Information Processing Systems, Proceedings of the 23rd Annual Conference on Neural Information Processing Systems, Vancuver, BC, Canada, 7-10 December 2009: Cambridge, MA, USA 2009; ,1953-1960.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.