SpectralNet swMATH ID: 26162 Software Authors: Uri Shaham, Kelly Stanton, Henry Li, Boaz Nadler, Ronen Basri, Yuval Kluger Description: SpectralNet: Spectral Clustering using Deep Neural Networks. Spectral clustering is a leading and popular technique in unsupervised data analysis. Two of its major limitations are scalability and generalization of the spectral embedding (i.e., out-of-sample-extension). In this paper we introduce a deep learning approach to spectral clustering that overcomes the above shortcomings. Our network, which we call SpectralNet, learns a map that embeds input data points into the eigenspace of their associated graph Laplacian matrix and subsequently clusters them. We train SpectralNet using a procedure that involves constrained stochastic optimization. Stochastic optimization allows it to scale to large datasets, while the constraints, which are implemented using a special-purpose output layer, allow us to keep the network output orthogonal. Moreover, the map learned by SpectralNet naturally generalizes the spectral embedding to unseen data points. To further improve the quality of the clustering, we replace the standard pairwise Gaussian affinities with affinities leaned from unlabeled data using a Siamese network. Additional improvement can be achieved by applying the network to code representations produced, e.g., by standard autoencoders. Our end-to-end learning procedure is fully unsupervised. In addition, we apply VC dimension theory to derive a lower bound on the size of SpectralNet. State-of-the-art clustering results are reported on the Reuters dataset. Our implementation is publicly available at https://github.com/KlugerLab/SpectralNet Homepage: https://arxiv.org/abs/1801.01587 Source Code: https://github.com/KlugerLab/SpectralNet Keywords: arXiv_publication; Machine Learning; arXiv_stat.ML; arXiv_cs.LG; Spectral Clustering; Deep Neural Networks Related Software: t-SNE; RCV1; SyncSpecCnn; SinGAN; LaplacianSmoothing-GradientDescent; GitHub; FaceNet; Algorithm 971; LAMG; SparseMatrix; SNAP Cited in: 5 Publications Standard Articles 1 Publication describing the Software Year SpectralNet: Spectral Clustering using Deep Neural Networks Uri Shaham, Kelly Stanton, Henry Li, Boaz Nadler, Ronen Basri, Yuval Kluger 2018 all top 5 Cited by 18 Authors 2 Kluger, Yuval 1 Azencot, Omri 1 Basu, Sugato 1 Cohen, Ido 1 Coifman, Ronald Raphaël 1 Davidson, Ian 1 Gilboa, Guy 1 Hu, Xiaozhe 1 Jaffe, Ariel 1 Landa, Boris 1 Lifshits, Pavel 1 Lin, Junyuan 1 Linderman, George C. 1 Mishne, Gal 1 Steinerberger, Stefan 1 Zhan, Tianyang 1 Zhang, Hongjing 1 Zikatanov, Ludmil T. Cited in 5 Serials 1 Journal of Applied Probability 1 SIAM Journal on Scientific Computing 1 Data Mining and Knowledge Discovery 1 SIAM Journal on Imaging Sciences 1 SIAM Journal on Mathematics of Data Science all top 5 Cited in 9 Fields 3 Statistics (62-XX) 2 Numerical analysis (65-XX) 1 Combinatorics (05-XX) 1 Partial differential equations (35-XX) 1 Calculus of variations and optimal control; optimization (49-XX) 1 Probability theory and stochastic processes (60-XX) 1 Computer science (68-XX) 1 Biology and other natural sciences (92-XX) 1 Information and communication theory, circuits (94-XX) Citations by Year