×

On tuning a mean-field model for semi-supervised classification. (English) Zbl 07565669

Summary: Semi-supervised learning (SSL) has become an interesting research area due to its capacity for learning in scenarios where both labeled and unlabeled data are available. In this work, we focus on the task of transduction – when the objective is to label all data presented to the learner – with a mean-field approximation to the Potts model. Aiming at this particular task we study how classification results depend on \(\beta\) and find that the optimal phase depends highly on the amount of labeled data available. In the same study, we also observe that more stable classifications regarding small fluctuations in \(\beta\) are related to configurations of high probability and propose a tuning approach based on such observation. This method relies on a novel parameter \(\gamma\) and we then evaluate two different values of the said quantity in comparison with classical methods in the field. This evaluation is conducted by changing the amount of labeled data available and the number of nearest neighbors in the similarity graph. Empirical results show that the tuning method is effective and allows NMF to outperform other approaches in datasets with fewer classes. In addition, one of the chosen values for \(\gamma\) also leads to results that are more resilient to changes in the number of neighbors, which might be of interest to practitioners in the field of SSL.

MSC:

82-XX Statistical mechanics, structure of matter
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Chapelle, O.; Scholkopf, B.; Zien, A., Semi-Supervised Learning (2006), Cambridge, MA: MIT Press, Cambridge, MA
[2] Van Engelen, J. E.; Hoos, H. H., A survey on semi-supervised learning, Mach. Learn., 109, 373-440 (2020) · Zbl 1441.68215 · doi:10.1007/s10994-019-05855-6
[3] Alpaydin, E., Introduction to Machine Learning (2004), Cambridge, MA: MIT Press, Cambridge, MA
[4] Mohri, M.; Rostamizadeh, A.; Talwalkar, A., Foundations of Machine Learning (2018), Cambridge, MA: MIT Press, Cambridge, MA · Zbl 1407.68007
[5] Chong, Y.; Ding, Y.; Yan, Q.; Pan, S., Graph-based semi-supervised learning: a review, Neurocomputing, 408, 216-230 (2020) · doi:10.1016/j.neucom.2019.12.130
[6] Breve, F., Interactive image segmentation using label propagation through complex networks, Expert Syst. Appl., 123, 18-33 (2019) · doi:10.1016/j.eswa.2019.01.031
[7] Zhang, P.; Zhuo, T.; Zhang, Y.; Tao, D.; Cheng, J., Online tracking based on efficient transductive learning with sample matching costs, Neurocomputing, 175, 166-176 (2016) · doi:10.1016/j.neucom.2015.10.046
[8] Liu, J.; Zhong, Q.; Yuan, Y.; Su, H.; Du, B., Semitext: scene text detection with semi-supervised learning, Neurocomputing, 407, 343-353 (2020) · doi:10.1016/j.neucom.2020.05.059
[9] Zhang, Y.; Wu, Z.; Peng, H.; Lin, S., A transductive approach for video object segmentation, 6949-6958 (2020)
[10] Wu, F. Y., The Potts model, Rev. Mod. Phys., 54, 235 (1982) · doi:10.1103/revmodphys.54.235
[11] Ott, T.; Kern, A.; Schuffenhauer, A.; Popov, M.; Acklin, P.; Jacoby, E.; Stoop, R., Sequential superparamagnetic clustering for unbiased classification of high-dimensional chemical data, J. Chem. Inf. Comput. Sci., 44, 1358-1364 (2004) · doi:10.1021/ci049905c
[12] Wang, F.; Wang, S.; Zhang, C.; Winther, O., Semi-supervised mean fields, Proc. 11th Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 596-603 (2007)
[13] Li, J.; Wang, F., Semi-supervised learning via mean field methods, Neurocomputing, 177, 385-393 (2016) · doi:10.1016/j.neucom.2015.11.042
[14] Karasuyama, M.; Mamitsuka, H., Adaptive edge weighting for graph-based learning algorithms, Mach. Learn., 106, 307-335 (2017) · Zbl 1453.68152 · doi:10.1007/s10994-016-5607-3
[15] Ma, J.; Xiao, B.; Deng, C., Graph based semi-supervised classification with probabilistic nearest neighbors, Pattern Recognit. Lett., 133, 94-101 (2020) · doi:10.1016/j.patrec.2020.01.021
[16] Zhu, X.; Ghahramani, Z., Learning from labeled and unlabeled data with label propagation (2002)
[17] Zhu, X.; Lafferty, J., Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning, 1052-1059 (2005)
[18] Zhou, D.; Bousquet, O.; Lal, T. N.; Weston, J.; Schölkopf, B., Learning with local and global consistency, 321-328 (2004)
[19] Jebara, T.; Wang, J.; Chang, S-F, Graph construction and b-matching for semi-supervised learning, 441-448 (2009)
[20] de Sousa, C. A R.; Rezende, S. O.; Batista, G. E A. P A., Influence of graph construction on semi-supervised learning, 160-175 (2013), Berlin: Springer, Berlin
[21] Subramanya, A.; Talukdar, P. P., Graph-based semi-supervised learning, Synthesis Lect. Artificial Intell. Mach. Learn., 8, 1-125 (2014) · Zbl 1296.68002 · doi:10.2200/s00590ed1v01y201408aim029
[22] Tibély, G.; Kertész, J., On the equivalence of the label propagation method of community detection and a Potts model approach, Physica A, 387, 4982-4984 (2008) · doi:10.1016/j.physa.2008.04.024
[23] Blatt, M.; Wiseman, S.; Domany, E., Superparamagnetic clustering of data, Phys. Rev. Lett., 76, 3251 (1996) · doi:10.1103/physrevlett.76.3251
[24] Getz, G.; Shental, N.; Domany, E., Semi-supervised learning-a statistical physics approach (2005), Citeseer
[25] Shi, C.; Liu, Y.; Zhang, P., Weighted community detection and data clustering using message passing, J. Stat. Mech. (2018) · Zbl 1459.91149 · doi:10.1088/1742-5468/aaa8f5
[26] Opper, M.; Saad, D., Advanced Mean Field Methods: Theory and Practice (2001), Cambridge, MA: MIT Press, Cambridge, MA · Zbl 0994.68172
[27] Rasmussen, C. E,; Radford, N. M,; Hinton, G. E,; Van Camp, D,; Revow, M,; Ghahramani, Z,; Kustra, R.; Tibshirani, R., Delve: data for evaluating learning in valid experiments (1995)
[28] Dua, D.; Graff, C., UCI machine learning repository (2017)
[29] Hull, J. J., A database for handwritten text recognition research, IEEE Trans. Pattern Anal. Machine Intell., 16, 550-554 (1994) · doi:10.1109/34.291440
[30] Alcalá-Fdez, J.; Fernández, A.; Luengo, J.; Derrac, J.; García, S.; Sánchez, L.; Herrera, F., Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, J. Multiple-Valued Logic Soft Comput., 17, 255 (2011)
[31] Xuan Vinh, N.; Epps, J.; Bailey, J., Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance, J. Mach. Learn. Res., 11, 2837-2854 (2010) · Zbl 1242.62062
[32] Ritchie, D. M.; Kernighan, B. W.; Lesk, M. E., The C Programming Language (1988), Englewood Cliffs, NJ: Prentice Hall, Englewood Cliffs, NJ · Zbl 0697.68008
[33] Behnel, S.; Bradshaw, R.; Citro, C.; Dalcin, L.; Seljebotn, D. S.; Smith, K., Cython: the best of both worlds, Comput. Sci. Eng., 13, 31-39 (2010) · doi:10.1109/MCSE.2010.118
[34] Van Rossum, G.; Drake, F. L., Python 3 Reference Manual (2009), Scotts Valley, CA: CreateSpace, Scotts Valley, CA
[35] Dagum, L.; Menon, R., OPENMP: an industry standard API for shared-memory programming, IEEE Comput. Sci. Eng., 5, 46-55 (1998) · doi:10.1109/99.660313
[36] Virtanen, P., Scipy 1.0: fundamental algorithms for scientific computing in Python, Nat. Methods, 17, 261-272 (2020) · doi:10.1038/s41592-019-0686-2
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.