Independent component analysis based on information bottleneck. (English) Zbl 1436.62226

Ke, Qiao; Zhang, Jiangshe; Srivastava, H. M.; Wei, Wei; Chen, Guang-Sheng

Abstr. Appl. Anal. 2015, Article ID 386201, 8 p. (2015).

Summary: The paper is mainly used to provide the equivalence of two algorithms of independent component analysis (ICA) based on the information bottleneck (IB). In the viewpoint of information theory, we attempt to explain the two classical algorithms of ICA by information bottleneck. Furthermore, via the numerical experiments with the synthetic data, sonic data, and image, ICA is proved to be an edificatory way to solve BSS successfully relying on the information theory. Finally, two realistic numerical experiments are conducted via FastICA in order to illustrate the efficiency and practicality of the algorithm as well as the drawbacks in the process of the recovery images the mixing images.

MSC:

62H25	Factor analysis and principal components; correspondence analysis
62B10	Statistical aspects of information-theoretic topics
62H35	Image analysis in multivariate analysis

Keywords:

principal component analysis; information

References:

[1]	Gokcay, E.; Principe, J. C., Information theoretic clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 2, 158-171, (2002)
[2]	Kraskov, A.; Stögbauer, H.; Andrzejak, R. G., Hierarchical clustering using mutual information, Europhysics Letters, 70, 2, 278-284, (2005)
[3]	Bell, A. J.; Sejnowski, T. J., An information-maximization approach to blind separation and blind deconvolution, Neural Computation, 7, 6, 1129-1159, (1995)
[4]	Hyvärinen, A.; Hurri, J.; Hoyer, P. O., Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, (2009), Springer · Zbl 1178.68622
[5]	Cover, T. M.; Thomas, J. A., Elements of Information Theory, (2012), John Wiley & Sons
[6]	Zhang, H.; Liu, X.; Wang, J., Robust \(H_\infty\) sliding mode control with pole placement for a fluid power electrohydraulic actuator (EHA) system, The International Journal of Advanced Manufacturing Technology, 73, 5-8, 1095-1104, (2014)
[7]	Zhang, H.; Shi, Y.; Wang, J., On energy-to-peak filtering for nonuniformly sampled nonlinear systems: a Markovian jump system approach, IEEE Transactions on Fuzzy Systems, 22, 1, 212-222, (2014)
[8]	Zhang, H.; Wang, J., Combined feedback-feedforward tracking control for networked control systems with probabilistic delays, Journal of the Franklin Institute, 351, 6, 3477-3489, (2014) · Zbl 1290.93046
[9]	Hyvärinen, A.; Oja, E., Independent component analysis: algorithms and applications, Neural Networks, 13, 4-5, 411-430, (2000)
[10]	Zhang, H.; Zhang, X.; Wang, J., Robust gain-scheduling energy-to-peak control of vehicle lateral dynamics stabilisation, Vehicle System Dynamics, 52, 3, 309-340, (2014)
[11]	Wei, W.; Qi, Y., Information potential fields navigation in wireless Ad-Hoc sensor networks, Sensors, 11, 5, 4794-4807, (2011)
[12]	Wei, W.; Shen, P.; Zhang, Y.; Zhang, L., Information fields navigation with piece-wise polynomial approximation for high-performance OFDM in WSNs, Mathematical Problems in Engineering, 2013, (2013) · Zbl 1394.94806
[13]	Shuai, Z.; Zhang, H.; Wang, J., Lateral motion control for four-wheel-independent-drive electric vehicles using optimal torque allocation and dynamic message priority scheduling, Control Engineering Practice, 24, 55-66, (2014)
[14]	Tishby, N.; Pereira, F. C.; Bialek, W., The information Bottleneck method, Proceedings of the 37th annual Allerton Conference on Communication, Control, and Computing
[15]	Rice, J., Mathematical Statistics and Data Analysis, (2006), Cengage Learning