×

Correntropy: Implications of nonGaussianity for the moment expansion and deconvolution. (English) Zbl 1217.94084

Summary: The recently introduced correntropy function is an interesting and useful similarity measure between two random variables which has found myriad applications in signal processing. A series expansion for correntropy in terms of higher-order moments of the difference between the two random variables has been used to try to explain its statistical properties for uses such as deconvolution. We examine the existence and form of this expansion, showing that it may be divergent, e.g., when the difference has the Laplace distribution, and give sufficient conditions for its existence for differently characterized sub-Gaussian distributions. The contribution of the higher-order moments can be quite surprising, depending on the size of the Gaussian kernel in the definition of the correntropy. In the blind deconvolution setting we demonstrate that statistical exchangeability explains the existence of sub-optimal minima in the correntropy cost surface and show how the positions of these minima are controlled by the size of the Gaussian kernel.

MSC:

94A17 Measures of information, entropy
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Abramowitz, M.; Stegun, I.: Handbook of mathematical functions, (1965) · Zbl 0171.38503
[2] Benveniste, A.; Goursat, M.; Ruget, G.: Robust identification of a nonminimum phase system: blind adjustment of a linear equalizer in data communications, IEEE transactions on automatic control 25, 385-399 (1980) · Zbl 0439.93055 · doi:10.1109/TAC.1980.1102343
[3] Binmore, K. G.: Mathematical analysis, (1982) · Zbl 0484.26002
[4] Boyd, S.; Vandenberghe, L.: Convex optimization, (2004) · Zbl 1058.90049
[5] Brillinger, D. R.: Time series: data analysis and theory, (1981) · Zbl 0486.62095
[6] Buldygin, V. V.; Kozachenko, Yu.V.: Sub-Gaussian random variables, Ukrainian mathematical journal 32, 483-489 (1980) · Zbl 0479.60012 · doi:10.1007/BF01087176
[7] Dingle, R. B.: Asymptotic expansions: their derivation and interpretation, (1973) · Zbl 0279.41030
[8] Garling, D. J. H.: Inequalities: A journey into linear analysis, (2007) · Zbl 1135.26014
[9] Gunduz, A.; Principe, J. C.: Correntropy as a novel measure for nonlinearity tests, Signal processing 89, 14-23 (2009) · Zbl 1151.94366 · doi:10.1016/j.sigpro.2008.07.005
[10] B.-C. Ihm, D.-J. Park, Y.-H. Kwon, Blind separation for mixtures of sub-Gaussian and super-Gaussian sources, in: Proceedings of the IEEE International Symposium on Circuits and Systems, vol. III, Geneva, May 28–31, 2000, pp. 738–741.
[11] Jeong, K. -H.; Liu, W.; Han, S.; Hasanbelliu, E.; Principe, J. C.: The correntropy MACE filter, Pattern recognition 42, 871-885 (2009) · Zbl 1178.68500 · doi:10.1016/j.patcog.2008.09.023
[12] Joiner, B. L.; Rosenblatt, J. R.: Some properties of the range in samples from tukey’s symmetric lambda distributions, Journal of American statistical association 66, 394-399 (1971) · Zbl 0226.62005 · doi:10.2307/2283943
[13] Kahane, J. P.: Local properties of functions in terms of random Fourier series, Studia Mathematica 19, 1-25 (1960) · Zbl 0096.11402
[14] T.-W. Lee, T.J. Sejnowski, Independent component analysis for mixed sub-Gaussian and super-Gaussian sources, in: Proceedings of the 4th Joint Symposium on Neural Computation, Institute for Neural Computation, UCSD, vol. 7, 1997, pp. 132–140.
[15] Li, R.; Liu, W.; Principe, J. C.: A unifying criterion for instantaneous blind source separation based on correntropy, Signal processing 87, 1872-1881 (2007) · Zbl 1186.94204 · doi:10.1016/j.sigpro.2007.01.022
[16] Liu, B.; Munson, D. C.: Generation of a random sequence having a jointly specified marginal distribution and autocovariance, IEEE transactions on acoustics, speech, and signal processing 30, No. 6, 973-983 (1982) · Zbl 0562.60051 · doi:10.1109/TASSP.1982.1163990
[17] Liu, W.; Pokharel, P. P.; Principe, J. C.: Correntropy: properties and applications in non-Gaussian signal processing, IEEE transactions on signal processing 55, 5286-5298 (2007) · Zbl 1390.94277
[18] Mallat, S.: A wavelet tour of signal processing, (1998) · Zbl 0937.94001
[19] Mallows, C. L.: Linear processes are nearly Gaussian, Journal of applied probability 4, 313-329 (1967) · Zbl 0155.25601 · doi:10.2307/3212026
[20] Neves, A.; Wada, C.; Suyama, R.; Attux, R.; Romano, J. M. T.: An analysis of unsupervised signal processing methods in the context of correlated sources, Independent component analysis and signal separation, lecture notes in computer science 5441 (2009) · Zbl 1301.94022
[21] Papoulis, A.: Probability, random variables, and stochastic processes, (1972) · Zbl 0227.94012
[22] J.A. Palmer, K. Kreutz-Delgado, A general framework for component estimation, in: Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation, Nara, Japa, April 1–4, 2003, pp. 41–46.
[23] Santamaría, I.; Pokharel, P. P.; Principe, J. C.: Generalized correlation function: definition, properties, and application to blind equalization, IEEE transactions on signal processing 54, No. 6, 2187-2197 (2006) · Zbl 1374.94594
[24] Taylor, R. L.; Hu, T. -C.: Sub-Gaussian techniques in proving strong laws of large numbers, The American mathematical monthly 94, 295-299 (1987)
[25] Walden, A. T.: A comparison of stochastic gradient and MED-type deconvolution algorithms, Signal processing 15, 203-211 (1988)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.