Independent component analysis (ICA) by entropy bound minimization (EBM) and entropy rate bound minimization (ERBM)

We present two complex-valued ICA algorithms that account for nonorthogonal separation of the sources. Entropy bound, rather than entropy, is minimized. It is known that in ICA, there is no need to estimate the density or entropy of sources with great precision. In these algorithms, instead of minimizing the entropy, we are minimizing the entropy bound, which can be quickly calculated using numerical methods. A decoupling technique is used for the optimization of separation matrix. Specifically, the separation matrix is optimized row by row, as in an orthogonal separation algorithm. This decoupling technique makes the fast nonorthogonal separation of a large number of sources be possible.

  1. Complex-valued ICA by entropy bound minimization (complex ICA-EBM) [1]
  2. Complex-valued ICA by entropy rate bound minimization (complex ICA-ERBM) [2]

Complex-valued ICA-EBM

This is the complex version of the EBM algorithm. Unlike real-valued ICA where a univariate density is approximated, here a bivariate density model is used. In complex ICA-EBM, weighted linear combinations and elliptical distributions are considered, and they provide a rich array of bivariate distributions for density matching by starting again the four measuring functions used for real-valued EBM.

Complex-valued ICA-ERBM

CERBM is the complex version of the ERBM algorithm. It uses the mutual information rate as the cost function, and estimates the entropy rate by assuming signal is generated by an invertible filter driven by an i.i.d. random process. CERBM exploits all three important types of diversity: non-Gaussianity, sample dependence, and noncircularity.


References:

[1] X.-L. Li and T. Adali, "Complex independent component analysis by entropy bound minimization," IEEE Trans. Circuits and Systems I, vol. 57, no. 7, pp. 1417-1430, July 2010
[2] G.-S. Fu, R. Phlypo, M. Anderson, and T. Adali, "Complex Independent Component Analysis Using Three Types of Diversity: Non-Gaussianity, Nonwhiteness, and Noncircularity," IEEE Trans. Signal Processing, vol. 63, no. 3, pp. 794-805, Feb. 2015