Single Dataset Order Selection Algorithms

We present three algorithms for estimating the number of independent source components. The algorithms are based on Wax and Kailath's formulation for model selection, that implements information theoretic criterion (ITC) [4]. This ITC formulation, however, assumes the data samples to be i.i.d., which is not always fulfilled by various datasets, e.g., videos, fMRI, etc. The algorithms presented here, thus, down-sample the data to obtain i.i.d. samples [1], [2] or take the sample dependence into account [3], to estimate the order.

  1. Eigenvalues of the down-sampled data based order selection (E-DS) [1]
  2. Joint estimation of the down-sampling depth and order based order selection (E-JDS) [2]
  3. Entropy-rate based order selection by finite memory length model (ER-FM) and autoregressive model (ER-AR) [3]

E-DS

E-DS implements a down-sampling technique to estimate i.i.d. samples from dependent samples. Entropy rate is used to measure the sample dependence and is compared with a stationary white Gaussian sequence that has an upper bound of √ 2πe . The sampling depth is increased until the entropy of the new sampled sequence achieves the upper bound, at which the samples are said to be i.i.d. The down-sampled data is used to estimate the order using the information theoretic criterion (ITC) described in [4].

E-JDS

E-JDS algorithm is an iterative algorithm that jointly estimates the down-sampling depth and order. This algorithm assumes the data samples to be generated by an autoregressive (AR) model and implements a hypothesis test to decide whether the samples are i.i.d. (AR order is zero) or dependent (AR order is positive). The order is estimated using the information theoretic criterion formulation described in [4].

ER-FM and ER-AR

ER-FM and ER-AR algorithms estimate the order for samples with dependence, by using all the samples to estimate the likelihood. The likelihood estimators assume the signal to have finite memory length (ER-FM) or that the signals can be modeled by an AR model (ER-AR).


References:

[1] Y.-O. Li, T. Adali, and V. D. Calhoun, "Estimating the number of independent components for fMRI data," Human Brain Mapping, vol. 28, no. 11, pp. 1251-1266, 2007.
[2] X.-L Li, S. Ma, V. D. Calhoun, and T. Adali, "Order detection for fMRI analysis: Joint estimation of down-sampling depth and order by information theoretic criteria," 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1019-1022, March, 2011.
[3] G.-S. Fu, M. Anderson, and T. Adali, "Likelihood estimators for dependent samples and their application to order detection," IEEE Trans. on Signal Processing, vol. 62, no. 16, pp. 4237-4244, Aug. 2014.
[4] M. Wax, T. Kailath, "Detection of signals by information theoretic criteria," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, no. 2, pp. 387-392, 1985.