The Generalized Spike Process, Sparsity, and Statistical Independence

Summary

We consider the best sparsijying basis (BSB) and the kurtosis maximizing basis (KMB) of a particularly simple stochastic process called the “generalized spike process”. The BSB is a basis for which a given set of realizations of a stochastic process can be represented most sparsely, whereas the KMB is an approximation to the least statistically-dependent basis (LSDB) for which the data representation has minimal statistical dependence. In each realization, the generalized spike process puts a single spike with amplitude sampled from the standard normal distribution at a random location in an otherwise zero vector of length n.

We prove that both the BSB and the KMB select the standard basis, if we restrict our basis search to all possible orthonormal bases in ℝⁿ. If we extend our basis search to all possible volume-preserving invertible linear transformations, we prove the BSB exists and is again the standard basis, whereas the KMB does not exist. Thus, the KMB is rather sensitive to the orthonormality of the transformations, while the BSB seems insensitive. Our results provide new additional support for the preference of the BSB over the LSDB/KMB for data compression. We include an explicit computation of the BSB for Meyer's discretized ramp process.

1. Introduction

This paper is a sequel to our previous paper [3], where we considered the best sparsijying basis (BSB), and the least statistically-dependent basis (LSDB) for input data assumed to be realizations of a very simple stochastic process called the “spike process.” This process, which we will refer to as the “simple” spike process for convenience, puts a unit impulse (i.e., constant amplitude of 1) at a random location in a zero vector of length n.