Inverse transports from samples (density estimation)

Here we discuss settings where one is able to sample the distribution \(\nu_\pi\) (or is provided a finite number of samples) and wants to characterize its density \(\pi\). This kind of problems go under the name of density estimation. We then seek a transport map \(S:\mathbb{R}^d \rightarrow \mathbb{R}^d\) that pushes forward the target density \(\pi\) to an amenable reference density \(\rho\). This is achieved solving the following problem:

\[\begin{split}S^\star &= \arg\min_{S \in \mathcal{T}} \mathcal{D}_{\rm KL}\left( S_\sharp \nu_\pi \middle\Vert \nu_\rho \right) = \arg\min_{S \in \mathcal{T}} \mathcal{D}_{\rm KL}\left( \nu_\pi \middle\Vert S^\sharp \nu_\rho \right) \\ &= \arg\min_{S \in \mathcal{T}} \mathbb{E}_\pi \left[ \log \frac{\pi}{S^\sharp \rho} \right] = \arg\min_{S \in \mathcal{T}} \mathbb{E}_\pi \left[ - \log S^\sharp \rho \right]\end{split}\]

where the expectation with respect to \(\pi\) can be approximated because we can sample (or we are given samples of) the distribution \(\nu_\pi\).

For more details on the topic we refer to the available literature [TM4].

Contents

• Inverse transport
  • Variational solution
• Sparse transports for Markov random fields
  • Sparse inverse transports
  • Variational solution
• Sparsity identification algorithm (SING)
  • Non-Gaussian data