What is it about?
We are all familiar with the calculation of Kullback-Leibler (and similar) divergences. We need probability density functions (pdfs). But what can we do if we do not have them or if they do not exist? Especially in high dimensional (10-300) cases? Can we use probability characteristic functions (pcfs) instead? Moreover, if we cannot even store high-dimensional probability density functions, how can we perform calculations with them? We suggest using low-rank tensor data formats and low-rank calculus.
Featured Image
Photo by Agence Olloweb on Unsplash
Why is it important?
The representation as a low-rank element of a high order tensor space allows to reduce the computational complexity and storage cost from exponential in the dimension to almost linear. This will reduce the computing time from years to minutes (for certain class of problems).
Perspectives
The task considered here was the numerical computation of characterising statistics of high-dimensional pdfs, as well as their divergences and distances, assuming that the pdf is discretised on a regular grid in the numerical implementation. Even for moderate dimensions, full storage and computation with such objects quickly becomes infeasible. We have shown that high dimensional pdfs, pcfs and some functions of them can be approximated and represented in a low-rank tensor data format. The use of low-rank tensor techniques helps to reduce computational complexity and storage costs from exponential to linear. Computations in high-dimensional statistics and high-dimensional data analysis will be possible and fast with the results obtained.
Dr. Alexander Litvinenko
Rheinisch Westfalische Technische Hochschule Aachen
Read the Original
This page is a summary of: Computing f‐divergences and distances of high‐dimensional probability density functions, Numerical Linear Algebra with Applications, September 2022, Wiley,
DOI: 10.1002/nla.2467.
You can read the full text:
Contributors
The following have contributed to this page
