Picking out the important variables in complex data, with no labels needed

What is it about?

When you have a huge data set — say, the activity of tens of thousands of genes measured across many tissues and treatments — the hard part is deciding which variables actually matter. Our earlier method, tensor decomposition–based unsupervised feature extraction, does this without any labels or prior knowledge, and it has worked well in practice. But it had no statistical foundation. This paper supplies one. We rebuild Tucker decomposition as a set of Bayesian linear regressions, which lets us assign a proper P-value to every variable. The key choice is what we assume follows a bell curve: earlier Bayesian versions assumed the decomposition itself did, which makes it impossible to spot the outliers we are looking for. We instead make that assumption about the leftover error. Tested on synthetic data, chaotic dynamical systems, and mouse gene expression, the method picks out the right variables — and shows why the older approach worked.

Featured Image

Why is it important?

Most feature selection methods need labels: you must already know which samples are patients and which are controls, or which conditions differ. But in many real situations the labels are unavailable, expensive, unreliable, or simply not the interesting question — you want the data to tell you what the categories are. Unsupervised methods fill that gap, and ours can be applied to almost any data that can be arranged as a table or a multi-way array. Placing it on a Bayesian footing matters for a practical reason. Researchers need to justify their choices to reviewers, and "these variables looked like outliers" is weaker than a P-value derived from an explicit statistical model. Our formulation supplies the latter without changing the results in practice. More broadly, the same procedure worked on gene expression, on a chaotic dynamical system, and on synthetic benchmarks, which suggests it is not tuned to one field.

Perspectives