Collaborative automated feature engineering

What is it about?

In this work, through scrutiny of the prior automated feature engineering (AutoFE) methods, we characterize several research challenges that remained in this regime, concerning system-wide efficiency, efficacy, and practicality toward production. We then propose Catch, a full-fledged new AutoFE framework that comprehensively addresses the aforementioned challenges. The core to Catch composes a hierarchical-policy reinforcement learning scheme that manifests a collaborative feature engineering exploration and exploitation grounded on the granularity of the whole feature set. At a higher level of the hierarchy, a decision-making module controls the post-processing of the attained feature engineering transformation. We extensively experiment with Catch on 26 academic standardized tabular datasets and 9 industrialized real-world datasets. Measured by numerous metrics and analyses, Catch establishes a new state-of-the-art, from perspectives performance, latency as well as its practicality towards production.

Featured Image

Photo by ThisisEngineering RAEng on Unsplash

Photo by ThisisEngineering RAEng on Unsplash

Why is it important?

Feature engineering often plays a crucial role in building mining systems for tabular data, which traditionally requires experienced human experts to perform.

Perspectives