How Machine Learning Is Transforming Database Query Optimization: A Systematic Review

What is it about?

Relational database management systems rely on query optimizers to choose the fastest way to execute SQL queries. While traditional optimizers rely on hand-crafted cost formulas and rigid statistical heuristics developed decades ago, modern systems are increasingly integrating machine learning to improve performance. This paper provides a systematic, PRISMA-guided review examining recent breakthroughs in learned query optimization. We categorize and evaluate how machine learning intervenes across five core stages of the classical query processing pipeline: 1) Component-level estimation: Replacing heuristic models for cardinality and cost estimation with neural architectures. 2) Learned join enumeration: Using reinforcement learning to search combinatorial plan spaces. 3) Hint steering: Guiding existing database optimizers safely using lightweight learned hints (e.g., Bao, AutoSteer). 4) End-to-end learned optimizers: Fully learned systems that replace the legacy query engine (e.g., Neo, Balsa). 5) Emerging trends: Recent paradigms including Large Language Models (LLMs) and retrieval-augmented reasoning for database tuning.

Featured Image

Photo by Kevin Ku on Unsplash

Photo by Kevin Ku on Unsplash

Why is it important?

Despite promising benchmarks, learned query optimizers face critical roadblocks before they can be safely trusted in production database environments. Machine learning models often suffer from high training costs, out-of-distribution drift when data updates, and unpredictable tail-latency degradation when a model mispredicts. This review bridges the gap between academic research prototypes and real-world database deployment. By identifying persistent bottlenecks our survey provides a concrete roadmap for researchers and database architects working toward robust, hybrid learned data systems.

Perspectives