Structured negative sampling and LLM-judged relevance for IKEA product search

What is it about?

The Challenge: Understanding Users' Intent Search engines learn by example, and showing them what doesn't match a query is just as important as showing them what does. By using AI to automatically generate and evaluate highly specific, challenging "wrong answers" from the IKEA catalog, we taught our search model to understand the distinctions that matter to real shoppers. This approach successfully bridged the gap between offline model accuracy and real-world shopping behavior. The Solution: Smarter Training at Scale We taught the IKEA product search to be significantly sharper by feeding it harder, smarter "wrong answers." We selected these tricky examples directly from structured product data, focusing on specific product features like size and material. This forced the model to learn fine-grained distinctions instead of easy ones. Furthermore, rather than relying on a small set of hand-labeled examples, we used Large Language Models as a judge, rating how well every product matched every user query. This gave us richer, highly accurate training data at a massive scale. The Real-World Impact This method improved our accuracy in offline testing. However, when we first tested online, the gains didn't immediately translate. We realized that for many popular searches, people often don't click on anything at all—meaning even better rankings couldn't change the outcome of those specific sessions. That crucial insight reshaped our approach. Our latest work turns offline accuracy into real impact by learning how people truly search and behave, adjusting for those zero-click journeys. By aligning our AI-generated training data with real-world user intent, we successfully translated offline accuracy into a measurable, intuitive improvement for the everyday shopper.

Featured Image

Photo by Growtika on Unsplash

Photo by Growtika on Unsplash

Why is it important?

As AI increasingly reshapes the e-commerce landscape, the information retrieval field frequently optimizes for isolated offline benchmarks that do not capture authentic shopping behavior. Our research addresses this critical disconnect by grounding model evaluation in empirical user experiences. This research stands out by effectively aligning sophisticated retrieval models with the practical realities of user behavior. We achieve this through two primary innovations: 1. Catalog-Grounded Training Data: Rather than utilizing generic datasets, we systematically generate precise, highly challenging negative examples derived directly from IKEA product metadata. This compels the retriever to recognize nuanced distinctions among catalog items, closely mirroring the specificity of human intent. 2. Behavior-Driven Evaluation: We anchor our evaluation frameworks in empirical search behavior, accounting for the complex and multifaceted ways consumers navigate product catalogs and make purchasing decisions. Ultimately, this work extends beyond incremental metric improvements on standard benchmarks. It establishes a fundamentally robust retrieval architecture that measurably comprehends and assists the end user.

Perspectives