What is it about?
Modern AI assistants often retrieve real documents before answering a question. This approach, called Retrieval-Augmented Generation (RAG), is meant to stop the AI from making things up. This research study analyzed 500 such AI-generated answers and found that the AI still invents numbers such as drive times, percentages, and quantities that do not appear in the documents it read. Out of all the faulty answers, 38.2% contained fabricated numbers. The study then tested whether popular quality-checking methods can catch these errors, including the use of a more advanced AI model as a fact-checker. The advanced AI checker caught only 25.7% of the fabricated numbers, even when it was explicitly instructed to verify them. In contrast, a very simple check, which verifies whether each number in the answer literally appears in the source documents, caught all of them (100% recall).
Featured Image
Photo by Growtika on Unsplash
Why is it important?
Fabricated numbers in AI-generated answers are especially dangerous in domains such as finance and medicine, where a wrong dose or a wrong percentage can have serious consequences. The findings of this research study reveal a counterintuitive weakness: current language models cannot reliably verify numerical accuracy, not even when they are used as dedicated fact-checkers, because of how they process numbers internally. The fix, however, does not require a more advanced AI. A basic number-matching check against the source documents outperformed every sophisticated method tested in this study. The practical message for anyone deploying AI systems is simple: never let a language model be the sole judge of numerical accuracy, and always add a direct number-verification step. This work provides the first systematic measurement of how common numerical fabrication is in retrieval-based AI systems, and it shows that the bar for catching these errors is much lower than expected.
Perspectives
I carried out this work as an independent researcher, because the question felt urgent and largely unexamined. It is often assumed that retrieval solves hallucination, yet every benchmark I studied explicitly excluded numerical errors from its scope. What surprised me the most was not that simple string matching worked, but how badly a state-of-the-art AI judge failed at a task that a regular expression could do perfectly. I hope this study encourages the field to test simple baselines before reaching for larger models, and encourages practitioners to add basic numerical checks to systems where accuracy truly matters.
Sera Singha Roy
Read the Original
This page is a summary of: Numerical Hallucinations in Retrieval-Augmented Generation: Detection and Analysis, July 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3805712.3809882.
You can read the full text:
Resources
Contributors
The following have contributed to this page
