Multi-Modal Semantic Parsing for the Interpretation of Tombstone Inscriptions

What is it about?

This study explores how artificial intelligence can help preserve historical and cultural information found on tombstones. Tombstones often include valuable details—such as names, dates, places, family ties, and professions—but they are difficult to read automatically because of weathering, old fonts, and multiple languages. To address this, we built a system that uses modern vision–language models—AI tools that can understand both images and text—to “read” tombstone photos and turn them into structured data that historians and researchers can easily search and analyze. We also connected our system to online databases of places, occupations, and word meanings, so it can correctly interpret specialized terms. Our approach greatly improves accuracy compared to traditional text-recognition methods and remains reliable even when inscriptions are faded or damaged. This work represents a new step toward the digital preservation of cultural heritage and supports historical, linguistic, and genealogical research on cemeteries worldwide.

Featured Image

Why is it important?

Many tombstones are deteriorating due to weather, time, and neglect, causing the loss of valuable cultural and genealogical information. Our work offers the first AI-based system that can automatically interpret tombstone inscriptions from photos, turning them into structured, searchable data. By combining vision–language models with external knowledge sources, we bridge the gap between image understanding and historical research. This technology not only supports digital heritage preservation but also opens new opportunities for historians, linguists, and the public to explore the past in a scalable, data-driven way.

Perspectives