What is it about?

Answering scientific questions using large language models (LLMs) is a growing area of research. In our work, we explored how well these models can translate everyday scientific questions into SPARQL, a language used to query knowledge graphs (structured databases of information). We tested several modern models using two well-known benchmarks, SciQA and DBLP-QuAD. By combining techniques like fine-tuning and smart prompt design (known as few-shot prompting), we achieved strong results. However, we also found that the current benchmarks may be too easy for today’s advanced models. This suggests a need for more difficult tests to truly measure progress. We also identified common mistakes the models make and discussed how lessons from one domain can help improve performance in others.

Featured Image

Why is it important?

Scientific knowledge is growing rapidly, and finding accurate answers quickly is crucial for researchers, students, and policymakers. Large language models can help by turning natural questions into precise database queries, saving time and improving access to complex information. Our study shows how to boost the performance of these models and highlights the need for better tools to test them. This work helps advance the development of AI systems that can support real scientific discovery.

Perspectives

This study highlights the effectiveness of combined fine-tuning and prompting strategies for semantic parsing in scientific question answering. However, the limited complexity of current benchmarks restricts the assessment of true model capabilities. Future research should focus on developing more challenging datasets, improving domain adaptation, and integrating structured knowledge to enhance accuracy and interpretability in natural language to SPARQL translation.

Antonello Meloni
Universita degli Studi di Cagliari

Read the Original

This page is a summary of: Exploring Large Language Models for Scientific Question Answering via Natural Language to SPARQL Translation, ACM Transactions on Intelligent Systems and Technology, August 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3757923.
You can read the full text:

Read

Contributors

The following have contributed to this page