Questions and answers from the Holy Qur’an to help design automated-answering applications

What is it about?

AyaTEC is a collection of 207 Arabic questions and their 1,762 answers; each answer is represented as a sequence of verses from the Holy Qur'an, and was annotated by three Qur'an scholars. The questions cover 11 topic categories of the Holy Qur'an that satisfy the information needs of both curious and skeptical users. Performance evaluation of Arabic question answering (QA) research on the Holy Qur’an was merely an ad-hoc effort that relied mainly on direct manual judgement of answers returned by the QA system in response to different sets of questions, in the absence of having a full set of possible answers to those questions. As such, AyaTEC was designed to fill this gap. This is called a "test collection".

Featured Image

Why is it important?

AyaTEC is the first fully-reusable test collection for Arabic question answering on the Holy Qur’an, where all the qur’anic verses that directly answer the questions were exhaustively extracted and annotated. To facilitate the use of AyaTEC in evaluating Arabic QA systems on the Holy Qur’an, we propose several evaluation measures to support the different types of questions and the nature of the answers being qur'anic verses. To ensure a fair comparison of systems based on the proposed evaluation measures, partial matching was integrated in these measures. We make AyaTEC publicly available to the research community.