The Choice of Textual Knowledge Base in Automated Claim Checking

What is it about?

In automated claim checking, we want to determine whether a statement is factually true or false. To achieve this, we build natural language processing systems which can automatically check whether a statement is supported or refuted by trustworthy facts in a textual knowledge base (for example Wikipedia). Most previous work optimized systems given both a dataset of statements and existing facts in a textual knowledge base. We take the opposite approach - taking the system as given, we explore the choice of the knowledge base.

Featured Image

Photo by Markus Spiske on Unsplash

Photo by Markus Spiske on Unsplash

Why is it important?

Automated claim checking is one approach to tackle mis- and disinformation in the digital age. To achieve this, we need reliable and robust systems. However, most systems only work in the domain they have been trained on, for example a system trained on Wikipedia usually performs quite poorly in checking scientific claims. As a remedy, there have been many systems proposed in many different domains. In this work, we investigate if we can take a system as given, but build knowledge bases which are sufficient to automatically check claims from other domains. We find that this is the case and that claim checking systems can be transferred to new domains if we have access to a knowledge base from that new domain. Second, we do not find a universally best knowledge base, and combining multiple knowledge bases does not tend to improve performance beyond using the closest-domain knowledge base.

Perspectives