A Dataset for Sentence Retrieval for Open-Ended Dialogues dialogue retrieval sentence retrieval

What is it about?

We address the task of sentence retrieval for open-ended dialogues. Our task is to retrieve sentences from a document corpus that contain information useful for generating the next turn in a given dialogue. To address this task, we constructed a dataset that includes open-ended dialogues from Reddit, candidate sentences from Wikipedia for each dialogue and human annotations for the sentences. We report the performance of several retrieval baselines, including neural retrieval models, over the dataset. To adapt neural models to the types of dialogues in the dataset, we explored an approach to induce a large-scale weakly supervised training data from Reddit.

Featured Image