Towards Open-domain Dialogue Response Generation for Non-English languages

What is it about?

This work pioneers few-shot cross-lingual transfer learning (FS-XLT) and multitask learning (MTL) in the context of open-domain dialogue generation for non-English languages with limited dialogue data. In our preliminary experiments, the problem of catastrophic forgetting is observed in both FS-XLT and MTL. To mitigate the issue, we propose a simple yet effective prompt learning approach that can preserve the multilinguality of multilingual pre-trained language model (mPLM) in FS-XLT and MTL by bridging the gap between pre-training and fine-tuning with Fixed-prompt LM Tuning and our hand-crafted prompts. We experimented with 3 Germanic languages (i.e. Danish, German and Norwegian) and 3 Romance languages (i.e. Spanish, Italian and Portuguese) to demonstrate the effectiveness of our approach.

Featured Image

Photo by Possessed Photography on Unsplash

Photo by Possessed Photography on Unsplash

Why is it important?

Dialogue systems for non-English languages have long been under-explored. There are more than 7,000 languages in the world, but even the most advanced dialogue systems like ChatGPT support just around 100 languages. However, all the human beings, no matter what languages they speak, are entitled the rights for accessing advanced technologies from the filed of artificial intelligence, including dialogue systems. In light of this, it is critical to explore effective approaches to building inclusive dialogue systems for non-English languages. Considering the data scarcity issue, we explore few-shot cross-lingual transfer learning and multi-task learning for open-domain dialogue generation with only a limited number of training data in non-English languages, which sheds light on the study of non-English dialogue systems.