A dynamic distribution adaptation approach to improve transfer learning

What is it about?

Supervised learning is commonly used to train machine-learning models. In this method, a model predicts an outcome after learning how the output is related to the input from labeled data. While widely used, this method requires a great deal of time to gather and label the data. The results of the model are also adversely affected by biases in the data. Transfer learning is used to overcome this limitation. With this method, good-quality datasets can be used to build models that work outside the domain the data is taken from. For instance, a model can be trained to recognize handwritten numbers from printed images of numbers. Here, the handwritten numbers are the target domain and the printed images of numbers are the source domain. In this paper, Dynamic Distribution Adaptation (DDA) is proposed as a new method to improve the performance of transfer learning. DDA considers the relative importance of the marginal and conditional data distributions in both domains. This improves knowledge transfer. The authors also present two learning methods based on the DDA. For this, they use tests carried out with datasets used to benchmark domain adaptation algorithms. The findings show that DDA improves the results of transfer learning than existing methods.

Featured Image

Photo by Markus Spiske on Unsplash

Photo by Markus Spiske on Unsplash

Why is it important?

Models trained by transfer learning must learn to recognize bias in the data. They must also account for the differences between the data in the two domains. The accuracy of the models is improved by reducing the data distributions (marginal and conditional) between the source and target domains. Current learning methods give the same importance to both distributions. The DDA on the other hand dynamically learns the importance of each distribution. This improves knowledge transfer between the two domains. KEY TAKEAWAY: DDA reduces the distribution divergence between the domains to improve the results of transfer learning. It can be used to save time and effort when training new models.