Towards Continual Visual Anomaly Detection in the Medical Domain

What is it about?

In this study, we focus on teaching deep learning systems how to automatically detect unusual or abnormal patterns in medical images—such as X-rays, CT scans, or MRI scans, that might indicate disease. This task is known as Visual Anomaly Detection (VAD).

Featured Image

Photo by Umanoide on Unsplash

Photo by Umanoide on Unsplash

Why is it important?

Normally, deep learning systems learn from examples of both healthy and unhealthy cases. However, in medicine, abnormal or diseased images are much rarer and harder to collect. Because of this, recent models have adopted the paradigm of training exclusively on normal, healthy images, simplifying real-world deployment by removing the need for scarce or hard-to-collect anomalous data. Once the model is put into use, it looks at new medical images and checks whether they differ from what it learned to be normal. If something looks unusual, the system marks it as a possible abnormality. What makes this approach especially useful is that it doesn’t just say whether an image looks abnormal, it also shows where the abnormality might be. By highlighting the suspicious areas, it helps doctors better understand and verify the model’s findings, improving the decision-making process. In addition, a major challenge is that medical imaging covers many different body parts and scanning technologies. When a system learns to detect problems in one area (for example, lungs), it often “forgets” what it learned when trained on another (for example, brain scans). This problem, known as Catastrophic Forgetting, is addressed in the literature through a research area called Continual Learning, in which models learn from new data while retaining knowledge from previously seen data. Therefore, we applied Visual Anomaly Detection (VAD) models to the medical imaging domain and incorporated Continual Learning techniques to enable the model to learn the normal distribution of new medical domains while preserving previously acquired knowledge. This approach allows the system to remain up to date and capable of identifying abnormalities across multiple medical sources. We evaluated our method using the BMAD dataset collection and the well-established PatchCore model, adapted for continual learning, referred to as PatchCoreCL.

Perspectives