An AI assistant that asks patients questions and helps doctors generate better medical reports

What is it about?

Doctors make diagnoses by combining medical images with information gathered from patients. However, many existing AI systems mainly analyze medical images and do not actively collect additional clinical information through conversation. In this study, we developed PAMA (ProActive Multimodal Agentic System), an AI system that asks medically relevant questions, gathers information about symptoms and medical history, and combines these responses with chest X-ray images to generate diagnostic reports. By using medical knowledge to guide the conversation, PAMA collects more useful clinical information while avoiding unnecessary or repetitive questions. We evaluated PAMA on two public chest X-ray datasets and found that it generated more accurate diagnostic reports and improved disease identification compared with existing AI methods. Our work demonstrates that combining proactive patient interaction with medical imaging can make AI-assisted diagnosis more accurate, informative, and clinically useful.

Featured Image

Photo by Nappy on Unsplash

Photo by Nappy on Unsplash

Why is it important?

Current AI systems for medical report generation often rely mainly on medical images and have limited ability to actively gather additional clinical information. Our work addresses this limitation by enabling AI to interact with patients, ask relevant follow-up questions, and combine these responses with medical images to support diagnosis. This more closely reflects how clinicians make decisions in real practice. By improving both report generation and disease identification, our approach may help develop more reliable, interactive, and patient-centered AI tools for healthcare