Quality of Erectile Dysfunction Information from AI Chatbots

What is it about?

The study evaluates the quality of information presented by artificial intelligence chatbots, specifically focusing on erectile dysfunction (ED). The most popular search queries on ED were collected from Google Trends and the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) website. The validated instruments Patient Education Materials Assessment Tool (PEMAT) and DISCERN were used to evaluate understandability, actionability, and overall information quality. The Flesch-Kincaid formula was used to evaluate readability. The AI chatbots included in the study were ChatGPT, Perplexity, Chatsonic, and Microsoft Bing AI. Most chatbots cited sources frequently from reputable websites such as Mayo Clinic, Urology Care Foundation, Johns Hopkins Medicine, and the NIDDK. ChatGPT was the only chatbot that did not cite sources in its responses. Actionability was found to be low across all chatbots, while understandability was moderate. Responses were written at a difficult reading level, and response length was short. There was very little to no misinformation on AI chatbots, with ChatGPT having some misinformation on topics regarding the onset action time and duration of action of commonly prescribed phosphodiesterase type 5 inhibitors.

Featured Image

Why is it important?

This research is important because it assesses the quality and accuracy of medical information provided by AI chatbots, specifically in relation to erectile dysfunction (ED). As AI chatbots become more popular, it is crucial to evaluate the information they provide to ensure it is reliable and actionable for users. Key Takeaways: 1. The study used the top five Google Trends queries related to ED and the headers of the ED page on the NIDDK website as inputs into AI chatbots. 2. Most AI chatbots cited sources frequently from Mayo Clinic, Urology Care Foundation, Johns Hopkins Medicine, and the NIDDK websites. 3. ChatGPT was the only AI chatbot that did not cite sources in its responses. 4. The responses from AI chatbots were of high quality and contained accurate information, but had limitations in terms of readability, understandability, and actionability for the average healthcare consumer. 5. AI chatbots lack visual aids to help explain complex health topics, which lower their understandability scores. 6. The content provided by AI chatbots is generally more accurate compared to social media platforms, but it still has limitations that need to be addressed.