Researchers: Chatty AIs provide sometimes dangerous medical tips

Dresden, 3 July 2023. Freely accessible artificial intelligences (AI) such as ChatGPT are not a reliable source of medical advice. Prof Stephen Gilbert from the „Else-Kröner-Fresenius-Center for Digital Health“ at TU Dresden has pointed this out. However, as more and more sick people are googling symptoms and remedies, and the growing use of AI in search engines is also foreseeable, Gilbert is therefore calling for certification procedures for virtual chatbots, such as those used for medical products.

62% of Germans consult Dr Google first

According to a Bitkom survey 62 per cent of Germans consult „Dr Google“ or other internet or app-supported sources before seeing a real doctor. In comparison: in 2020, this rate was only 53 per cent. And because AIs are now able to provide astoundingly detailed explanations, doctors fear that more and more internet users are tending to trust the information they receive in this way.

It has been shown that such entertainment AIs „äußcan only provide reliable information when they are confronted with medical questions“, Gilbert and his colleagues admit. „They often generate convincing statements that are demonstrably false or provide inappropriate answers.“ In particular, AIs would not know „medical ground truths“, i.e. a „process method that ensures that the data on which the analysis is based is up-to-date, accurate, complete and well maintained 

Gilbert's claim „To earn their place in the medical repertoire, chatbots must be designed for greater accuracy, with safety and clinical effectiveness proven and approved by regulators.

Article of the "Oiger" from 03.07.2023