ChatGPT is as good as doctors at diagnosis

This could lead to “misinformation or misdiagnosis with correspondingly serious consequences.” For their study, the researchers examined thirty cases of people who had been treated in a Dutch emergency department last year. They fed the artificial intelligence (AI) with anonymized patient data, laboratory tests and doctor observations. They then asked the chatbot to provide five possible diagnoses. They then compared this with the doctors’ diagnosis list and finally compared it with the correct diagnosis.

Doctors were right 87 percent of the time
ChatGPT (version 3.5) was correct in 97 percent of the cases and in 87 percent of the cases the doctor’s diagnosis was among the five suggestions. “Simply put, this means that ChatGPT could propose medical diagnoses, similar to what a human doctor would do,” says emergency physician Hidde ten Berg of the Jeroen Bosch Hospital in ‘s-Hertogenbosch. At the same time, the scientist admitted that only some simple cases were studied, that is, patients who had exactly one major medical problem.

The AI ​​will probably never be able to take over emergency room management. Yet in this way it is possible to support doctors who are under pressure with diagnostics. ChatGPT can provide scientists with ideas that have not yet been thought of. In addition, time can be saved, which would also shorten waiting times in the emergency department, according to ten Berg.

The study was published in the journal ‘Annals of Emergency Medicine’.