ChatGPT responses to patients rated more empathetic than doctors – research
ChatGPT also appeared to score higher than doctors on the quality of responses to patients.
Your support helps us to tell the story
From reproductive rights to climate change to Big Tech, The Independent is on the ground when the story is developing. Whether it's investigating the financials of Elon Musk's pro-Trump PAC or producing our latest documentary, 'The A Word', which shines a light on the American women fighting for reproductive rights, we know how important it is to parse out the facts from the messaging.
At such a critical moment in US history, we need reporters on the ground. Your donation allows us to keep sending journalists to speak to both sides of the story.
The Independent is trusted by Americans across the entire political spectrum. And unlike many other quality news outlets, we choose not to lock Americans out of our reporting and analysis with paywalls. We believe quality journalism should be available to everyone, paid for by those who can afford it.
Your support makes all the difference.A chatbot appears to be “significantly more empathetic” than doctors when responding to questions from patients, according to scientists.
Researchers asked a team of licensed healthcare professionals to rate responses from doctors as well as ChatGPT, a computer program designed to simulate online conversations with humans.
They found that the proportion of responses rated “empathetic” or “very empathetic” was higher for ChatGPT than for physicians.
ChatGPT also appeared to score higher than doctors on the quality of responses to patients.
Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit
Writing in the journal Jama Internal Medicine, the researchers said further studies are needed to evaluate whether chatbots such as ChatGPT can be used in clinical settings to help reduce burnout in doctors and other healthcare professionals.
They said: “In this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum.
“Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit.
“Randomised trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.”
For the study, the researchers looked at the questions asked by patients on the social media forum Reddit, which were answered by a verified physician.
The same questions were then put forward to ChatGPT.
Responses from doctors and ChatGPT were anonymised and randomly rated by healthcare professionals.
The human relationship and care-giving is a vital part of the healthcare process and something which cannot be automated or replaced by chatbots such as ChatGPT
Of the 195 questions and responses, results showed that, overall, evaluators preferred the chatbot’s responses to the doctors’.
ChatGPT responses were also rated significantly more empathetic than physician responses, the researchers said, and the proportion of responses rated as “good” or “very good” quality was higher for the chatbot than for physicians.
Commenting on the study, Mirella Lapata, professor of natural language processing, University of Edinburgh, said: “The study assesses ChatGPT’s ability to provide responses to patient questions and compares these to answers written by physicians.
She added: “Without controlling for the length of the response, we cannot know for sure whether the raters judged for style (eg, verbose and flowery discourse) rather than content.”
Dr Mhairi Aitken, ethics research fellow at The Alan Turing Institute, said that it is important to consider the perspectives of patients and not just professionals when using chatbots.
She added: “It’s important to note that while some people may feel comfortable receiving medical advice from a chatbot, or for a chatbot to assist in a physician’s advice, for many patients the human relationship and care-giving is a vital part of the healthcare process and something which cannot be automated or replaced by chatbots such as ChatGPT.
“A human doctor is able to adjust their language, manner, and approach in response to social cues and interactions, whereas a chatbot will produce more generic language without awareness of social contexts.”