“The team randomly sampled 195 exchanges from AskDocs where a verified physician responded to a public question.”
The physicians in this study were answering strangers on Reddit. ChatGPT beat them on quality 78.5% to 22.1% and on empathy 45.1% to 4.6%, judged by a panel of three. That proves a doctor typing a quick Reddit reply is terse, not that a chatbot is safe. UC San Diego Health is already starting to draft patient messages with ChatGPT on the strength of it.