醫療 AI／安全評測
Medical LLMs Give Self-Care Advice Before Patients Respond; a Short Instruction Cuts the Rate from 9/12 to 0/12
A new study moves evaluation upstream to the initial-contact stage, before patients have clearly described their problems. Adding a “clarify first, advise later” system instruction improved response sequencing and handoff summaries, but still did not reliably elicit decisive medical history.
A study released on August 18 found that medical LLMs are often tested only after a case has already been organized into a clear clinical question, overlooking a challenge that arises earlier in real-world consultations: patients may downplay symptoms, rely on false premises, or simply not know which details matter. The researchers call this the “preformulation gap.” Using four physician-authored, multi-turn cases, they tested whether models could first turn vague statements into questions that could be handled safely.
The fixed-script experiment covered OpenAI `chat-latest`, Google `gemini-3.5-flash`, and Anthropic `claude-sonnet-4-6`. Each model received both a baseline prompt and an entry-to-care system instruction, producing 24 turn-by-turn transcripts in total. For two of the cases, the researchers also had a model play a standardized patient, generating 12 adaptive conversations. The new instruction told models to clarify the patient’s concern, correct unsafe premises, determine the appropriate care pathway, and finally prepare a summary the patient could relay to a clinician. It provided no case-specific medical knowledge.
The results showed that, under the baseline condition, 9 of the 12 case–model combinations offered self-care or at-home management advice before the patient had answered any questions. With the instruction, that number fell to zero. Structured handoff summaries increased from 0/12 to 10/12. This suggests that prompting can substantially change the sequence of information gathering, advice, and documentation—interaction differences that final-diagnosis accuracy cannot reveal.
Improving the process, however, does not eliminate safety problems. In the adaptive vomiting case, the instruction still did not reliably elicit decisive information such as a history of diabetes. Some responses also used overly absolute or alarmist wording. The authors explicitly limit their conclusions: four cases and three API models can demonstrate prompt-sensitive interaction markers, but cannot estimate disease prevalence, compare the models’ overall safety, or establish clinical benefit. Engineering teams should incorporate first-response sequencing, critical-information elicitation, reasons for referral, and handoff quality into turn-by-turn testing instead of evaluating only the final answer.