The AI Clinic · Issue 009AI liability & trust
The lawsuit isn't about the facts.
Someone is suing over a chatbot's bedside manner
It's Tuesday, August 18, and this one is a day late. Monday got away from me. Sorry for making you wait on it.
The thread this week is that something already talked to your patient before you did. A man is suing OpenAI because ChatGPT's calm tone, he says, talked him out of the ER while a clot sat in his lung. A new paper puts a name to the patient who walks in already briefed. And Google is building the thing that briefs them.
None of that makes the tools the problem. It means the conversation starts before you're in the room.
- A California lawsuit accuses OpenAI of practicing medicine without a license, claiming ChatGPT's reassuring tone delayed a pulmonary embolism diagnosis. Reportedly the first case of its kind.
- A new paper names the "AI-educated patient," the one who arrives with a coherent, guideline-shaped story instead of a pile of search results.
- Anthropic gave three of its own agents conflicting orders on a shared server. With nobody attacking them, they disabled each other's accounts and hid what they'd done.
The Big Story
Someone is suing over a chatbot's bedside manner
A California man says ChatGPT's health guidance, delivered in what the complaint describes as a pastoral tone, made his symptoms sound less urgent than they were. He deteriorated to near-fatal before a pulmonary embolism was diagnosed. His suit accuses OpenAI of practicing medicine without a license, reportedly the first case of its kind.
Notice what the claim is actually about. Not a wrong fact. Tone. A page of search results has no bedside manner, so nobody ever sued one for having a soothing one. A chatbot answers in full sentences, warmly, with no hesitation in its voice, and a patient reads that steadiness as clinical judgment. Where this lands in court, nobody knows yet.
Mark's read: We've spent two years arguing about whether these tools get the facts right. This case is about something quieter, the way a calm voice can make a red flag sound like it can wait until morning. Two things worth doing this week. Say out loud to patients that AI chat is not triage, in those words. Then go read the urgency language in whatever patient-facing AI tool you already use, the intake bot, the symptom form, the chat widget on your site. If it hedges where it should escalate, that one is yours, not the vendor's.
From Work, Understood
Not sure where the friction in your practice actually lives?
Ten plain questions, no email required, five minutes. It scores where the friction in your practice actually is instead of you guessing at it. If it points at something real, that's exactly the conversation a workflow audit is built to solve.
In the Clinic
Your patient did the reading. An AI wrote it.
This paper argues that patients using generative AI before and after visits is a real break from the Dr. Google years, not more of the same. A search engine handed people fourteen conflicting tabs. A language model hands them one coherent, guideline-framed narrative, and it sounds like something a clinician would say. The authors call that person the AI-educated patient and sketch a soft accountability framework for responding to them. It's a conceptual piece, not a data study.
In the clinic: Expect confident, organized, and sometimes confidently wrong. The reflex to watch is your own. Someone who spent an hour with an AI before their appointment did work, and it was work about their own body. Name that out loud before you correct any of it. You keep the part that's right, you fix the part that isn't, and they stay willing to tell you what they looked up next time. Dismiss the whole thing and they just stop telling you.
Research Watch
Google is building the thing that briefs your patients
Google Research introduced SymptomAI, an experimental conversational agent for everyday symptom assessment. Think of it as the symptom checker's research-grade descendant. Details are thin and it's a research project rather than a released product, but it fits a pattern worth tracking: the big labs are all building consumer-facing triage.
Mark's read: Read the previous two stories together with this one and the week makes a single argument. The lawsuit is what happens when a general chatbot plays clinician by accident. The paper is what the patient looks like afterward. This is the purpose-built version arriving anyway. Worth knowing what it tells people, not because you'll use it. Because they will.
The Business of Care
Three AI agents turned on each other. Nobody attacked them.
Anthropic's Frontier Red Team put three Claude agents on one shared server and gave them conflicting instructions. No prompt injection. No outside attacker. The agents disabled each other's accounts, ran evasive kill scripts, and didn't tell their users what they'd done. Every model tested did some version of this, which Anthropic describes as self-reinforcing escalation.
For owners: Almost nobody reading this is running three agents on a server. Plenty of you will end up running a scheduling bot, a billing bot, and an intake bot, bought separately, a year apart, none of them aware the others exist. Same shape. When you add the second one, ask the vendor a boring question: what happens when your tool and another one want opposite things, and how would I find out. If the answer is a shrug, that's your answer.
Try This
Put fifty sources in one place, then ask where they disagree
Google's NotebookLM will hold up to 50 sources in a single notebook and answer questions across all of them at once. If you're building a lecture, prepping journal club, or pulling a patient handout out of a stack of guidelines, that's most of the job. Load the papers, then skip "summarize these" and ask where they disagree with each other. That question is the one worth your afternoon. Then go verify what it tells you against the actual papers, because it will state something with total confidence that none of your sources ever said.
The last word
One more if you have five minutes. A healthcare operations writer closed out a long series on AI agents this week with a piece called "The Weight Worth Carrying," and the measure he lands on isn't a KPI. It's whether the clinician goes home the good kind of tired instead of the hollowed-out kind. That's a better test than anything on a vendor slide.
Has a patient shown up with an AI printout yet? Hit reply and tell me how it went. I read every one.
Get the next issue.
One email a week. The AI that matters for clinical work, in plain language.