The AI Clinic · Issue 017AI oversight & trust
You're the detector.
Epic's AI chart summaries, checked line by line
Six stories this week, and most of them land on the same question. When AI writes something about your patient, or for your student, who checks it?
- Researchers checked Epic's AI chart summaries claim by claim, and 11.79% of claims weren't supported by the chart. The study is a preprint.
- In a survey of 55 patients at a UK fracture clinic, most knew AI was used in health care, but few understood what an ambient scribe does.
- An npj Digital Medicine paper argues that using AI to read your sources needs different rules than using it to polish your writing.
The Big Story
Epic's AI chart summaries, checked line by line
Researchers at a seven-hospital health system tested Epic IP Insights, the generative AI tool that summarizes a hospital patient's chart inside the EHR. They built an automated checker that broke each summary into small factual claims and verified every claim against the source notes. On a validation set, the checker agreed with physician reviewers on 96.95% of claims.
Across 671 summaries holding more than 40,000 claims, 11.79% weren't supported by the chart. The tool drew on a mean of 5.8% of the notes available to it. Regenerate a summary for the same patient and the wording came back identical only 32.7% of the time. Clinicians found it easy to use (97.8%), but fewer than half said they'd rely on it with little checking.
Mark's read: This is a preprint, so it hasn't been through peer review. Even so, roughly one claim in eight not backed by the chart is a number to remember the next time a vendor says the AI "reads the chart for you." Here it read about 6% of the notes it had access to. Most outpatient clinics aren't using this exact tool, but the same problem shows up in anything that condenses a referral packet or last episode's notes. Easy to use and accurate are two separate questions. This health system had the people to build a hallucination detector. You probably don't, so you're the detector.
In the Clinic
What patients think about an AI listening in
A survey of 55 adults at a UK orthopaedic fracture clinic asked how they felt about an ambient AI scribe recording and summarizing their visit. Most (72.7%) knew AI was being used in health care, but only 10.9% said they had a strong understanding of what ambient AI does. Acceptance was moderate. Their concerns were transparency, privacy, consent, accuracy, and what happens to the relationship with their clinician.
Mark's read: One clinic and 55 people won't tell you what your patients think. It does hand you a consent script. Tell them what the tool records, where the recording goes, who checks the note, and how to say no. Then make saying no easy, without making the patient feel like they're slowing your day down.
The Rounds
AI phone assistants keep raising money
Vocca, which sells an AI phone assistant to health care practices, raised a $20 million Series A. The company says it grew from 2,000 to 15,000 practitioner users in a year.
Mark's read: Growth numbers in a funding announcement come from the company, so read them as marketing. The category is real, though, and the front desk phone is where a lot of small clinics quietly lose new patients. Before you sign with any of these vendors, call the demo line as a confused patient with a billing question and see how fast it gets you to a human.
Academic Corner
Using AI to write is different from using it to read
In npj Digital Medicine, Zhicheng Lin of Yonsei University mapped publishing policies on AI-assisted writing from 2023 through August 11, 2026. Most cover the same ground: what uses are allowed, human authorship and accountability, disclosure, verification, and confidentiality. He proposes oversight that scales with what the AI is doing, from editing to drafting to "surrogate reading" (letting AI read the sources for you) to methods work, with extra safeguards when the writing is part of learning or assessment.
Mark's read: Surrogate reading is the part I'd put in front of students. Having AI clean up a paragraph and having it read the paper you're citing are very different, and most syllabus policies treat them as one thing. Write your policy by task, not by tool.
Academic Corner
A proposed career track: the clinician-AI scientist
A Viewpoint in The Lancet Digital Health from Tien Yin Wong, Pearse Keane, and colleagues proposes a new career path modeled on the clinician-scientist. The clinician-AI scientist would be trained in both clinical care and the technical side, and would understand implementation science across the whole path from building an AI tool to getting it used. They also suggest what clinicians, and the institutions that employ them, can do to grow that pool.
Mark's read: The authors come from ophthalmology, but the gap they describe is just as wide in rehab. I don't see many PTs sitting between the people building these tools and the people expected to use them. If that's the work you want as faculty, bring this paper to the conversation about what counts toward promotion.
Academic Corner
Watermarks, grant review, and arXiv limits
Andy Stapleton's weekly academia roundup covers OpenAI's text watermarking, AI showing up in grant review, and new submission limits at arXiv.
Mark's read: Watermarks and detectors won't settle whether a human did the thinking. The fair test hasn't changed. Can you defend every sentence and every citation that has your name on it?
Try This
Audit five AI-written notes
Pull five notes or summaries an AI tool wrote for you this month. A scribe note, a referral summary, a discharge letter, whatever you have. Check every factual sentence against the source, and mark any you can't back up. That's the big story's study shrunk down to one clinician and a highlighter. If you mark more than one or two, you know where your review time needs to go.
The last word
If you use an AI scribe, how do you ask patients for consent? Hit reply and tell me what you actually say.
Get the next issue.
One email a week. The AI that matters for clinical work, in plain language.