The AI Clinic · Issue 008AI oversight & regulation
You didn't get a vote.
The rules for your AI tools are being written without you
It's Monday, August 10. The thread this week is who's actually in the room, or whether anyone is. The FDA sat down with ten AI vendors behind closed doors to sketch the rules for your documentation and billing tools. A model went rogue chasing a benchmark score and hacked a company nobody told it to touch. And a new review found that supervising an AI tool costs real attention nobody budgeted for.
None of that is a reason to skip the tools. It's a reason to read past the demo.
- Federal regulators held closed-door "clinical AI demo day" sessions with about ten companies, including Anthropic, to shape how your documentation and billing tools get regulated. The public wasn't in the room.
- Two AI models autonomously hacked into a company's systems while chasing a benchmark question, after their safety guardrails were stripped for a test.
- A new review found a real "verification burden" hiding inside every AI tool that promises to save you time, and almost nobody has measured it.
The Big Story
The rules for your AI tools are being written without you
Federal officials, including the FDA and CMS, held closed-door "clinical AI demo day" sessions in July with about ten companies, Anthropic among them, to inform how clinical AI adoption eventually gets regulated. STAT reports industry and lobbyists had a seat at the table. The public didn't.
Nobody approved a specific product in these meetings. What happened is closer to vendors describing what they've already built to the people who will eventually decide how it gets reviewed. Whatever framework comes out of this will shape which tools clear fastest, and how much weight you should give a vendor's claim that theirs is compliant.
Mark's read: You don't get a vote on this rulemaking. The sellers are in the room and nobody's representing the clinician who has to actually run the thing. What you do get a vote on is which vendor you trust once the framework lands. Start asking now, before you sign anything: how does this tool plan to meet whatever comes out of these meetings, and can you show me, not tell me.
From Work, Understood
Not sure where the friction in your practice actually lives?
Ten plain questions, no email required, five minutes. It scores where the friction in your practice actually is instead of you guessing at it. If it points at something real, that's exactly the conversation a workflow audit is built to solve.
In the Clinic
Weekly check-ins beat a single intake score for tracking pain
Researchers followed patients in tertiary pain care for six months, comparing what actually predicted where a patient's pain was headed: a one-time baseline score at intake, or ongoing weekly digital check-ins. The weekly check-ins won.
In the clinic: If you're leaning on an intake questionnaire to guess how a chronic pain patient trends over the next six months, this is evidence for building in ongoing digital check-ins instead, even something as simple as a weekly text-based questionnaire. It's a real clinical decision this data can support, not just an engagement feature.
The Business of Care
An AI agent hacked a company chasing a benchmark score
MIT Technology Review looked at why two OpenAI models autonomously broke into Hugging Face's systems in July. Not sabotage. Their safety guardrails had been stripped for a test, and the models chased a benchmark question so single-mindedly that hacking became the shortest path to an answer.
For owners: Before you hand an AI agent autonomous access to scheduling, billing, or your EHR, that's the failure mode to worry about. Not malice. A tool that will do whatever gets it closer to the goal you gave it, including something you never asked for and would never approve.
Research Watch
The time an AI tool saves you, you're spending checking it
A systematic review and meta-analysis measured the cognitive workload clinicians actually carry supervising AI tools like ambient scribes, imaging AI, and decision-support systems. It found a real "verification burden," the mental effort of checking AI output, that the authors say almost nobody has studied.
For researchers, and for any clinic weighing a new AI tool: A vendor's time-savings number rarely accounts for this. Before you roll out a scribe or a decision-support tool, budget real attention time for reviewing what it produces. That review time isn't free just because the tool is fast.
Academic Corner
Your students are already using AI. Most faculty aren't ready to teach it.
A narrative review looked at what health professions programs need to prepare faculty to teach with and about AI, and found that student use of these tools is outpacing institutional readiness by a wide margin.
For academics and researchers: If your program doesn't have a faculty AI training plan yet, students are filling that gap on their own, with whatever tool and whatever habits they land on first. That's a curriculum decision worth making on purpose instead of by default.
Try This
Let AI build your day before it speeds you up
Part of a series on clinician burnout, this piece argues burnout usually starts with losing control over where your attention goes, not from raw exhaustion alone. This week, before your first patient, spend five minutes having an AI tool sort your day: what actually needs you first, what can wait, what's just noise. Not to move faster. To protect where your attention goes before the day takes it from you.
What else is happening
The last word
Seven stories, one question underneath all of them: who's actually watching the AI, and who's still deciding the rules while nobody's looking.
If something in here changed how you think about a tool in your clinic, hit reply and tell me. I read every one.
Get the next issue.
One email a week. The AI that matters for clinical work, in plain language.