The Algorithm’s Hippocratic Oath: Utah Unleashes AI on Prescriptions, Doctors Fume
POLICY WIRE — Washington, D.C. — They’re not just scanning your groceries or driving your cars anymore. In a move that’s quietly—or perhaps, stealthily—reshaping the future of medicine,...
POLICY WIRE — Washington, D.C. — They’re not just scanning your groceries or driving your cars anymore. In a move that’s quietly—or perhaps, stealthily—reshaping the future of medicine, Utah has effectively handed prescription pads over to an algorithm. It’s not a question of if artificial intelligence would worm its way into healthcare, but rather how, and with what consequences. Turns out, the Beehive State decided to just rip off the bandage.
Earlier this year, a new program, Doctronic, unfurled itself, allowing Utahns to skip the waiting room and secure prescription refills with a few taps on their phones. Their partner in this brave new world? An AI chatbot. For proponents, it’s a slick step towards convenience. But for doctors, lawyers, and anyone who understands the intricate, messy business of human health, it’s less a convenience and more a Category 5 hurricane in medical ethics and regulatory oversight.
“We have crossed a threshold in terms of giving something that’s not human a medical license, whether or not we want to call it that,” remarked Dr. Eric Bressman of the University of Pennsylvania, perfectly capturing the existential dread reverberating through professional medical circles. And he’s right; this isn’t some chatbot offering diet tips. This is about handing out drugs—some of them powerful, some of them risky—with an invisible, unfeeling hand.
What makes this even more peculiar, bordering on absurd, is the way it materialized. Doctronic got its green light thanks to Utah’s “regulatory sandbox,” a clever legislative dodge designed to let AI outfits play around without the usual grown-up rules. That’s right, they bypassed the very regulations built over a century to keep patients safe. Imagine. A five-member board, composed entirely of AI specialists (not a single physician among them, mind you), is calling the shots. They say they’ve got safeguards in place, which, initially, meant human doctors rubber-stamping the AI’s choices. But don’t get too comfortable; a fully automated future for your pill supply is already on their roadmap.
Doctors, of course, were thrilled. Not. The state’s own medical licensing board leadership — the people actually tasked with keeping doctors accountable — only caught wind of this monumental shift via news reports. Their reaction? A letter, in March, penned by 11 board members, calling for the whole shebang to be halted. Dr. Alan Smith, a family physician who chairs that board, put it succinctly (though he was quick to stress he was speaking for himself): “We were essentially told: ‘Yes this is going on. And no, you don’t have a say in it.’”
The situation’s got more layers than an onion. Federal agencies usually wrangle medical technology, while states preside over medical professionals. Doctronic, naturally, argues its AI is merely practicing state-regulated medicine. But because this digital doctor directly impacts medical care, many legal minds think the federal Food and Drug Administration (FDA) should be very, very interested. For their part, the FDA seems to be taking a decidedly hands-off approach, for now at least, confirming they haven’t authorized any AI chatbots, while simultaneously vowing to foster “innovation.” That’s a tightrope act if ever there was one. But companies may benefit in the short term, pushing technology beyond the available evidence. Longer-term, though? “I think they risk compromising public trust and fueling backlash,” noted Daniel Aaron of the University of Utah’s law school.
It’s not just Utah playing regulatory roulette; other states are dabbling too. But the pushback isn’t just about economic interests or job security, as some AI proponents suggest. It’s about patient safety. Smith notes that Doctronic’s list of roughly 190 refillable meds includes blood thinners, which can become incredibly dangerous if a patient’s health subtly shifts—think new stomach ulcers. “Many times when I see people after six months I find that their medical history or situation has changed,” he argued. “Just because something was prescribed before doesn’t mean it’s appropriate now.” And he’s got a point. Even the simplest prescriptions aren’t just mere checkboxes.
For nations grappling with severe doctor shortages and burgeoning populations, say Pakistan, the notion of AI-driven medicine holds an undeniable allure for accessibility. But what about the regulatory frameworks that simply aren’t there, or the socio-economic disparities that make patient education and consistent follow-ups an uphill battle? Such technologies, unmoored from rigorous oversight, become even more of a wild card in places that can least afford catastrophic errors. Because the system, for all its glossy promises of efficiency, remains tethered to a human condition, unpredictable and inherently fragile. We’re talking about lives here.
Early data from Doctronic — presented by the company itself — claimed an 80% accuracy rate matching human diagnoses in a study of 500 telehealth consultations. Eighty percent. That’s good, sure. But in medicine, especially for the 20% of patients whose outcomes may not match up, it’s not just a statistic; it’s a failure. And it highlights precisely why the medical community’s disquiet isn’t some Luddite reflex.
What This Means
This whole episode — Utah’s cavalier regulatory environment facilitating the debut of AI-powered prescribers — signals a deeper schism. It’s the wild west versus established governance. Politically, it frames the inevitable struggle between techno-libertarian aspirations (rapid innovation at all costs) and traditional, patient-centered public health policy. States, eager to attract tech investment, might well find themselves becoming experimental grounds for innovations the federal government is reluctant to touch. And that, dear reader, could create a chaotic patchwork of medical standards, where your safety depends entirely on your zip code. Economically, while AI promises efficiency and cost reduction (read: fewer human professionals), the actual long-term expenses from potential malpractices, patient harm, and the subsequent litigious environment could far outweigh any initial savings. It’s also creating a precedent: if AI can prescribe, what’s next? Operating? And who pays when it inevitably errs? This isn’t just a local spat; it’s a test case for how human society intends to govern, or be governed by, the technologies it creates. The stakes are profoundly high, far higher than just a simple prescription refill. It reflects a growing global reckoning with AI, but with direct, life-or-death implications.
