A prescription renewal is not the same thing as a refill.

That distinction is at the center of a debate in Utah over whether an AI system should be allowed to help decide when certain existing prescriptions can be renewed. A refill generally means a pharmacy is dispensing medication that has already been authorized. A renewal means issuing a new prescription for a medication a patient has taken before: a decision that must still take into consideration symptoms, adverse effects, lab monitoring, drug interactions, and whether the treatment remains appropriate.

That is why Utah’s pilot with Doctronic has drawn attention beyond primary care. The program does not include oncology therapeutics, but it is testing a question cancer care may eventually have to confront: when does AI stop supporting clinical work and start doing it?

The current phase still includes physician review, but later phases would allow the AI to move further on its own. To Utah officials and Doctronic, the pilot is a controlled way to test whether AI can expand access to routine care. To critics, it asks regulators to accept too much uncertainty before the evidence is in.

“The question that we need to ask is, is the technology getting ahead of the evidence, and what’s the risk to patients and public trust for doing that?” Daniel G. Aaron, JD, an associate professor at the University of Utah’s S.J. Quinney College of Law and co-author of a recent JAMA Viewpoint on the Doctronic pilot, explained to ASCO AI in Oncology.

How the Pilot Is Structured

Utah’s Office of Artificial Intelligence Policy (OAIP) is overseeing the pilot through what the state calls a regulatory mitigation agreement—a type of regulatory sandbox that allows a company to test a limited use case under state oversight, rather than asking regulators to fully approve or reject it upfront. In written responses, the office said the agreement is part of a broader effort “to evaluate real-world AI use cases under structured oversight.”

“The goal is not to endorse any particular company or product,” the office said, “but to determine whether certain AI-assisted health-care workflows can be deployed safely and responsibly, and in ways that improve access or value for Utahns.”

The office described the pilot as focused on AI-assisted prescription renewals, “not on open-ended autonomous medical practice.” Prescription-renewal workflows may involve narrower clinical questions than new diagnosis or treatment initiation, the office said, but still require “careful safeguards, escalation pathways, and human clinical oversight.”

Doctronic describes the pilot in more direct terms. Byron Crowe, MD, Chief Medical Officer at Doctronic, said the Utah pilot allows the company’s AI “to determine whether a patient’s existing prescription is reasonable and safe to renew, and then to autonomously send that prescription to a pharmacy.”

In the first phase, however, that determination still goes through a physician.

“In the first phase of the program, physicians review every prescription renewal determination made by the AI and decide whether it is safe to proceed or whether more information is required from the patient,” Dr. Crowe said. “When the supervising physician determines that more information is needed, they conduct a standard telehealth visit with the patient.”

Then, in the second and third phases, physician review would take place retroactively “on a sample of renewals to ensure ongoing quality and safety,” Dr. Crowe said.

Safety Questions and Competing Views

That shift is one reason the pilot has drawn criticism from the Utah Medical Licensing Board. In an April 20 letter, the board said it learned of the agreement only after implementation and called for the program to be “immediately suspended pending further discussion.” The board argued that continuing a medication can require reassessment and clinical decision-making, including monitoring for adverse effects, contraindications, drug interactions, and whether the therapy remains appropriate.

Aaron said the current physician review is an important backstop, but it does not answer the larger question of whether the AI system is safe enough to move into later phases.

For the first 250 patients, he said, physicians are essentially checking whether the AI’s determinations match their own. But without more transparency, it is difficult to know how often they agree, where they disagree, and what those disagreements mean.

“We don’t know whether this AI is actually practicing medicine effectively; whether the drugs it’s prescribing are appropriate,” Aaron said.

He said 250 cases may not be enough to establish reliability across a formulary that includes 192 commonly prescribed drugs. Some medications may not appear at all in the first phase, and risk can vary widely by drug and patient.

That concern is not limited to whether the AI can recognize obvious contraindications. Aaron said a renewal request can quickly become a diagnostic question. If a patient reports a headache, leg pain, fatigue, or other symptom, the system may need to determine whether the symptom is unrelated, an adverse effect, a sign of worsening disease, or a new clinical concern.

“Renewals are pretty complicated,” Aaron said. “It is prescribing a drug. It’s different from a refill.”

The company has noted that the Doctronic AI system includes safeguards to catch cases where renewal may not be appropriate. Dr. Crowe said the AI uses “safety layers encompassing both AI and deterministic checks,” including medication verification from a national prescription database, drug-safety checks against standardized reference data, and an AI consult intended to identify new patient concerns or the need for laboratory monitoring.

Responding to the medical board’s concerns, Dr. Crowe said Doctronic is “participating in the process exactly as designed, with defined safeguards, physician oversight of every prescription in the first phase of the program, and continued physician involvement throughout.”

“We remain focused on demonstrating safe, evidence-based expansion of routine care access,” he said.

The state has made a similar argument: that the pilot allows Utah to test a new AI workflow under supervision, rather than allowing the technology to develop without clear state visibility. The Office of Artificial Intelligence Policy said Doctronic is required to provide regular reports intended to identify adverse events, complaints, patterns in physician review, cases requiring more patient information, and other signals relevant to safety and public value.

Recently, the Utah OAIP provided an update on the first 5 months of the pilot program, noting that no serious safety incidents have been reported to date to the Office from either Doctronic or the public, and the pilot remains in phase one. The OAIP noted that they are still collecting reports from Doctronic physicians, but have yet to make firm determinations about the performance of the AI prescription renewal pilot. They have also initiated an independent review of the interactions.

The report noted that the AI system renewed prescriptions 72% of the time without needing to escalate the case to a physician. Physicians agreed with the renewal 91% of the time. In cases where the AI system requested escalation or more information, reviewers determined that the escalation was appropriate 69% of the time; the other 31% of the time, the escalation was considered overly cautious.

Legal and Regulatory Considerations

Aaron said his concern is not only what data will eventually be released, but whether the pilot is designed to capture the right data in the first place. If follow-up relies too heavily on patients responding to text messages, he said, the program could miss patients who become too sick to respond, seek care elsewhere, or experience serious complications outside the platform. Without stronger systems for tracking outcomes, he said, regulators may not know whether patients were harmed, or whether the AI’s recommendations were actually safe. Aaron said the pilot sits at the intersection of state medical-practice law and federal regulation because the AI is involved in prescribing drugs and may itself function as a medical device. Still, outside experts said the quality of the evidence matters as much as the existence of reporting.

Sara Gerke, a health law scholar at the University of Illinois College of Law, and co-author of a recent Perspective in The New England Journal of Medicine about the implications of autonomous AI in patient care, said that the relationship between state and federal law “stands out most” in the Doctronic pilot.

“It is highly questionable whether an AI tool can independently renew prescriptions under the Federal Food, Drug, and Cosmetic Act,” Gerke said. “Even if certain requirements are waived under a state pilot program, Doctronic must still comply with applicable federal law.”

Gerke said there may be lower-risk renewal scenarios where AI could potentially be used safely, such as a patient with well-controlled hypertension. But she said autonomous tools should ideally be rigorously tested and assessed by the FDA for safety and effectiveness before deployment.

Why Oncology Professionals Should Pay Attention

The Utah pilot may seem far removed from daily cancer care, given that cancer treatments were deliberately excluded from the initial stages of the program. But, Aaron said, oncology professionals should pay attention because the same regulatory pathway could eventually be used for more complex areas of AI use in medicine.

“Almost certainly, I don’t think Doctronic wants to stop at those drugs,” Aaron said. “I could certainly see Utah or another state allowing the use of AI to prescribe cancer drugs now or in the future, either as a recommendation tool or perhaps even autonomously eventually.”

That does not mean autonomous AI oncology prescribing is imminent. Cancer care involves layers of complexity that go well beyond many routine prescription renewals, including disease stage, molecular testing, treatment history, toxicities, performance status, comorbidities, patient preferences, and rapidly evolving evidence. In oncology, AI may first enter in more limited or clinician-supervised roles: drafting prior authorizations, monitoring symptoms between visits, helping triage patient messages, summarizing records, or reinforcing decisions about supportive-care medications.

But those lower-risk uses can still raise similar questions: What level of physician oversight is required? What data should be public? Who is responsible if something goes wrong? And when does a tool that supports a clinician begin to substitute for one?

Aaron said he is not opposed to AI in medicine. He said AI may eventually help clinicians make better decisions, including by accounting for more information than a human could reasonably process alone.

“Maybe AI will be able to pick drugs very accurately that benefit patients and take into account large amounts of data to do so that are beyond the human capacity,” he said. “But to get there, we need to gather the proper evidence and evaluate it transparently, so that patients are actually benefiting and this isn’t just driven by financial goals.”

For now, Utah’s pilot is testing that balance in prescription renewals. Oncology is not the test case. But if AI systems move from answering questions to helping determine care, the oncology community will have to decide where lines should be drawn—and what evidence should be required before AI can advance.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.