As agentic AI becomes increasingly capable of performing clinical functions, the American Medical Association (AMA) has outlined a governance framework intended to guide decisions about what the system should be permitted to do and who should be held accountable for their actions and outcomes.

Developed through the AMA Center for Digital Health and AI, the framework is based on the governing principle that “clinical authority should be earned through evidence and maintained through demonstrated performance,” with authority and oversight evolving alongside changes in evidence, performance, risks, or the technology itself.

The framework identifies four interconnected considerations for governing agentic AI:

• Clinical function: Define the clinical task an agentic system is expected to perform, the patient population and care setting, and its intended use. Potential functions range from documentation and monitoring to triage, recommendations, diagnosis, treatment, and prescribing.

• Evidence and risk: Evaluate a system’s performance, safety, clinical benefit, applicability to the intended patient population, and uncertainty in proportion to the potential consequences of failure. Greater autonomy and more consequential risks require stronger supporting evidence.

• Authority and oversight: Establish what a system is permitted to do and the human or institutional oversight required for safe use.

• Accountability: Determine responsibility for AI-related risks and outcomes based on stakeholders’ authority, control, knowledge of risks, and ability to prevent or mitigate harm. Stakeholders considered in allocating accountability include developers, deployers, health systems, physicians, and payers.

The framework also outlines a process for continuous performance evaluation, depicting an ongoing cycle of deployment, monitoring, learning, and reassessment rather than treating the initial evaluation of a system as the endpoint of governance.

These considerations will guide the AMA’s work with physicians and other stakeholders as they consider how regulation, licensure (ie, authorization for an AI system to operate in a defined clinical capacity), and liability should apply to agentic AI assuming clinical functions.

“The AMA is bringing physicians together with those who build, deploy, and regulate AI to shape responsible governance…,” the framework states. “As AI capabilities advance, evidence, authority, oversight, and accountability must remain aligned in service of better patient care.”

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.