Health systems are being pitched a growing number of AI tools for clinical, administrative, and operational work. But before those tools are purchased, hospitals and health-care organizations may not always have a clear, consistent way to ask vendors for the information they need.

A new framework from the Health AI Partnership (HAIP), published in NEJM AI, is designed to give procurement teams a more detailed checklist for those conversations. The HAIP AI Vendor Disclosure Framework outlines what health-care delivery organizations should ask before buying an AI system, including what the tool does, how it was tested, what happens to the organization’s data, what it will take to integrate, what it will cost, and who is responsible for monitoring it after launch.

Although the framework is not oncology-specific, its procurement questions may also be relevant to cancer care organizations considering AI tools.

“Instead of robust evidence, vendor engagement teams are often presented with marketing claims that lack transparency regarding the system’s intended use, performance metrics, data sources, and limitations,” the authors, including senior author Mark Sendak, MD, MPP, the Population Health & Data Science Lead at the Duke Institute for Health Innovation, wrote.

What Health Systems Should Ask Vendors

The framework is built upon key assessment questions in five areas of disclosure, beginning with a basic but often complicated question: What does the AI system actually do?

In terms of system capabilities and intended use, vendors are asked to define the tool’s purpose, intended users, affected populations, implementation setting, and point of use during a care episode. They are also asked to explain the limitations of the tool. The authors distinguish between experimental tools, which may have restricted uses and minimal validation, and well-established tools, which have been validated across settings and supported effectiveness and safety analyses.

The next set of questions addresses whether the tool performs as advertised. In the solution performance and compliance domain, vendors are asked to describe how the model was evaluated, which performance metrics were used, and what results were observed. The framework also asks for subgroup performance assessments, bias evaluation and mitigation steps, known risks, and regulatory status. If a product is not regulated, vendors are asked to explain why it does not meet the criteria for a regulated device. The area also includes a question about contract expectations, including liability and indemnification terms.

Vendors are then asked to explain how they would secure the organization’s data, whether they would use those data for secondary purposes, and how long the data would be retained. The framework also addresses who owns the data, how data would be transferred, and what happens at the end of the relationship, including plans for data destruction if the vendor goes out of business or discontinues the product.

The framework then moves from the product itself to the practical work of implementation. Vendors are asked how the AI system would connect with existing health IT systems, what technical infrastructure is required, which integration paths are available, and what staff would be needed to implement and manage the tool. Cost is included here as well, with the framework asking vendors to account for up-front costs, ongoing operational costs, and potential hidden expenses.

The last area, life cycle management, focuses on what happens after the tool goes live. Vendors are asked who will maintain and update the system, what performance thresholds would trigger remediation, and how users would be able to flag workflow problems. The framework also asks vendors to define adverse event reporting procedures, long-term performance and value metrics, and the point of contact for system oversight.

Building the Vendor Evaluation Framework

The framework was developed using a participatory design approach. Four vendor engagement team members from three U.S.-based health-care delivery organizations within HAIP collaborated over 3 months. The organizations included academic medical centers and a state health plan with existing AI governance structures.

Developing the framework included a review of existing vendor disclosure materials, semistructured interviews to identify gaps and priorities, and three rounds of structured feedback.

The first prototype was built around five shared priorities: solution scope, model development and performance, monitoring and evaluation, bias assessment, and insights and reporting. Stakeholder feedback led to several additions.

The framework was then expanded to include large language model–specific issues, including prompt engineering and hallucination risk. More details on total cost of ownership, data stewardship and governance, vendor communication quality, and service-level agreement requirements were also added.

The final version clarified responsibilities for vendors and health-care delivery organizations, added continuous monitoring with defined intervention thresholds, required disclosure of failure modes and mitigation strategies, and included exit and decommissioning criteria.

Implementation Challenges

For health-care organizations, the framework turns vendor review into a more structured process: what needs to be known before purchase, what should be written into the contract, and what must be monitored after implementation. It may also give vendors a clearer sense of what information health systems expect before adopting an AI system.

The authors noted, however, that the framework does not eliminate the need for internal technical expertise. Health-care organizations still need staff who can collect vendor information, interpret responses, and decide whether the disclosures are sufficient to support procurement.

That requirement may be easier for larger organizations with established AI governance programs. Smaller or underresourced organizations, including some safety-net providers, may have more difficulty turning technical disclosures into purchasing decisions. The authors suggested that shared resources and collaborative networks could help build capacity as AI procurement becomes more complex. They also described the framework as a “communication tool” between vendors and health-care organizations, rather than a compliance exercise.

DISCLOSURES: This work was funded by the Gordon and Betty Moore Foundation. For full disclosures of the study authors, visit ai.nejm.org. A downloadable vendor assessment form is also available at healthaipartnership.org.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.