Clinicians submitted millions of queries to the AI-enabled clinical information platform OpenEvidence, with use varying across clinician types, specialties, and career stages and questions more frequently focused on treatment and management than diagnosis, according to a study led by researchers at Harvard Medical School and Brigham and Women’s Hospital in Boston and published in JAMA.

“Oncology and hematology accounted for a particularly high share of use relative to their representation in the physician workforce, underscoring the importance of evaluating these tools around the decisions clinicians actually face,” said first author Vishal R. Patel, MD, MPH, in an interview with ASCO AI in Oncology.

Who Is Using OpenEvidence?

OpenEvidence is an AI-powered clinical decision support platform for health-care professionals that uses retrieval-augmented generation and large language models to provide answers to medical questions based on trusted medical sources. 

In a cross-sectional study looking at platform usage among U.S. clinicians, corresponding author Anupam B. Jena, MD, PhD, and colleagues analyzed use from 952,008 clinicians, including 556,200 physicians, and examined 271 million queries submitted between January 1, 2024, and June 30, 2026. A stratified random sample of 38.8 million queries was categorized by clinical purpose and topic using a large language model. During the 2.5-year period, monthly active users grew from 355 to 703,265, with the average number of queries per active user per month increasing from 18.0 to 39.5.

Nurse practitioners and physician assistants submitted the most queries, each submitting an average of 44.5 queries per active user-month compared with 37.1 among physicians.

Physicians within 10 years of medical school graduation submitted a disproportionate share of queries, accounting for 24.3% of the national workforce but 45.2% of users and 61.8% of queries (use ratio = 2.54)—thus, those early in their careers are “leading the way,” coauthor Michael Liu-Hass, MD, MPhil, wrote in a statement. Similarly, residents and fellows represent 14.8% of the workforce but submitted 27.2% of queries (use ratio = 1.84).

By physician specialty, oncology and hematology (1.9% of the workforce vs 3.8% of queries; use ratio = 2.00), internal medicine (14.5% vs 27.3%; 1.88), and family medicine (12.5% vs 22.5%; 1.80) showed higher relative use. In contrast, relative use was lower among procedural and diagnostic specialties, such as ophthalmology (1.9% vs 0.4%; 0.23), radiology (4.3% vs 1.1%; 0.24), and orthopedics (3.0% vs 0.7%; 0.24).

What Are They Using OpenEvidence For?

According to Dr. Liu-Hass, use of the platform was largely driven by “real clinical decisions,” which accounted for 51.9% of queries from physicians. Among individual query purposes, treatment and management represented the largest share (33.1%), whereas diagnosis accounted for 13.9%.

Asked how these real-world patterns of use should inform future evaluations of clinical AI tools, Dr. Patel explained: “Evaluations should go beyond diagnostic accuracy to assess the reliability of guidance on treatment selection, medication dosing, and ongoing management. We also need to understand how clinicians use that guidance because an accurate answer alone does not establish better care.”

Physicians also frequently submitted queries outside their own specialties, Dr. Liu-Hass wrote, ranging from 35% among oncologists and hematologists to 88% among emergency department physicians. The investigators identified such cross-specialty querying as another pattern of real-world use that differs from the focus of current evaluations of clinical AI.

Among other limitations, they noted that the analysis was restricted to a single AI-enabled clinical information platform and that they could not determine whether queries occurred during patient encounters, clinical preparation, or nonclinical activities. However, "engagement is not evidence of patient benefit," Rohan Khera, MD, MS, and coauthors pointed out in an accompanying editor's note. 

“Study findings suggest a growing share of knowledge access is shifting from recall to retrieval,” the investigators concluded. “Whether this transition improves the quality and efficiency of care will depend on the accuracy of these tools, the training clinicians receive in using them, and the willingness of institutions to adapt how AI is taught, evaluated, and governed.”

Dr. Patel added, “The next step is to determine whether use changes clinical decisions and improves patient outcomes and whether those effects vary across clinicians and patient populations. Our study describes use rather than benefit, so safety, appropriate reliance, and effects on clinical learning remain important questions.”

DISCLOSURES: The study received federal funding. Dr. Patel reported receiving grant support from the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health. Dr. Jena reported receiving personal fees from Analysis Group, Freakonomics, M.D., Doubleday Books, The New York Times, Los Angeles Times, The Wall Street Journal, and Harry Walker Agency, as well as serving as an unpaid board member of the United Network for Organ Sharing. Dr. Liu-Hass reported receiving consulting fees from Accompany Health. Other study authors not mentioned in this article are employed by OpenEvidence. For complete author disclosures, visit jamanetwork.com.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.