Richard Gawel , 2025-05-02 12:52:00
Key takeaways:
- Artificial intelligence can scan electronic health records for risk factors indicating potential primary immunodeficiencies.
- These systems can accelerate diagnoses of primary immunodeficiencies by 3 years.
Artificial intelligence can improve care for patients with primary immunodeficiencies, but human input is still necessary, Nicholas L. Rider, DO, said at Updates in Primary Immunodeficiency 2025.
“AI can bring value in health care, and certainly bring value to our patients with primary immune disorders, but we need to set it up for success, remember how it’s used, and where it can be pointed to,” Rider, professor in the department of health systems and implementation science, section of allergy-immunology, Virginia Tech Carillion School of Medicine, said during his presentation.
AI in health care
As early as the 1950s, Rider said, health care was using computational approaches to support diagnoses and provide clinical decision support.
“Around 2011, deep neural networks were used to ingest very disparate, complicated health care data to make predictions, but it was not widely implemented,” he said.
ChatGPT was then “released into the wild” in November 2022, and the impact of large language models (LLMs) in health care grew, Rider said.
Rider noted a difference between automatic thinking such as recognition and recall, which may be fast but prone to error, and thinking that requires effort such as remembering patterns and reasoning, which is slower but more reliable.
LLMs are good at recall when there is a large body of information to draw from such as PubMed or the entire internet, Rider said, but they are not as good at reasoning.
“There are some AI models that are good at reasoning, but largely what we’re implementing today are models that are good at recall,” he said. “We’re looking at models that can do prediction and recall efficiently.”
Current LLMs can assist in increasing patient engagement, personalizing patient care, improving clinical documentation, automating health records and attaining prior insurance authorization, Rider said.
“We’re not really feeling the difference yet as practicing clinicians,” he said.
The diagnostic odyssey
There are more than 550 described primary immunodeficiency (PI) disorders across 10 International Union of Immunological Societies categories, Rider said, with susceptibility to infection, pathological inflammation, immune dysregulation, cancer predisposition and atopic disease.
“We think about using AI to model clinical immunology. We’re thinking about a very diverse and complex environment,” he said.
Although PI is seen as a rare disease, Rider said that he and his colleagues used an Optum data set of 73 million individuals to find a prevalence of 60 per 100,000, or a minimum of 198,870 patients across the United States.
“Collectively, these disorders are not rare, but we do estimate that about 50% of these individuals are going undiagnosed,” he said.
Mean times to diagnosis include 5 years for immune dysregulatory disease, 8.8 years for common variable immunodeficiency and 9.5 years for primary antibody deficiencies, Rider said.
“This leads to everything we don’t want in health care — these long diagnostic odysseys that are expensive, painful and lead to sub-optimal outcomes,” he said. “Can AI make a difference here?”
Multiple studies have shown that AI and machine learning can be used to review electronic health records and claims data, single nucleotide variants or flow cytometry data to predict risks for disease, genomic risk factors and lab phenotypes, Rider said.
In addition to using these population-level risk stratification methodologies, Rider continued, open-source tools need to be built, and knowledge and systems need to be scaled to systematically shorten the diagnostic lag.
“We want to build tools that will be useful for anyone, whether you’re in a tip-of-the-spear center like the Cleveland Clinic or many of the institutions that are represented here, or you’re in a small health system in rural America,” he said.
AI in practice
Rider and his colleagues have implemented a two-step process for identifying patients at risk for PI in the Texas Children’s Health Plan.
First, the Software for Primary Immunodeficiency Recognition Intervention & Tracking (SPIRIT) Analyzer ingests claims data and other information from EHRs from everyone enrolled in the health care system.
The tool, which is freely available and updated, enumerates prioritized codes via weighted risk factors to identify patients at high, medium, low and no risk for a primary disorder.
“If we run SPIRIT Analyzer across the population, about 1% of that population is going to have an apparent amount of PI risk that would warrant potential referral,” Rider said.
“The second step we really wanted to focus on was prioritizing those individuals of a high risk as who should be seen first by a clinical immunologist and who could maybe have a subset of labs and/or be evaluated by their primary care doc and maybe be triaged,” he continued.
A machine learning model then stratifies these individuals for prioritization referral, according to Rider.
“We can expedite the diagnosis by at least 3 years,” he said.
Rider and his colleagues analyzed how this system performed between September 2020 and March 2022 among 427,110 (53% female; 46% Hispanic) individuals with Medicaid coverage.
At baseline, prevalence of PI was 70 per 100,000 individuals. The system identified 297 individuals at high risk and 6,330 at low to medium risk for PI.
Reviews of the data at 6-month intervals found new diagnoses for PI, including 27 in the baseline high-risk group (9%), 95 in the baseline low-to-medium risk group (1.5%) and 914 among patients who were not identified with any risk (0.2%).
Individuals in the high-risk and low- to medium-risk groups at baseline also had statistically significantly higher rates of hospitalization than the group with no identified risk at baseline.
“This very simple, straightforward tool that can ingest very simple claims data can actually give you a pretty good snapshot today of those individuals who are high risk and are much more likely to have a diagnosis of PI down the line,” Rider said. “They’re also going to have higher health care utilization, presumably due to their underlying immune disorder.”
Additional verification
In another study, Rider and his colleagues extracted clinical notes from 6,401 patients with confirmed primary immune disorder or inborn errors of immunity, including 952,356 notes from before their diagnoses and 1,399,954 from afterward. The study also included 2,352,310 notes from 28,100 controls.
Natural language processing (NLP) examined 122,726 enriched terms, 308 terms relevant to the domain and 16 underlying concepts in the clinical notes using four different models.
“Almost 100% of the time, it made the right ascertainment of primary immune disorder vs. control,” Rider said.
Areas under the curve included 0.95 when analyzing notes from 36 months before diagnosis, 0.94 when analyzing notes from 18 months before diagnosis, 0.96 when analyzing notes from 12 months before the diagnosis and 0.97 when analyzing notes from 6 months before diagnosis and from the date of diagnosis.
“We can identify high-risk individuals that ultimately will be enriched for the diagnosis of PI,” Rider said. “In an NLP filter, we can shorten this diagnostic odyssey by at least 3 years.”
Other LLMs that are now available online also can perform population-level risk stratification, Rider said, noting a study that asked six LLMs to use 7.3 billion to 200 billion parameters to assess 25 previously published, de-identified, verified primary disorder cases.
“We graded each of the models in terms of its ability to get the right diagnosis that we ultimately knew and also provide useful information,” Rider said. “The models with more parameters were more accurate. They generally did a better job of ascertaining the right diagnosis.”
The six LLMs had a mean diagnostic accuracy of 72.4% with individual accuracies of 42.3%, 53.8%, 61.5%, 88.5%, 92.3%, and 96.2%.
Based on the Revised IDEA score, which Rider said measures the usefulness of clinical responses or notes, the models with higher diagnostic accuracy also provided more helpful and useful recommendations.
“The goal here, then, is to say, OK, what are the limitations of these LLMs, and how can we improve these LLMs and potentially deploy one of them as an expert across the internet so that anyone with internet access would have essentially access to an expert?” Rider said.
AI adoption
Rider said that he and his colleagues are interested in facilitating the adoption of AI.
“This is challenging,” he said. “We can build models, but until the models can be adopted into health systems, it’s going to be hard for them to make a difference.”
Rider cited the success of Project ECHO, which began with hepatologists who provided subspecialty expert information to primary care providers in rural areas so patients would not have to travel long distances to tertiary care centers.
Using this model, Rider and his colleagues launched Project ECHO for Primary Immune Disorder, or PEPI, last year. Nine countries have participated in the project, which has two goals.
“One is just to provide free educational information in CME, which we’re looking to provide this year,” Rider said. “The other is to be a resource to anyone, especially rural clinicians in the Appalachian region, which is close to where we are.”
Clinicians can schedule analytical case consultations with de-identified health information with the PEPI team.
“We’re open to anyone in the world, and we want to be supportive,” Rider said.
In addition to PEPI, Rider said that he and his colleagues “want to drive web-based guidance through the AI,” as described through their study of six LLMs.
“We also want to build models that not only do the what, but do the why,” he said. “They predict why this population of patients maybe is not doing as well. AI can also support in this direction.”
These elements together represent a learning health system, Rider said. According to the National Academy of Medicine, Rider said, “science, informatics, incentives and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral byproduct of the delivery experience” in a learning health system.
“This is what we want to build,” Rider said. “A learning health system is really driving at improving health care value, so providing greater quality at the lowest cost possible. But AI is only a part of that.”
For example, a learning health system would gather population health data in a repository and use AI to ascertain risks and diagnoses and then find patients who would need referral to a clinical immunologist, Rider said.
The system also would look at the population of patients with PI and possibly mine information from the literature, he continued. Once patients are diagnosed, AI could then support what comes next, such as determining why patients are not responding to therapy and what else could be useful.
“This is really a systematic practice of measuring and iteratively improving care,” Rider said.
But health care systems might not be ready to implement these systems, with culture and complexity presenting challenges, he added.
“The health system cannot be ramshackle. It has to be rooted and situated and shaped in a way where it can actually make use of AI to the best of its ability,” Rider said.
Yet Rider remains optimistic about AI’s potential in PI care, especially as more research into its use is developed.
For example, the second Artificial Intelligence and Primary Immunodeficiency conference was held in March in New York City with 130 attendees and more than 30 speakers from 16 countries, Rider said, with more conferences planned.
“We’re starting to see this community grow, and this is going to help us identify use cases, as well as gaps and opportunities that AI has for our field. Stay tuned,” he said.
References:
For more information:
Kristen J. Polinski, PhD, can be reached at allergy@healio.com.