AI-based large language models may benefit pre-surgery decisions in epilepsy

Key takeaways:

• ChatGPT was better at predicting seizure semiology for frontal, temporal lobes vs. epileptologists.
• Epileptologists were more accurate when interpreting semiology for the cingulate cortex.

AI-based large language models may be feasible to assist clinicians for pre-surgical decision making in those with drug-resistant focal epilepsy, according to a study published in the Journal of Internet Medical Research.

“Seizure semiology has a clinical value to pinpoint the locations of epileptogenic zones,” Feng Liu, PhD, MEng, an assistant professor in the department of systems and enterprises at the Schaefer School of Engineering and Science at the Stevens Institute of Technology in Hoboken, N.J., told Healio.

“Interpreting seizure semiology requires extensive expertise and training, and the variations in their descriptions make the interpretation more complicated,” Liu said.

Large language models (LLMs) such as ChatGPT have been explored for their potential as AI-based analysis tools, which may be suited to interpret detailed seizure semiology descriptions for accurate epileptogenic zone (EZ) localization, Liu and colleagues wrote.

As such, they sought to evaluate ChatGPT’s ability to correctly interpret seizure semiology ahead of epilepsy surgery compared with the performance of epileptologists.

Their study comprised two distinct data cohorts: a publicly sourced cohort of 852 semiology-EZ pairs (average age, 23.66 years; 47.4% men) from 193 peer-reviewed journal publications from the previous 20 years, and a private cohort of 184 semiology-EZ pairs (average age, 28.67 years; 51.1% men) collected from Far Eastern Memorial Hospital (FEMH) in Taiwan from 2017 to 2021.

ChatGPT-4 was utilized to predict the most likely EZ locations using both zero-shot prompting (ZSP) and few-shot prompting (FSP).

Concurrently, eight epileptologists filled out online surveys asking them to interpret 100 randomly selected semiology records. Responses from human and artificial subjects were subsequently compared with regional sensitivity (RSens), weighted sensitivity (WSens) and net positive inference rate (NPIR) metrics for all regions of the brain.

In the publicly sourced cohort, results showed that, for RSens, ChatGPT registered 0.88 in FSP and 0.9 in ZSP in the frontal lobes; for the temporal lobes, it was 0.83 for FSP and 0.81 for ZSP. In the occipital lobe, ZSP and FSP were 0.42 and for the parietal lobe, 0.26 (ZSP) and 0.22 (FSP). For WSens, ZSP and FSP values were 0.69 and 0.67, respectively.

The RSens for the privately sourced cohort were as follows: in the frontal lobe, ZSP and FSP were both 0.87; in the temporal lobe, FSP was 0.83 and ZSP 0.81; in the occipital lobe ZSP and FSP achieved 0.38 and in the parietal lobe, ZSP was 0.34 and FSP 0.32. WSens values were 0.74 for FSP and 0.73 for ZSP.

Researchers additionally reported that ChatGPT-4 outperformed a selection of five epileptologists in seizure semiology for frontal and temporal lobes, with comparable results evaluating the parietal and occipital lobes.

Conversely, epileptologists were more accurate when interpreting semiology for the cingulate cortex, indicating that LLM performance wanes when analyzing less frequently observed regions of the brain.

Data further showed ChatGPT-4’s superiority in WSens, with an FSP of 0.63 and ZSP of 0.61 compared with a range of 0.49 to 0.51 from medical professionals.

However, Liu and colleagues wrote, no significant differences were observed between ChatGPT and the epileptologists regarding NPIR.

“This study showed that leveraging large language models, such as ChatGPT, can provide a very reasonable interpretation of seizure semiology compared to epileptologists with years of experience,” Liu said. “This result motivated us to develop a fine-tuned LLM for interpreting the seizure semiology and prediction of epileptogenic zones.”

Reference:

ChatGPT helps pinpoint precise locations of seizures in the brain, aiding neurosurgeons. https://www.stevens.edu/news/chatgpt-helps-pinpoint-precise-locations-of-seizures-in-the-brain-aiding. Published May 12, 2025. Accessed May 14, 2025.

For more information:

Feng Liu, PhD, MEng, can be reached at neurology@healio.com.