The interpretation of urodynamic studies (UDS) requires clinical expertise and proficiency in ICS terminology [1]. Large language models (LLMs) have demonstrated capabilities in complex medical tasks, but their performance in structured urodynamic interpretation has not been evaluated. Our hypothesis is that current LLMs can achieve substantial agreement with expert urologists. The aim was to compare three LLMs (Claude Opus 4.6, DeepSeek V3, Gemini 3 Pro) against a panel of expert urologists (gold standard) in the interpretation of UDS using Cohen’s Kappa index (κ), and to determine whether the observed discordances have clinical relevance for therapeutic decision-making.

Observational concordance study. Twenty consecutive UDS were randomly selected. Two expert urologists established the reference interpretation by consensus, assigning a primary diagnosis (the one that answers the urodynamic question and guides therapeutic decision-making) and a secondary diagnosis (additional relevant findings), coded into 9 categories: detrusor overactivity (DO) with incontinence, DO without incontinence, DO with low compliance, low bladder compliance, detrusor underactivity/acontractility, urgency without DO, stress urinary incontinence, bladder outlet obstruction, and no relevant pathology. Additionally, 14 binary variables of urodynamic findings and the type of leakage (4 categories) were recorded. Each study was independently interpreted by three LLMs using a standardised prompt following ICS terminology [1]. Cohen’s κ with 95% CI was calculated for each variable [2]. Discordances in the primary diagnosis were evaluated by the experts to determine their impact on therapeutic management.

Results are summarised in Table 1. Claude achieved the highest overall agreement (90.6%), followed by DeepSeek (79.1%) and Gemini (77.6%). Claude reached almost perfect agreement for both the primary (κ=0.885; 95% CI: 0.734–1.036) and secondary diagnosis (κ=0.868; 95% CI: 0.695–1.041), with only 2 discordances in the primary diagnosis: a misclassification between bladder outlet obstruction and detrusor underactivity, and an underestimation of low bladder compliance as a normal study. Both discordances were deemed clinically significant, as they would imply different therapeutic approaches (deobstructive surgery vs intermittent catheterisation; active treatment and upper tract monitoring vs discharge). For binary variables, Claude achieved κ≥0.81 in 5/14 and κ≥0.61 in 11/14. DeepSeek (κ primary diagnosis=0.662) and Gemini (κ=0.429) showed substantial and moderate agreement, respectively.

Claude demonstrated almost perfect agreement with expert urologists, clearly outperforming DeepSeek and Gemini. The two discordances in Claude’s primary diagnosis involved classically difficult distinctions in urodynamics (obstruction vs underactivity; subtle low compliance). Although both are clinically relevant, they represent scenarios where interobserver variability among urologists is also high. DeepSeek and Gemini showed limitations in assigning primary and secondary diagnoses (45% agreement in the latter), suggesting lower ability to integrate simultaneous findings. The sample size (N=20) limits the precision of estimates.

Current LLMs, particularly Claude, can achieve almost perfect agreement with expert urologists in the structured interpretation of UDS, although the few observed discordances carry clinically relevant implications. These findings suggest a potential role as a clinical decision-support tool in urodynamics, always under specialist supervision. Multicentre studies with larger samples are needed to confirm these results.

An International Urogynecological Association (IUGA)/International Continence Society (ICS) joint report on the terminology for female pelvic floor dysfunction. Int Urogynecol J. 2010;21(1):5-26; Neurourol Urodyn. 2010;29:4-20.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.