Original Article
Comparative analysis of the performance of ChatGPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Pro large language models on theoretical orthodontic questions
View article page
Buket ErdemORCID, Salih Baver AtaşORCID, Meriç ArslanORCID, Çağla ŞarORCID
Cite
Erdem B, Ataş SB, Arslan M, Şar Ç. Comparative analysis of the performance of ChatGPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Pro large language models on theoretical orthodontic questions. Int Dent Res. 2026;16:e260777. https://doi.org/10.5577/intdentres.777
- eISSN
- 2146-1767
- Submitted
- 6 September 2026
- Revised
- 30 September 2026
- Accepted
- 15 October 2026
- Published
- 19 October 2026
- Pages
- e260777
Abstract
Aim: This study aimed to comparatively evaluate the performance of ChatGPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Pro large language models (LLMs) in answering theoretical orthodontic questions under web search-enabled conditions.
Methodology: A total of 30 open-ended questions covering the topics of growth and development, functional analysis, mixed dentition, extraoral appliances, malocclusion, and orthodontic treatment planning were prepared from Contemporary Orthodontics (7th edition, 2025). The questions were presented to all three models under standardized conditions in separate sessions in March 2026 with web search enabled. The responses were anonymized and independently evaluated in a blinded manner by two orthodontists using a 5-point Likert scale incorporating accuracy, scientific adequacy, completeness, and clarity into a single overall score. Data were analyzed using IBM SPSS Statistics (version 31.0); the Friedman test was used for model comparisons, and statistical significance was set at p < 0.05.
Results: Mean performance scores were 4.40 ± 0.72 for ChatGPT-5.4, 4.43 ± 0.61 for Claude Sonnet 4.6, and 4.32 ± 0.88 for Gemini 3.1 Pro. Median scores were 4.50, 4.50, and 4.75, respectively. No statistically significant difference in overall performance was found among the models (χ² = 0.400; df = 2; p = 0.819; Kendall's W = 0.007). Subgroup analyses by topic category also revealed no significant differences in any category (all p > 0.05). Inter-rater reliability was limited (α = 0.706).
Conclusion: All three models achieved high-performance scores, with no statistically significant difference in overall performance among them; this does not establish equivalence. With independent verification, these LLMs may serve as supportive tools in orthodontic education.
Keywords
License
© 2026 The Author(s). Published by International Dental Research.