Pilot validation of a digital tool for predicting the risk of temporomandibular disorders: integration of Delphi and ICONIX
DOI:
https://doi.org/10.56294/hl2025879Keywords:
Temporomandibular Joint Disorders, Mobile Applications, Diagnostic Self Evaluation, Surveys and Questionnaires, Decision Support Systems ClinicalAbstract
Introduction: Temporomandibular disorders (TMD) are prevalent conditions that often remain underdiagnosed due to limited access to standardized tools. Objective: To conduct a preliminary evaluation of the diagnostic performance of a web-based screening application for TMD, developed as a digital health tool aimed at early detection and support clinical decision-making. Methods: A cross-sectional pilot study was conducted between March and May 2024 at the Department of Clinical Surgery, School of Stomatology, Benemérita Autonomous University of Puebla (Mexico). A convenience sample of 18 undergraduate students (>18 years) completed a 10-item self-administered web questionnaire, designed using the Delphi method and weighed through the ICONIX modeling approach. Clinical evaluation was performed blinded to the application results, using the Diagnostic Criteria for Temporomandibular Disorders (DC/TMD) as the reference standard. Statistical analysis included descriptive statistics, Student’s t-test, Cohen’s kappa coefficient, and ROC curve analysis. Results: The web application demonstrated promising diagnostic performance, correctly identifying 12 of 13 clinically confirmed TMD cases. Sensitivity was 92.3%, specificity 60%, positive predictive value 85.7%, and overall, diagnostic accuracy 83.3%. The area under the ROC curve (AUC) was 0.762, with a Youden index of 0.52. Cohen’s kappa coefficient was 0.56, indicating a moderate level of agreement between digital classification and clinical diagnosis. No significant differences were observed in mandibular functional variables between groups with and without TMD. Conclusion: This preliminary pilot study suggests that the web-based application is a feasible and moderately accurate tool for syndromic risk stratification of TMD. While the results are encouraging, the small sample size and limited population generalizability highlight the need for future studies with larger and more diverse samples to validate its diagnostic performance.
References
1. Garstka AA, Kozowska L, Kijak K, Brzózka M, Gronwald H, Skomro P, Lietz-Kijak D. Accurate Diagnosis and Treatment of Painful Temporomandibular Disorders: A Literature Review Supplemented by Own Clinical Experience. Pain Res Manag. 2023 Jan 31; 2023:1002235. DOI: 10.1155/2023/1002235.
2. Wilkie G, Al-Ani Z. Temporomandibular joint anatomy, function and clinical relevance. Br Dent J. 2022 Oct;233(7):539-546. DOI: 10.1038/s41415-022-5082-0.
3. Ferrillo M, Giudice A, Marotta N, Fortunato F, Di Venere D, Ammendolia A, Fiore P, de Sire A. Pain Management and Rehabilitation for Central Sensitization in Temporomandibular Disorders: A Comprehensive Review. Int J Mol Sci. 2022 Oct 12;23(20):12164. DOI: 10.3390/ijms232012164.
4. Daline IH, Slade GD, Fouad AF, Nixdorf DR, Tchivileva IE. Prevalence of painful temporomandibular disorders in endodontic patients with tooth pain. J Oral Rehabil. 2023 Jul;50(7):537-547. doi: 10.1111/joor.13457.
5. Mallya SM, Ahmad M, Cohen JR, Kaspo G, Ramesh A. Recommendations for imaging of the temporomandibular joint. Position statement from the American Academy of Oral and Maxillofacial Radiology and the American Academy of Orofacial Pain. Oral Surg Oral Med Oral Pathol Oral Radiol. 2022 Nov;134(5):639-648. DOI: 10.1016/j.oooo.2022.06.007.
6. Zieliński G, Pająk-Zielińska B, Ginszt M. A Meta-Analysis of the Global Prevalence of Temporomandibular Disorders. J Clin Med. 2024 Feb 28;13(5):1365. DOI: 10.3390/jcm13051365.
7. Melo V, Monteiro L, Orge C, Sales M, Melo J, Rodrigues B, Melo A. Prevalence of temporomandibular disorders in the Brazilian population: A systematic review and meta-analysis. Cranio. 2025 Jul;43(4):629-636. DOI: 10.1080/08869634.2023.2276627.
8. Qvintus V, Sipilä K, Le Bell Y, Suominen AL. Prevalence of clinical signs and pain symptoms of temporomandibular disorders and associated factors in adult Finns. Acta Odontol Scand. 2020 Oct;78(7):515-521. DOI: 10.1080/00016357.2020.1746395.
9. Murphy MK, MacBarb RF, Wong ME, Athanasiou KA. Temporomandibular disorders: a review of etiology, clinical management, and tissue engineering strategies. Int J Oral Maxillofac Implants. 2013 Nov-Dec;28(6):e393-414. DOI: 10.11607/jomi.te20.
10. List T, Jensen RH. Temporomandibular disorders: Old ideas and new concepts. Cephalalgia. 2017 Jun;37(7):692-704. DOI: 10.1177/0333102416686302.
11. Da-Cas CD, Valesan LF, Nascimento LPD, Denardin ACS, Januzzi E, Fernandes G, Stuginski-Barbosa J, Mendes de Souza BDM. Risk factors for temporomandibular disorders: a systematic review of cohort studies. Oral Surg Oral Med Oral Pathol Oral Radiol. 2024 Oct;138(4):502-515. DOI: 10.1016/j.oooo.2024.06.007.
12. Son C, Park YK, Park JW. Long-term evaluation of temporomandibular disorders in association with cytokine and autoantibody status in young women. Cytokine. 2021 Aug;144:155551. DOI: 10.1016/j.cyto.2021.155551.
13. Li DTS, Leung YY. Temporomandibular Disorders: Current Concepts and Controversies in Diagnosis and Management. Diagnostics (Basel). 2021 Mar 6;11(3):459. DOI: 10.3390/diagnostics11030459.
14. Patel K, Eley KA, Cascarini L, Watt-Smith S, Larkin M, Lloyd T, Maddocks C, McLaren E, Stovell R, McMillan R. Temporomandibular disorders-review of evidence-based management and a proposed multidisciplinary care pathway. Oral Surg Oral Med Oral Pathol Oral Radiol. 2023 Jul;136(1):54-69. DOI: 10.1016/j.oooo.2023.02.001.
15. Story WP, Durham J, Al-Baghdadi M, Steele J, Araujo-Soares V. Self-management in temporomandibular disorders: a systematic review of behavioural components. J Oral Rehabil. 2016 Oct;43(10):759-70. DOI: 10.1111/joor.12422.
16. Lam J, Svensson P, Alstergren P. Internet-Based Multimodal Pain Program With Telephone Support for Adults With Chronic Temporomandibular Disorder Pain: Randomized Controlled Pilot Trial. J Med Internet Res. 2020 Oct 13;22(10):e22326. DOI: 10.2196/22326.
17. Cho YK, Jung YL, Im A, Hong SJ, Kim K. Social Media-Based Pain Neuroscience Education for Temporomandibular Joint Disorder: A Randomized Controlled Trial. Pain Manag Nurs. 2025 Jun;26(3):e261-e269. DOI: 10.1016/j.pmn.2024.12.010.
18. Palmer, J.; Penlington, C.; Durham, J. Supported self-management in temporomandibular disorders: A systematic review of behavioural components. Surgery. 2023;16(2):228–236. DOI: 10.1111/ors.12773
19. Cuschieri S. The STROBE guidelines. Saudi J Anaesth. 2019 Apr;13(Suppl 1):S31-S34. DOI: 10.4103/sja.SJA_543_18
20. World Medical Association. World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Participants. JAMA. 2025 Jan 7;333(1):71-74. DOI: 10.1001/jama.2024.21972.
21. Kotiranta U, Suvinen T, Kauko T, Le Bell Y, Kemppainen P, Suni J, Forssell H. Subtyping patients with temporomandibular disorders in a primary health care setting on the basis of the research diagnostic criteria for temporomandibular disorders axis II pain-related disability: a step toward tailored treatment planning? J Oral Facial Pain Headache. 2015 Spring;29(2):126-34. DOI: 10.11607/ofph.1319.
22. Kotiranta U, Forssell H, Kauppila T. Painful temporomandibular disorders (TMD) and comorbidities in primary care: associations with pain-related disability. Acta Odontol Scand. 2019 Jan;77(1):22-27. DOI: 10.1080/00016357.2018.1493219.
23. Penlington C, Bowes C, Taylor G, Otemade AA, Waterhouse P, Durham J, Ohrbach R. Psychological therapies for temporomandibular disorders (TMDs). Cochrane Database Syst Rev. 2022 Aug 11;8(8):CD013515. DOI: 10.1002/14651858.CD013515
24. Ekberg E, Nilsson IM, Michelotti A, Al-Khotani A, Alstergren P, Rodrigues Conti PC, Durham J, Goulet JP, Hirsch C, Kalaykova S, Kapos FP, King CD, Komiyama O, Koutris M, List T, Lobbezoo F, Ohrbach R, Palermo TM, Peck CC, Penlington C, Restrepo C, Rodrigues MJ, Sharma S, Svensson P, Visscher CM, Wahlund K, Rongo R; International Network for Orofacial Pain and Related Disorders Methodology (INfORM). Diagnostic criteria for temporomandibular disorders-INfORM recommendations: Comprehensive and short-form adaptations for adolescents. J Oral Rehabil. 2023 Nov;50(11):1167-1180. DOI: 10.1111/joor.13488.
25. Rongo R, Ekberg E, Nilsson IM, Al-Khotani A, Alstergren P, Rodrigues Conti PC, Durham J, Goulet JP, Hirsch C, Kalaykova SI, Kapos FP, King CD, Komiyama O, Koutris M, List T, Lobbezoo F, Ohrbach R, Palermo TM, Peck CC, Penlington C, Restrepo C, Rodrigues MJ, Sharma S, Svensson P, Visscher CM, Wahlund K, Michelotti A. Diagnostic criteria for temporomandibular disorders in children and adolescents: An international Delphi study-Part 2-Development of Axis II. J Oral Rehabil. 2022 May;49(5):541-552. DOI: 10.1111/joor.13301.
26. Antunes da Cunha T, Alstergren P, Pereira Júnior FJ, Gonçalves DAG, Chaves TC, Biasotto-Gonzalez DA. Brazilian Portuguese version of the Diagnostic Criteria for Temporomandibular Disorders (DC/TMD) Axis I: Cross-cultural adaptation, criterion validity, and reliability. Braz J Phys Ther. 2025 Mar-Apr;29(2):101187. DOI: 10.1016/j.bjpt.2025.101187
27. Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine (Baltimore). 2023 Feb 17;102(7):e32829. DOI: 10.1097/MD.0000000000032829.
28. Spranger J, Homberg A, Sonnberger M, Niederberger M. Reporting guidelines for Delphi techniques in health sciences: A methodological review. Z Evid Fortbild Qual Gesundhwes. 2022 Aug;172:1-11. DOI: 10.1016/j.zefq.2022.04.025.
29. Tavares LF, Brusaca LA, Calixtre LB, Locks F, Oliveira AB. O.3.2-8 Sedentary behaviour, physical activity, and sleep patterns of women with chronic temporomandibular disorders on days with and without pain: a cross-sectional study. Eur J Public Health. 2023 Sep 11;33(Suppl 1):ckad133.153. DOI: 10.1093/eurpub/ckad133.153
30. Bhargava, D., ed. Temporomandibular Joint Disorders: Principles and Current Practice. Singapore: Springer, 2021. Print.
31. Khatoon, S, Suresh S, Ilyas M, Rani P, Khokhar M, Ahmed S. Evaluate of the prevalence and severity of symptoms of temporomandibular joint dysfunction (TMJD). Sindh Institute of Oral Health Sciences. 2020; 27(6); 1157-1164.
32. Sierwald I, John MT, Schierz O, Hirsch C, Sagheri D, Jost-Brinkmann PG, Reissmann DR. Association of temporomandibular disorder pain with awake and sleep bruxism in adults. J Orofac Orthop. 2015 Jul;76(4):305-17. DOI: 10.1007/s00056-015-0293-5.
33. Ohrbach R, editor. Diagnostic Criteria for Temporomandibular Disorders: Assessment Instruments. Version 15May2016. www.rdc-tmdinternational.org. Accessed on July 1, 2025.
34. Rosenberg D, Stephens M. Use Case Driven Object Modeling with UML: Theory and Practice. Berkeley, CA: Apress; 2007.
35. Pellizzoni L, E Silva SA, Falavigna A. Multilanguage health record database focused on the active follow-up of patients and adaptable for patient-reported outcomes and clinical research design. Int J Med Inform. 2020 Mar;135:104065. DOI: 10.1016/j.ijmedinf.2019.104065.
36. Srivastava KC, Shrivastava D, Khan ZA, Nagarajappa AK, Mousa MA, Hamza MO, Al-Johani K, Alam MK. Evaluation of temporomandibular disorders among dental students of Saudi Arabia using Diagnostic Criteria for Temporomandibular Disorders (DC/TMD): a cross-sectional study. BMC Oral Health. 2021 Apr 26;21(1):211. DOI: 10.1186/s12903-021-01578-0.
37. Casanova-Rosado JF, Medina-Solís CE, Vallejos-Sánchez AA, Casanova-Rosado AJ, Hernández-Prado B, Avila-Burgos L. Prevalence and associated factors for temporomandibular disorders in a group of Mexican adolescents and youth adults. Clin Oral Investig. 2006 Mar;10(1):42-9. DOI: 10.1007/s00784-005-0021-4.
38. Manjunatha BS, Alzahrani MS, Alotaibi OI, Amith HV, Alshamrani AS. Relationship between bad oral habits, signs, and symptoms of temporomandibular joint disorders among Saudi population: A cross-sectional study. J Oral Maxillofac Pathol. 2023 Jan-Mar;27(1):115-120. DOI: 10.4103/jomfp.jomfp_381_22.
39. Chattrattrai T, Aarab G, Su N, Blanken TF, Mitrirattanakul S, Lobbezoo F. The association of self-reported awake bruxism and sleep bruxism with temporomandibular pain and dysfunction in adult patients with temporomandibular disorders. Clin Oral Investig. 2023 Dec;27(12):7501-7511. DOI: 10.1007/s00784-023-05338-y.
40. Figueiredo C, Afonso A, Caramelo F, Corte-Real A. Temporomandibular joint trauma and disability assessment - A longitudinal exploratory study. J Forensic Leg Med. 2021 Aug;82:102230. DOI: 10.1016/j.jflm.2021.102230.
41. Vanecek RJ, Talcott GW, Tabor, A, McGeary DD, Lang M, Ohrbach R. Prevalence of TMD and PTSD symptoms in a military sample. J Appl Biobehav Res 2011, 16(3-4); 121–137. DOI: 10.1111/j.1751-9861.2011.00069.x
42. Kumar LS, Naik Z, Panwar A, Sridhar M, Bagewadi A. Prevalence of temporomandibular disorders and their correlation with gender, anxiety, and depression in dental students – A cross-sectional study. J Indian Acad Oral Med Radiol 2022; 34(3): 281–285. DOI: 10.4103/jiaomr.jiaomr_65_22
43. Ujin Yap A, Cao Y, Zhang MJ, Lei J, Fu KY. Age-related differences in diagnostic categories, psychological states and oral health-related quality of life of adult temporomandibular disorder patients. J Oral Rehabil. 2021 Apr;48(4):361-368. DOI: 10.1111/joor.13121.
44. Skopec M, Issa H, Reed J, Harris M. The role of geographic bias in knowledge diffusion: a systematic review and narrative synthesis. Res Integr Peer Rev. 2020 Jan 15;5:2. DOI: 10.1186/s41073-019-0088-0.
45. Hess DR. Observational Studies. Respir Care. 2023 Nov;68(11):1585-1597. DOI: 10.4187/respcare.11170.
46. Zieliński G, Gawda P. Defining Effect Size Standards in Temporomandibular Joint and Masticatory Muscle Research. Med Sci Monit. 2025 May 1;31:e948365. DOI: 10.12659/MSM.948365.
47. Shreffler J, Huecker MR. Diagnostic Testing Accuracy: Sensitivity, Specificity, Predictive Values and Likelihood Ratios. 2023 Mar 6. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2025 Jan
48. Monaghan TF, Rahman SN, Agudelo CW, Wein AJ, Lazar JM, Everaert K, Dmochowski RR. Foundational Statistical Principles in Medical Research: Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value. Medicina (Kaunas). 2021 May 16;57(5):503. DOI: 10.3390/medicina57050503
49. Madadizadeh F, Ghafari H, Bahariniya S. Kappa Statistics: A Method of Measuring Agreement in Dental Examinations. Open Public Health J. 2023; 16; E18749445259818. DOI: 10.2174/0118749445259818231016040344
50. Nahm FS. Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol. 2022 Feb;75(1):25-36. DOI: 10.4097/kja.21209. Epub 2022 Jan 18
51. Zhu J, Chen Z, Zhao J, Yu Y, Li X, Shi K, Zhang F, Yu F, Shi K, Sun Z, Lin N, Zheng Y. Artificial intelligence in the diagnosis of dental diseases on panoramic radiographs: a preliminary study. BMC Oral Health. 2023 Jun 3;23(1):358. DOI: 10.1186/s12903-023-03027-6.
52. Vlăduțu D, Popescu SM, Mercuț R, Ionescu M, Scrieciu M, Glodeanu AD, Stănuși A, Rîcă AM, Mercuț V. Associations between Bruxism, Stress, and Manifestations of Temporomandibular Disorder in Young Students. Int J Environ Res Public Health. 2022 Apr 29;19(9):5415. DOI: 10.3390/ijerph19095415.
53. Karthik R, Hafila MIF, Saravanan C, Vivek N, Priyadarsini P, Ashwath B. Assessing Prevalence of Temporomandibular Disorders among University Students: A Questionnaire Study. J Int Soc Prev Community Dent. 2017 Jun;7(Suppl 1):S24-S29. DOI: 10.4103/jispcd.JISPCD_146_17.
54. Saczuk K, Lapinska B, Wawrzynkiewicz A, Witkowska A, Arbildo-Vega HI, Domarecka M, Lukomska-Szymanska M. Temporomandibular Disorders, Bruxism, Perceived Stress, and Coping Strategies among Medical University Students in Times of Social Isolation during Outbreak of COVID-19 Pandemic. Healthcare (Basel). 2022 Apr 15;10(4):740. DOI: 10.3390/healthcare10040740.
55. Bof de Andrade F, Teixeira DSDC, Moreira RDS, de Oliveira C. Prevalence and associations of temporomandibular disorders in older Brazilian adults. Gerodontology. 2024 Jun;41(2):263-268. DOI: 10.1111/ger.12701.
56. Simmatis L, Barnett C, Marzouqah R, Taati B, Boulos M, Yunusova Y. Reliability of Automatic Computer Vision-Based Assessment of Orofacial Kinematics for Telehealth Applications. Digit Biomark. 2022 Jul 21;6(2):71-82. DOI: 10.1159/000525698.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Irene Aurora Espinosa-de Santillana , Héctor Fuentes-Barría, Olga Patricia López-Soto , Brandon Eduardo Hernández-López , Elena Aurora Popoca-Hernández, Cybele Neves-Moutinho, Raúl Aguilera-Eguía (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
