Generative artificial intelligence for diabetic retinopathy screening in Hong Kong
Keywords:
Diabetic retinopathy, Generative artificial intelligence, Mass screening, Sensitivity and specificityAbstract
Objectives: To compare the performances of EyeArt and endocrinologists in diagnosing more-than-mild DR (mtmDR) among patients with diabetes in a Hong Kong hospital.
Methods: Medical records of consecutive patients with diabetes mellitus aged ≥18 years who were screened for diabetic retinopathy (DR) at Pamela Youde Nethersole Eastern Hospital in February 2021 were retrospectively reviewed. Two fundus photographs (one macula-centered and one optic nerve head-centered) were taken from each eye and saved electronically. The photographs were independently graded by endocrinologists, and then by the EyeArt system and by two ophthalmologists specializing in vitreoretinal diseases (the gold standard). mtmDR is defined as moderate nonproliferative DR, severe nonproliferative DR, or proliferative DR, with or without the presence of clinically significant macular edema. Performances of EyeArt and the endocrinologists in diagnosing mtmDR were determined, as were factors associated with EyeArt’s diagnostic accuracy and the variability between EyeArt and endocrinologists.
Results: In total, 107 men and 57 women (328 eyes) were included in the analysis. Diagnostic performances of EyeArt and endocrinologists were comparable in terms of sensitivity (88.0% vs 81.5%), specificity (90.9% vs 93.1%), positive predictive value (46.8% vs 53.7%), and negative predictive value (98.8% vs 98.1%). Gwet’s first-order agreement coefficient was 0.832 (p<0.001), demonstrating almost perfect agreement between EyeArt and the endocrinologists. Inaccurate diagnosis by EyeArt was associated with the presence of hypertensive retinopathy (χ2=17.44, p<0.001).
Conclusion: EyeArt demonstrated high sensitivity and specificity in diagnosing mtmDR among patients with diabetes, with almost perfect agreement with the endocrinologists. EyeArt can be used to screen for DR.
References
1. International Diabetes Federation. IDF Diabetes Atlas 2025. Accessed 25 June 2024. Available from: https://diabetesatlas.org/resources/idf-diabetes-atlas-2025/
2. American Diabetes Association Professional Practice Committee. 16. Diabetes care in the hospital: standards of care in diabetes-2024. Diabetes Care 2024;47(Suppl 1):S295-S306.
3. Teo ZL, Tham YC, Yu M, et al. Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis. Ophthalmology 2021;128:1580-91.
4. The Diabetic Retinopathy Barometer Report Global Findings. Accessed 25 June 2024. Available from: https://www.iapb.org/wp-content/uploads/DR-Global-Report-1.pdf
5. Li KKW, Wong L, McGhee SM, et al. Diabetic retinopathy screening for specialist care. Hong Kong Med J 2019;25(Suppl 9):S4-7.
6. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 2018;1:39.
7. Ipp E, Liljenquist D, Bode B, et al. Pivotal evaluation of an artificial intelligence system for autonomous detection of referrable and vision-threatening diabetic retinopathy. JAMA Netw Open 2021;4:e2134254.
8. Bernhardt M, Jones C, Glocker B. Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms. Nat Med 2022;28:1157-8.
9. Burlina P, Joshi N, Paul W, Pacheco KD, Bressler NM. Addressing artificial intelligence bias in retinal diagnostics. Transl Vis Sci Technol 2021;10:13.
10. Au SCL, Shum GTH, Chong SSY, Ko CKL. Accuracy of the EyeArt artificial intelligence system in detecting referable diabetic retinopathy in patients with diabetes in Hong Kong. Hong Kong J Ophthalmol 2023;27:54-9.
11. Wilkinson CP, Ferris FL 3rd, Klein RE, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology 2003;110:1677-82.
12. National Eye Institute Strategic Plan. Vision for the Future 2021-2025. Accessed 25 June 2024. Available from: https://www.nei.nih.gov/sites/default/files/2021-12/NEI-StrategicPlan-VisionForTheFuture_508_edit.pdf
13. NHS Diabetic Eye Screening Programme: grading definitions for referable disease. Accessed 25 June 2024. Available from: https://www.gov.uk/government/publications/diabetic-eye-screening-retinal-image-grading-criteria/nhs-diabetic-eye-screening-programme-grading-definitions-for-referable-disease-start-date-october-01
14. Wong TY, Sun J, Kawasaki R, et al. Guidelines on diabetic eye care: the International Council of Ophthalmology recommendations for screening, follow-up, referral, and treatment based on resource settings. Ophthalmology 2018;125:1608-22.
15. EyeArt. Diabetic Retinopathy Screening Report. Accessed 25 June 2024. Available from: https://www.eyenuk.com/wp-content/uploads/2019/03/EyeArt_OUS_combined_20190327.pdf
16. Ohno-Matsui K, Kawasaki R, Jonas JB, et al. International photographic classification and grading system for myopic maculopathy. Am J Ophthalmol 2015;159:877-83.e7.
17. Haynes A, Lenz A, Stalder O, Limacher A. presize: an R-package for precision-based sample size calculation in clinical research. J Open Source Softw 2021;6:3118.
18. Lian JX, Gangwani RA, McGhee SM, et al. Systematic screening for diabetic retinopathy (DR) in Hong Kong: prevalence of DR and visual impairment among diabetic population. Br J Ophthalmol 2016;100:151-5.
19. Agresti A, Coull BA. Approximate is better than ‘exact’ for interval estimation of binomial proportions. Am Stat 1998;52:119-26.
20. Pembury Smith MQR, Ruxton GD. Effective use of the McNemar test. Behav Ecol Sociobiol 2020;74:133.
21. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
22. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005;85:257-68.
23. Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008;61:29-48.
24. Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL. A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol 2013;13:61.
25. Konstantinidis M, Le LW, Gao X. An empirical comparative assessment of inter-rater agreement of binary outcomes and multiple raters. Symmetry 2022;14:262.
26. Xie Q. Agree or disagree? A demonstration of an alternative statistic to Cohen’s Kappa for measuring the extent and reliability of agreement between observers. Accessed 25 June 2024. Available from: https://nces.ed.gov/FCSM/pdf/J4_Xie_2013FCSM.pdf
27. Trajman A, Luiz RR. McNemar chi2 test revisited: comparing sensitivity and specificity of diagnostic examinations. Scand J Clin Lab Invest 2008;68:77-80.
28. Li S, Zhao R, Zou H. Artificial intelligence for diabetic retinopathy. Chin Med J (Engl) 2021;135:253-60.
29. Heydon P, Egan C, Bolter L, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol 2021;105:723-8.
30. Automated grading in the Diabetic Eye Screening Programme. External review against programme appraisal criteria for the UK National Screening Committee. Accessed 25 June 2024. Available from: https://assets.publishing.service.gov.uk/media/619e80088fa8f5037e8ccb2f/Evidence_summary_AI_in_DESP_2021.pdf
31. Liu J, Gibson E, Ramchal S, et al. Diabetic retinopathy screening with automated retinal image analysis in a primary care setting improves adherence to ophthalmic care. Ophthalmol Retina 2021;5:71-7.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Hong Kong Journal of Ophthalmology
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Journal has a fully Open Access policy and publishes all articles under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. For any use other than that permitted by this license, written permission must be obtained from the Journal.
