Accuracy of the EyeArt artificial intelligence system in detecting referable diabetic retinopathy in patients with diabetes in Hong Kong
Keywords:
Artificial intelligence; , Diabetic retinopathy, Macular edemaAbstract
Objective: This study evaluated the real-world performance in an Asian population with diabetes mellitus (DM) of the EyeArt system, which is the only United States Food and Drug Administration–approved system available in Hong Kong.
Methods: Fundus photographs taken from December 2020 to June 2021 during a diabetic retinopathy (DR) screening program in patients with DM who attended Pamela Youde Nethersole Eastern Hospital in Hong Kong were retrieved. Two vitreoretinal-trained ophthalmologists independently graded the fundus photographs for DR and concomitant pathologies. Discrepancies were resolved through discussion with a senior vitreoretinal ophthalmologist for a final decision. Fundus photographs (in jpg format) were then uploaded to the EyeArt AI system. The EyeArt system’s sensitivity and specificity on detecting (1) referable DR (moderate or severe non-proliferative DR, proliferative DR, or diabetic macular edema [DME]) and (2) any DR or DME were calculated based on the ophthalmologists’ diagnoses (the gold standard).
Results: 23 men and 79 women (93 Chinese, 3 Pakistanis, 2 Thais, and 4 Indonesians) aged 25 to 77 years with type1 (n=8) or type 2 (n=94) DM were included for analysis. The mean duration of DM was 4.9 years; 56 (54.9%)
patients were newly diagnosed with DM. 33 (32.4%) patients were taking ≥1 (up to 4) oral hypoglycemic agents; none were taking ≥5. 16 (15.7%) patients required both oral hypoglycemic agents and insulin for DM control; 31 (30.4%) patients injected insulin every day. As diagnosed by the ophthalmologists, 41 (20.1%) of 204 eyes had DR and/or DME, whereas 26 (25.5%) of 102 patients had referable DR. There was no discrepancy between the two ophthalmologists’ gradings. On detecting referable DR, the EyeArt system had 96.2% sensitivity, 94.7% specificity, 86.2% positive predictive value, 98.6% negative predictive value, and 95.1% accuracy. On detecting any DR or DME, the EyeArt system had 92.7% sensitivity, 93.7% specificity, 79.2% positive predictive value, 98.0% negative predictive value, and 93.5% accuracy.
Conclusion: The EyeArt system is a reliable AI tool for DR screening in patients with DM in Hong Kong. Its usage in primary care and public healthcare programs should be considered.
References
Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017;69S:S36-S40.
Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. Artificial Intelligence in Healthcare 2020:25-60.
Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J 2019;6:94-8.
Fasterholdt I, Naghavi-Behzad M, Rasmussen BSB, et al. Value assessment of artificial intelligence in medical imaging: a scoping review. BMC Med Imaging 2022;22:187.
Wong MCS, Huang J, Kong APS. Diabetes screening revisited: issues related to implementation. Hong Kong Med J 2020;26:283-5.
Sinclair SH, Schwartz SS. Diabetic retinopathy: an underdiagnosed and undertreated inflammatory, neuro-vascular complication of diabetes. Front Endocrinol (Lausanne) 2019;10:843.
Gangwani RA, Lian JX, McGhee SM, Wong D, Li KK. Diabetic retinopathy screening: global and local perspective. Hong Kong Med J 2016;22:486-95.
Abramoff MD, Niemeijer M, Russell SR. Automated detection of diabetic retinopathy: barriers to translation into clinical practice. Expert Rev Med Devices 2010;7:287-96.
Grzybowski A, Brona P, Lim G, et al. Artificial intelligence for diabetic retinopathy screening: a review. Eye (Lond) 2020;34:451-60.
Gunasekeran DV, Ting DSW, Tan GSW, Wong TY. Artificial intelligence for diabetic retinopathy screening, prediction and management. Curr Opin Ophthalmol 2020;31:357-65.
Ipp E, Liljenquist D, Bode B, et al. Pivotal evaluation of an artificial intelligence system for autonomous detection of referrable and vision-threatening diabetic retinopathy. JAMA Netw Open 2021;4:e2134254.
Lim JI, Regillo CD, Sadda SR, et al. Artificial intelligence detection of diabetic retinopathy: subgroup comparison of the EyeArt System with ophthalmologists' dilated examinations. Ophthalmol Sci 2022;3:100228.
Mak CY, Yam JC, Chen LJ, Lee SM, Young AL. Epidemiology of myopia and prevention of myopia progression in children in East Asia: a review. Hong Kong Med J 2018;24:602-9.
Wong TY, Sun J, Kawasaki R, et al. Guidelines on Diabetic Eye Care: The International Council of Ophthalmology Recommendations for Screening, Follow-up, Referral, and Treatment Based on Resource Settings. Ophthalmology 2018;125:1608-22.
Negida A, Fahim NK, Negida Y. Sample size calculation guide - Part 4: How to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, and the area under the ROC curve. Adv J Emerg Med 2019;3:e33.
Fung MM, Yap MK, Cheng KK. Community-based diabetic retinopathy screening in Hong Kong: ocular findings. Clin Exp Optom 2011;94:63-6.
Fu MSS, Lai LKP, Chan P, Chow K, Luk MMH, Chao DVK. The prevalence and associated factors of diabetic retinopathy in Chinese hypertensive patients newly diagnosed with type 2 diabetes mellitus: a cross sectional study in 3 primary care clinics in Hong Kong. Hong Kong Pract 2017;39:67-76.
Zhang Z, Zhang W, Xu Y, Liu D. Efficacy of hyperbaric oxygen therapy for diabetic foot ulcers: an updated systematic review and meta-analysis. Asian J Surg 2022;45:68-78.
Raumviboonsuk P, Krause J, Chotcomwongse P, et al. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. NPJ Digit Med 2019;2:25.
Sun J, Lin Q, Zhao P, et al. Reducing waiting time and raising outpatient satisfaction in a Chinese public tertiary general hospital: an interrupted time series study. BMC Public Health 2017;17:668.
Islam FMA. Accuracy and reliability of retinal photo grading for diabetic retinopathy: remote graders from a developing country and standard retinal photo grader in Australia. PLoS One 2017;12:e0179310.
Guo H, Zhao Y, Niu T, Tsui KL. Hong Kong Hospital Authority resource efficiency evaluation: via a novel DEA-Malmquist model and Tobit regression model. PLoS One 2017;12:e0184211.
Niki O, Saira G, Arvind S, Mike D. Cyber-attacks are a permanent and substantial threat to health systems: education must reflect that. Digit Health 2022;8:20552076221104665.
Kiener M. Artificial intelligence in medicine and the disclosure of risks. AI Soc 2021;36:705-13.
Radanliev P, De Roure D. Advancing the cybersecurity of the healthcare system with self-optimising and self-adaptative artificial intelligence (part 2). Health Technol (Berl) 2022;12:923-9.
Wong IYH, Wong RLM, Chan JCH, Kawasaki R, Chong V. Incorporating optical coherence tomography macula scans enhances cost-effectiveness of fundus photography-based screening for diabetic macular edema. Diabetes Care 2020;43:2959-66.
Wong RL, Tsang CW, Wong DS, et al. Are we making good use of our public resources? The false-positive rate of screening by fundus photography for diabetic macular oedema. Hong Kong Med J 2017;23:356-64.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Hong Kong Journal of Ophthalmology
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Journal has a fully Open Access policy and publishes all articles under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. For any use other than that permitted by this license, written permission must be obtained from the Journal.
