Artificial intelligence and external photographs in ophthalmology: a systematic review
Keywords:
Artificial intelligence, Face, OphthalmologyAbstract
We systematically reviewed the literature regarding the use of artificial intelligence trained with external photographs, defined as unprocessed clinical images taken from cameras and slit-lamps, for measurement of eyelid/periorbital parameters and detection and classification of multiple ophthalmic diseases including blepharoptosis, thyroid eye disease, eyelid tumors, keratitis, trachoma, pterygium, diabetic retinopathy, cataract, strabismus, and other oculofacial disorders.
References
1. Shen Y, Xiao T, Yi S, Chen D, Wang X, Li H. Person re-identification with deep Kronecker-product matching and group-shuffling random walk. IEEE Trans Pattern Anal Mach Intell 2021;43:1649-65.
2. Huang Z, Yu Y, Xu J, Ni F, Le X. PF-Net: point fractal network for 3D point cloud completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020: 7662-70.
3. Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 2017;124:962-9.
4. Yousefi S, Elze T, Pasquale LR, et al. Monitoring glaucomatous functional loss using an artificial intelligence-enabled dashboard. Ophthalmology 2020;127:1170-8.
5. Schmidt-Erfurth U, Vogl WD, Jampol LM, Bogunović H. Application of automated quantification of fluid volumes to anti-VEGF therapy of neovascular age-related macular degeneration. Ophthalmology 2020;127:1211-9.
6. Du R, Xie S, Fang Y, et al. Deep learning approach for automated detection of myopic maculopathy and pathologic myopia in fundus images. Ophthalmol Retina 2021;5:1235-44.
7. Roomaney I, Nyirenda C, Chetty M. Facial imaging to screen for fetal alcohol spectrum disorder: a scoping review. Alcohol Clin Exp Res 2022;46:1166-80.
8. Dias R, Torkamani A. Artificial intelligence in clinical and genomic diagnostics. Genome Med 2019;11:70.
9. Qin B, Liang L, Wu J, Quan Q, Wang Z, Li D. Automatic identification of Down syndrome using facial images with deep convolutional neural network. Diagnostics (Basel) 2020;10:487.
10. Mukhiddinov M, Djuraev O, Akhmedov F, Mukhamadiyev A, Cho J. Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors (Basel) 2023;23:1080.
11. Debnath T, Reza MM, Rahman A, Beheshti A, Band SS, Alinejad-Rokny H. Four-layer ConvNet to facial emotion recognition with minimal epochs and the significance of data diversity. Sci Rep 2022;12:6991.
12. Kizilgul M, Karakis R, Dogan N, et al. Real-time detection of acromegaly from facial images with artificial intelligence. Eur J Endocrinol 2023;188:lvad005.
13. Goetz RK, Hughes FE, Duignan ES, et al. A template for reducing ophthalmology outpatient waiting times: community ophthalmic care. Ir J Med Sci 2018;187:237-41.
14. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
15. Starck WJ, Griffin JE Jr, Epker BN. Objective evaluation of the eyelids and eyebrows after blepharoplasty. J Oral Maxillofac Surg 1996;54:297-303.
16. Moriyama T, Kanade T, Xiao J, Cohn JF. Meticulously detailed eye region model and its application to analysis of facial images. IEEE Trans Pattern Anal Mach Intell 2006;28:738-52.
17. Thomas PBM, Gunasekera CD, Kang S, Baltrusaitis T. An artificial intelligence approach to the assessment of abnormal lid position. Plast Reconstr Surg Glob Open 2020;8:e3089.
18. Van Brummen A, Owen JP, Spaide T, et al. PeriorbitAI: artificial intelligence automation of eyelid and periorbital measurements. Am J Ophthalmol 2021;230:285-96.
19. Chen HC, Tzeng SS, Hsiao YC, Chen RF, Hung EC, Lee OK. Smartphone-based artificial intelligence-assisted prediction for eyelid measurements: algorithm development and observational validation study. JMIR Mhealth Uhealth 2021;9:e32444.
20. Cao J, Lou L, You K, et al. A novel automatic morphologic analysis of eyelids based on deep learning methods. Curr Eye Res 2021;46:1495-502.
21. Zhu X, Song X, Min X, et al. Calculation of ophthalmic diagnostic parameters on a single eye image based on deep neural network. Multimed Tools Appl 2022;81:2311-31.
22. Hung JY, Perera C, Chen KW, et al. A deep learning approach to identify blepharoptosis by convolutional neural networks. Int J Med Inform 2021;148:104402.
23. Tabuchi H, Nagasato D, Masumoto H, et al. Developing an iOS application that uses machine learning for the automated diagnosis of blepharoptosis. Graefes Arch Clin Exp Ophthalmol 2022;260:1329-35.
24. Hung JY, Chen KW, Perera C, et al. An outperforming artificial intelligence model to identify referable blepharoptosis for general practitioners. J Pers Med 2022;12:283.
25. Lou L, Cao J, Wang Y, et al. Deep learning-based image analysis for automated measurement of eyelid morphology before and after blepharoptosis surgery. Ann Med 2021;53:2278-85.
26. Bahçeci Şimşek İ, Şirolu C. Analysis of surgical outcome after upper eyelid surgery by computer vision algorithm using face and facial landmark detection. Graefes Arch Clin Exp Ophthalmol 2021;259:3119-25.
27. Qu Y, Lin B, Li S, et al. Effect of multichannel convolutional neural network-based model on the repair and aesthetic effect of eye plastic surgery patients. Comput Math Methods Med 2022;2022:5315146.
28. Sun Y, Huang X, Zhang Q, et al. A fully automatic postoperative appearance prediction system for blepharoptosis surgery with image-based deep learning. Ophthalmol Sci 2022;2:100169.
29. Bahn RS, Heufelder AE. Pathogenesis of Graves' ophthalmopathy. N Engl J Med 1993;329:1468-75.
30. Bartalena L, Kahaly GJ, Baldeschi L, et al. The 2021 European Group on Graves' orbitopathy (EUGOGO) clinical practice guidelines for the medical management of Graves' orbitopathy. Eur J Endocrinol 2021;185:G43-G67.
31. Yoo TK, Choi JY, Kim HK. A generative adversarial network approach to predicting postoperative appearance after orbital decompression surgery for thyroid eye disease. Comput Biol Med 2020;118:103628.
32. Moon JH, Shin K, Lee GM, et al. Machine learning-assisted system using digital facial images to predict the clinical activity score in thyroid-associated orbitopathy. Sci Rep 2022;12:22085.
33. Karlin J, Gai L, LaPierre N, et al. Ensemble neural network model for detecting thyroid eye disease using external photographs. Br J Ophthalmol 2023;107:1722-9.
34. Huang X, Ju L, Li J, et al. An intelligent diagnostic system for thyroid-associated ophthalmopathy based on facial images. Front Med (Lausanne) 2022;9:920716.
35. Shao J, Huang X, Gao T, et al. Deep learning-based image analysis of eyelid morphology in thyroid-associated ophthalmopathy. Quant Imaging Med Surg 2023;13:1592-604.
36. Deprez M, Uffer S. Clinicopathological features of eyelid skin tumors. A retrospective study of 5504 cases and review of literature. Am J Dermatopathol 2009;31:256-62.
37. Adamopoulos A, Chatzopoulos EG, Anastassopoulos G, Detorakis E. Eyelid basal cell carcinoma classification using shallow and deep learning artificial neural networks. Evol Syst (Berl) 2021;12:583-90.
38. Li Z, Qiang W, Chen H, et al. Artificial intelligence to detect malignant eyelid tumors from photographic images. NPJ Digit Med 2022;5:23.
39. Hui S, Dong L, Zhang K, Nie Z. Noninvasive identification of benign and malignant eyelid tumors using clinical images via deep learning system. J Big Data 2022;9:84.
40. Lee MJ, Yang MK, Khwarg SI, et al. Differentiating malignant and benign eyelid lesions using deep learning. Sci Rep 2023;13:4103.
41. Austin A, Lietman T, Rose-Nussbaumer J. Update on the management of infectious keratitis. Ophthalmology 2017;124:1678-89.
42. Xu Y, Kong M, Xie W, et al. Deep sequential feature learning in clinical image classification of infectious keratitis. Engineering 2021;7:1002-10.
43. Ji Q, Jiang Y, Qu L, Yang Q, Zhang H. An image diagnosis algorithm for keratitis based on deep learning. Neural Process Lett 2022;54:2007-24.
44. Kuo MT, Hsu BW, Lin YS, et al. Comparisons of deep learning algorithms for diagnosing bacterial keratitis via external eye photographs. Sci Rep 2021;11:24227.
45. Natarajan R, Matai HD, Raman S, et al. Advances in the diagnosis of herpes simplex stromal necrotising keratitis: a feasibility study on deep learning approach. Indian J Ophthalmol 2022;70:3279-83.
46. Hung N, Shih AK, Lin C, et al. Using slit-lamp images for deep learning-based identification of bacterial and fungal keratitis: model development and validation with different convolutional neural networks. Diagnostics (Basel) 2021;11:1246.
47. Redd TK, Prajna NV, Srinivasan M, et al. Image-based differentiation of bacterial and fungal keratitis using deep convolutional neural networks. Ophthalmol Sci 2022;2:100119.
48. Zhang Z, Wang H, Wang S, et al. Deep learning-based classification of infectious keratitis on slit-lamp images. Ther Adv Chronic Dis 2022;13:20406223221136071.
49. Hu S, Sun Y, Li J, et al. Automatic diagnosis of infectious keratitis based on slit lamp images analysis. J Pers Med 2023;13:519.
50. Kogachi K, Lalitha P, Prajna NV, et al. Deep convolutional neural networks detect no morphological differences between culture-positive and culture-negative infectious keratitis images. Transl Vis Sci Technol 2023;12:12.
51. Loo J, Kriegel MF, Tuohy MM, et al. Open-source automatic segmentation of ocular structures and biomarkers of microbial keratitis on slit-lamp photography images using deep learning. IEEE J Biomed Health Inform 2021;25:88-99.
52. Loo J, Woodward MA, Prajna V, et al. Open-source automatic biomarker measurement on slit-lamp photography to estimate visual acuity in microbial keratitis. Transl Vis Sci Technol 2021;10:2.
53. Burton MJ, Mabey DC. The global burden of trachoma: a review. PLoS Negl Trop Dis 2009;3:e460.
54. World Health Organization. Trachoma control. A guide for programme managers. Accessed 19 February 2024. Available from: http://apps.who.int/iris/bitstream/10665/43405/1/9241546905_eng.pdf.
55. Kim MC, Okada K, Ryner AM, et al. Sensitivity and specificity of computer vision classification of eyelid photographs for programmatic trachoma assessment. PLoS One 2019;14:e0210463.
56. Socia D, Brady CJ, West SK, Cockrell RC. Detection of trachoma using machine learning approaches. PLoS Negl Trop Dis 2022;16:e0010943.
57. Chu WK, Choi HL, Bhat AK, Jhanji V. Pterygium: new insights. Eye (Lond) 2020;34:1047-50.
58. Nejima R, Masuda A, Minami K, Mori Y, Hasegawa Y, Miyata K. Topographic changes after excision surgery of primary pterygia and the effect of pterygium size on topographic restoration. Eye Contact Lens 2015;41:58-63.
59. Aidenloo NS, Motarjemizadeh Q, Heidarpanah M. Risk factors for pterygium recurrence after limbal-conjunctival autografting: a retrospective, single-centre investigation. Jpn J Ophthalmol 2018;62:349-56.
60. López YP, Aguilera LR. Automatic classification of pterygium-non pterygium images using deep learning. In: VipIMAGE 2019: 391-400.
61. Zulkifley MA, Abdani SR, Zulkifley NH. Pterygium-Net: a deep learning approach to pterygium detection and localization. Multimed Tools Appl 2019;78:34563-84.
62. Zhu S, Fang X, Qian Y, et al. Pterygium screening and lesion area segmentation based on deep learning. J Healthc Eng 2022;2022:3942110.
63. Wan C, Shao Y, Wang C, Jing J, Yang W. A novel system for measuring pterygium’s progress using deep learning. Front Med (Lausanne) 2022;9:819971.
64. Fang X, Deshmukh M, Chee ML, et al. Deep learning algorithms for automatic detection of pterygium using anterior segment photographs from slit-lamp and hand-held cameras. Br J Ophthalmol 2022;106:1642-7.
65. Hung KH, Lin C, Roan J, et al. Application of a deep learning system in pterygium grading and further prediction of recurrence with slit lamp photographs. Diagnostics (Basel) 2022;12:888.
66. Jais FN, Che Azemin MZ, Hilmi MR, Mohd Tamrin MI, Kamal KM. Postsurgery classification of best-corrected visual acuity changes based on pterygium characteristics using the machine learning technique. ScientificWorldJournal 2021;2021:6211006.
67. Liu Y, Xu C, Wang S, et al. Accurate detection and grading of pterygium through smartphone by a fusion training model. Br J Ophthalmol 2023;108:336-42.
68. Babenko B, Mitani A, Traynis I, et al. Detection of signs of disease in external photographs of the eyes via deep learning. Nat Biomed Eng 2022;6:1370-83.
69. Parikh R, Palmer V, Kumar A, Simon JW. Surgical confusions in ophthalmology: description, analysis, and prevention of errors from 2006 through 2017. Ophthalmology 2020;127:296-302.
70. Kiuchi G, Tanabe M, Nagata K, Ishitobi N, Tabuchi H, Oshika T. Deep learning-based system for preoperative safety management in cataract surgery. J Clin Med 2022;11:5397.
71. Huang X, Lee SJ, Kim CZ, Choi SH. An automatic screening method for strabismus detection based on image processing. PLoS One 2021;16:e0255643.
72. Schulz CB, Clarke H, Makuloluwe S, Thomas PB, Kang S. Automated extraction of clinical measures from videos of oculofacial disorders using machine learning: feasibility, validity and reliability. Eye (Lond) 2023;37:2810-6.
73. Greene JJ, Tavares J, Guarin DL, Hadlock T. Clinician and automated assessments of facial function following eyelid weight placement. JAMA Facial Plast Surg 2019;21:387-92.
74. Bellemo V, Lim ZW, Lim G, et al. Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. Lancet Digit Health 2019;1:e35-e44.
75. Dong L, Yang Q, Zhang RH, Wei WB. Artificial intelligence for the detection of age-related macular degeneration in color fundus photographs: a systematic review and meta-analysis. EClinicalMedicine 2021;35:100875.
76. Scruggs BA, Chan RVP, Kalpathy-Cramer J, Chiang MF, Campbell JP. Artificial intelligence in retinopathy of prematurity diagnosis. Transl Vis Sci Technol 2020;9:5.
77. Bojikian KD, Lee CS, Lee AY. Finding glaucoma in color fundus photographs using deep learning. JAMA Ophthalmol 2019;137:1361-2.
78. Zheng C, Johnson TV, Garg A, Boland MV. Artificial intelligence in glaucoma. Curr Opin Ophthalmol 2019;30:97-103.
79. Lim NC, Sundar G, Amrith S, Lee KO. Thyroid eye disease: a Southeast Asian experience. Br J Ophthalmol 2015;99:512-8.
80. He M, Li Z, Liu C, Shi D, Tan Z. Deployment of artificial intelligence in real-world practice: opportunity and challenge. Asia Pac J Ophthalmol (Phila) 2020;9:299-307.
81. Lee JG, Jun S, Cho YW, et al. Deep learning in medical imaging: general overview. Korean J Radiol 2017;18:570-84.
82. He J, Baxter SL, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019;25:30-6.
83. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019;380:1347-58.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Hong Kong Journal of Ophthalmology
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Journal has a fully Open Access policy and publishes all articles under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. For any use other than that permitted by this license, written permission must be obtained from the Journal.