Artificial intelligence and external photographs in ophthalmology: a systematic review

Authors

  • Kenneth Ka Hei Lai Tung Wah Eastern Hospital
  • Carmen Sze Ching Lo
  • Han Wang
  • Xiaoyan Hu
  • Fatema Aljufairi
  • Jake Sebastian
  • Chi Pui Pang
  • Kelvin Kam Lung Chong

Keywords:

Artificial intelligence, Face, Ophthalmology

Abstract

We systematically reviewed the literature regarding the use of artificial intelligence trained with external photographs, defined as unprocessed clinical images taken from cameras and slit-lamps, for measurement of eyelid/periorbital parameters and detection and classification of multiple ophthalmic diseases including blepharoptosis, thyroid eye disease, eyelid tumors, keratitis, trachoma, pterygium, diabetic retinopathy, cataract, strabismus, and other oculofacial disorders.

 

References

‎1‎. Shen Y, Xiao T, Yi S, Chen D, Wang X, Li H. ‎Person re-identification with deep Kronecker-‎product matching and group-shuffling random ‎walk. IEEE Trans Pattern Anal Mach Intell ‎‎2021;43:1649-65.‎

‎2‎. Huang Z, Yu Y, Xu J, Ni F, Le X. PF-Net: point fractal network ‎for 3D point cloud ‎completion. In: Proceedings of the IEEE/CVF Conference on Computer ‎Vision and Pattern ‎Recognition; 2020: 7662-70.‎

‎3‎. Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep ‎learning. ‎Ophthalmology 2017;124:962-9.‎

‎4‎. Yousefi S, Elze T, Pasquale LR, et al. Monitoring glaucomatous functional loss using an ‎artificial ‎intelligence-enabled dashboard. Ophthalmology 2020;127:1170-8.‎

‎5‎. Schmidt-Erfurth U, Vogl WD, Jampol LM, Bogunović H. Application of automated ‎‎quantification of fluid volumes to anti-VEGF therapy of neovascular age-related ‎macular ‎degeneration. Ophthalmology 2020;127:1211-9.‎

‎6‎. Du R, Xie S, Fang Y, et al. Deep learning ‎approach for automated detection of myopic ‎maculopathy and pathologic myopia in ‎fundus images. Ophthalmol Retina 2021;5:1235-44.‎

‎7‎. Roomaney I, Nyirenda C, Chetty M. Facial imaging to screen for fetal alcohol spectrum ‎‎disorder: a scoping review. Alcohol Clin Exp Res 2022;46:1166-80.‎

‎8‎. Dias R, Torkamani A. Artificial intelligence in clinical and genomic diagnostics. Genome ‎‎Med 2019;11:70.‎

‎9‎. Qin B, Liang L, Wu J, Quan Q, Wang Z, Li D. Automatic identification of Down syndrome ‎‎using facial images with deep convolutional neural network. Diagnostics (Basel) 2020;10:487.‎

‎10‎. Mukhiddinov M, Djuraev O, Akhmedov F, Mukhamadiyev A, Cho J. Masked face ‎emotion ‎recognition based on facial landmarks and deep learning approaches for visually ‎‎impaired people. Sensors (Basel) 2023;23:1080.‎

‎11‎. Debnath T, Reza MM, Rahman A, Beheshti A, Band SS, Alinejad-Rokny H. Four-layer ‎‎ConvNet to facial emotion recognition with minimal epochs and the significance of data ‎‎diversity. Sci Rep 2022;12:6991.‎

‎12‎. Kizilgul M, Karakis R, Dogan N, et al. Real-time detection of acromegaly from facial ‎images with artificial ‎intelligence. Eur J Endocrinol 2023;188:lvad005.‎

‎13‎. Goetz RK, Hughes FE, Duignan ES, et al. A ‎template for reducing ophthalmology ‎outpatient waiting times: community ophthalmic care. ‎Ir J Med Sci 2018;187:237-41.‎

‎14‎. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated ‎‎guideline for reporting systematic reviews. BMJ 2021;372:n71.‎

‎15‎. Starck WJ, Griffin JE Jr, Epker BN. Objective evaluation of the eyelids and eyebrows after ‎‎blepharoplasty. J Oral Maxillofac Surg 1996;54:297-303.‎

‎16‎. Moriyama T, Kanade T, Xiao J, Cohn JF. Meticulously detailed eye region model and its ‎application to analysis of ‎facial images. IEEE Trans Pattern Anal Mach Intell 2006;28:738-52.‎

‎17‎. Thomas PBM, Gunasekera CD, Kang S, Baltrusaitis T. An artificial intelligence approach ‎‎to the assessment of abnormal lid position. Plast Reconstr Surg Glob Open 2020;8:e3089.‎

‎18‎. Van Brummen A, Owen JP, Spaide T, et al. PeriorbitAI: artificial intelligence automation of ‎eyelid and periorbital ‎measurements. Am J Ophthalmol 2021;230:285-96.‎

‎19‎. Chen HC, Tzeng SS, Hsiao YC, Chen RF, Hung EC, Lee OK. Smartphone-based artificial ‎‎intelligence-assisted prediction for eyelid measurements: algorithm development and ‎‎observational validation study. JMIR Mhealth Uhealth 2021;9:e32444.‎

‎20‎. Cao J, Lou L, You K, et al. A novel automatic morphologic analysis ‎of eyelids based on ‎deep learning methods. Curr Eye Res 2021;46:1495-502.‎

‎21‎. Zhu X, Song X, Min X, et al. Calculation of ophthalmic diagnostic parameters on a single ‎eye image ‎based on deep neural network. Multimed Tools Appl 2022;81:2311-31.‎

‎22‎. Hung JY, Perera C, Chen KW, et al. ‎A deep learning approach to identify blepharoptosis by ‎convolutional neural networks. Int J ‎Med Inform 2021;148:104402.‎

‎23‎. Tabuchi H, Nagasato D, Masumoto H, et al. ‎Developing an iOS application that uses ‎machine learning for the automated diagnosis of ‎blepharoptosis. Graefes Arch Clin Exp ‎Ophthalmol 2022;260:1329-35.‎

‎24‎. Hung JY, Chen KW, Perera C, et al. An outperforming artificial intelligence model to ‎identify referable ‎blepharoptosis for general practitioners. J Pers Med 2022;12:283.‎

‎25‎. Lou L, Cao J, Wang Y, et al. Deep learning-based ‎image analysis for automated ‎measurement of eyelid morphology before and after ‎blepharoptosis surgery. Ann Med ‎‎2021;53:2278-85.‎

‎26‎. Bahçeci Şimşek İ, Şirolu C. Analysis of surgical outcome after upper eyelid surgery by ‎‎computer vision algorithm using face and facial landmark detection. Graefes Arch Clin Exp ‎‎Ophthalmol 2021;259:3119-25.‎

‎27‎. Qu Y, Lin B, Li S, et al. Effect of multichannel convolutional neural network-based model ‎on the ‎repair and aesthetic effect of eye plastic surgery patients. Comput Math Methods Med ‎‎‎2022;2022:5315146.‎

‎28‎. Sun Y, Huang X, Zhang Q, et al. A fully automatic ‎postoperative appearance prediction ‎system for blepharoptosis surgery with image-based ‎deep learning. Ophthalmol Sci ‎‎2022;2:100169.‎

‎29‎. Bahn RS, Heufelder AE. Pathogenesis of Graves' ophthalmopathy. N Engl J Med ‎‎1993;329:‎‎1468-75.‎

‎30‎. Bartalena L, Kahaly GJ, Baldeschi L, et al. The 2021 European Group on Graves' ‎orbitopathy ‎‎(EUGOGO) clinical practice guidelines for the medical management of Graves' ‎orbitopathy. ‎Eur J Endocrinol 2021;185:G43-G67.‎

‎31‎. Yoo TK, Choi JY, Kim HK. A generative adversarial network approach to predicting ‎‎postoperative appearance after orbital decompression surgery for thyroid eye disease. ‎Comput ‎Biol Med 2020;118:103628.‎

‎32‎. Moon JH, Shin K, Lee GM, et al. Machine learning-assisted ‎system using digital facial ‎images to predict the clinical activity score in thyroid-associated ‎orbitopathy. Sci Rep ‎‎2022;12:22085.‎

‎33‎. Karlin J, Gai L, LaPierre N, et al. Ensemble neural network model for detecting thyroid eye ‎disease ‎using external photographs. Br J Ophthalmol 2023;107:1722-9.‎

‎34‎. Huang X, Ju L, Li J, et al. An intelligent diagnostic ‎system for thyroid-associated ‎ophthalmopathy based on facial images. Front Med ‎‎(Lausanne) 2022;9:920716.‎

‎35‎. Shao J, Huang X, Gao T, et al. Deep learning-based image ‎analysis of eyelid morphology ‎in thyroid-associated ophthalmopathy. Quant Imaging Med ‎Surg 2023;13:1592-604.‎

‎36‎. Deprez M, Uffer S. Clinicopathological features of eyelid skin tumors. A retrospective ‎study ‎of 5504 cases and review of literature. Am J Dermatopathol 2009;31:256-62.‎

‎37‎. Adamopoulos A, Chatzopoulos EG, Anastassopoulos G, Detorakis E. Eyelid basal cell ‎carcinoma classification using shallow and deep ‎learning artificial neural networks. Evol Syst ‎‎(Berl) 2021;12:583-90.‎

‎38‎. Li Z, Qiang W, Chen H, et al. ‎Artificial intelligence to detect malignant eyelid tumors from ‎photographic images. NPJ ‎Digit Med 2022;5:23.‎

‎39‎. Hui S, Dong L, Zhang K, Nie Z. Noninvasive identification of benign and malignant eyelid ‎tumors using ‎clinical images via deep learning system. J Big Data 2022;9:84.‎

‎40‎. Lee MJ, Yang MK, Khwarg SI, et al. Differentiating malignant and benign eyelid lesions ‎using deep learning. Sci ‎Rep 2023;13:4103.‎

‎41‎. Austin A, Lietman T, Rose-Nussbaumer J. Update on the management of infectious ‎‎keratitis. Ophthalmology 2017;124:1678-89.‎

‎42‎. Xu Y, Kong M, Xie W, et al. Deep sequential feature learning in clinical image ‎classification of ‎infectious keratitis. Engineering 2021;7:1002-10.‎

‎43‎. Ji Q, Jiang Y, Qu L, Yang Q, Zhang H. An image diagnosis algorithm for keratitis based on ‎deep learning. Neural ‎Process Lett 2022;54:2007-24.‎

‎44‎. Kuo MT, Hsu BW, Lin YS, et al. Comparisons of deep learning algorithms for diagnosing ‎bacterial keratitis ‎via external eye photographs. Sci Rep 2021;11:24227.‎

‎45‎. Natarajan R, Matai HD, Raman S, et al. Advances in the diagnosis of herpes simplex ‎stromal necrotising ‎keratitis: a feasibility study on deep learning approach. Indian J Ophthalmol ‎‎2022;70:‎‎3279-83.‎

‎46‎. Hung N, Shih AK, Lin C, et al. Using slit-lamp images for deep learning-based ‎identification of ‎bacterial and fungal keratitis: model development and validation with different ‎‎convolutional neural networks. Diagnostics (Basel) 2021;11:1246.‎

‎47‎. Redd TK, Prajna NV, Srinivasan M, et al. Image-based differentiation of bacterial and ‎fungal keratitis using ‎deep convolutional neural networks. Ophthalmol Sci 2022;2:100119.‎

‎48‎. Zhang Z, Wang H, Wang S, et al. Deep learning-based classification of infectious keratitis ‎on slit-lamp ‎images. Ther Adv Chronic Dis 2022;13:20406223221136071.‎

‎49‎. Hu S, Sun Y, Li J, et al. Automatic diagnosis of infectious keratitis based on slit lamp ‎images ‎analysis. J Pers Med 2023;13:519.‎

‎50‎. Kogachi K, Lalitha P, Prajna NV, et al. Deep convolutional neural networks detect no ‎morphological ‎differences between culture-positive and culture-negative infectious keratitis ‎images. ‎Transl Vis Sci Technol 2023;12:12.‎

‎51‎. Loo J, Kriegel MF, Tuohy MM, et al. Open-source automatic segmentation of ocular ‎structures and biomarkers of ‎microbial keratitis on slit-lamp photography images using deep ‎learning. IEEE J Biomed ‎Health Inform 2021;25:88-99.‎

‎52‎. Loo J, Woodward MA, Prajna V, et al. Open-source automatic biomarker measurement on ‎slit-lamp photography ‎to estimate visual acuity in microbial keratitis. Transl Vis Sci Technol ‎‎2021;10:2.‎

‎53‎. Burton MJ, Mabey DC. The global burden of trachoma: a review. ‎PLoS Negl Trop Dis ‎‎2009;3:e460.‎

‎54‎. World Health Organization. Trachoma control. A guide for programme managers. Accessed ‎‎19 February 2024. Available from: ‎http://apps.who.int/iris/bitstream/10665/43405/1/9241546905_eng.pdf.‎

‎55‎. Kim MC, Okada K, Ryner AM, et al. Sensitivity and specificity of computer vision ‎classification of eyelid ‎photographs for programmatic trachoma assessment. PLoS One ‎‎2019;14:e0210463.‎

‎56‎. Socia D, Brady CJ, West SK, Cockrell RC. Detection of trachoma using machine learning ‎approaches. PLoS Negl ‎Trop Dis 2022;16:e0010943.‎

‎57‎. Chu WK, Choi HL, Bhat AK, Jhanji V. Pterygium: new insights. Eye (Lond) ‎‎2020;34:1047-50.‎

‎58‎. Nejima R, Masuda A, Minami K, Mori Y, Hasegawa Y, Miyata K. Topographic changes ‎after excision surgery of ‎primary pterygia and the effect of pterygium size on topographic ‎restoration. Eye Contact ‎Lens 2015;41:58-63.‎

‎59‎. Aidenloo NS, Motarjemizadeh Q, Heidarpanah M. Risk factors for pterygium recurrence ‎‎after limbal-conjunctival autografting: a retrospective, single-centre investigation. Jpn J ‎‎Ophthalmol 2018;62:349-56.‎

‎60‎. López YP, Aguilera LR. Automatic classification of pterygium-non pterygium images using ‎‎deep learning. In: VipIMAGE 2019: 391-400.‎

‎61‎. Zulkifley MA, Abdani SR, Zulkifley NH. Pterygium-Net: a deep learning approach to ‎‎pterygium detection and localization. Multimed Tools Appl ‎‎2019;78:34563-84.‎

‎62‎. Zhu S, Fang X, Qian Y, et al. Pterygium screening and lesion area ‎segmentation based on ‎deep learning. J Healthc Eng 2022;2022:3942110.‎

‎63‎. Wan C, Shao Y, Wang C, Jing J, Yang W. A novel system for measuring ‎pterygium’s ‎progress using deep learning. Front Med (Lausanne) 2022;9:819971.‎

‎64‎. Fang X, Deshmukh M, Chee ML, et al. Deep learning algorithms ‎for automatic detection of ‎pterygium using anterior segment photographs from slit-lamp and ‎hand-held cameras. Br J ‎Ophthalmol 2022;106:1642-7.‎

‎65‎. Hung KH, Lin C, Roan J, et al. Application of a deep learning ‎system in pterygium grading ‎and further prediction of recurrence with slit lamp ‎photographs. Diagnostics (Basel) ‎‎2022;12:888.‎

‎66‎. Jais FN, Che Azemin MZ, Hilmi MR, Mohd Tamrin MI, Kamal KM. Postsurgery ‎‎classification of best-corrected visual acuity changes based on pterygium characteristics ‎using ‎the machine learning technique. ScientificWorldJournal 2021;2021:6211006.‎

‎67‎. Liu Y, Xu C, Wang S, et al. Accurate ‎detection and grading of pterygium through ‎smartphone by a fusion training model. Br J Ophthalmol 2023;108:336-42.‎

‎68‎. Babenko B, Mitani A, Traynis I, et al. Detection of signs of disease in external photographs ‎of the eyes via ‎deep learning. Nat Biomed Eng 2022;6:1370-83.‎

‎69‎. Parikh R, Palmer V, Kumar A, Simon JW. Surgical confusions in ophthalmology: ‎description, analysis, and prevention of errors from 2006 through 2017. Ophthalmology ‎‎‎2020;127:296-302.‎

‎70‎. Kiuchi G, Tanabe M, Nagata K, Ishitobi N, Tabuchi H, Oshika T. Deep learning-based ‎system for preoperative safety management in ‎cataract surgery. J Clin Med 2022;11:5397.‎

‎71‎. Huang X, Lee SJ, Kim CZ, Choi SH. An automatic screening method for strabismus ‎‎detection based on image processing. PLoS One 2021;16:e0255643.‎

‎72‎. Schulz CB, Clarke H, Makuloluwe S, Thomas PB, Kang S. Automated extraction of ‎clinical ‎measures from videos of oculofacial disorders using machine learning: feasibility, ‎validity ‎and reliability. Eye (Lond) 2023;37:2810-6.‎

‎73‎. Greene JJ, Tavares J, Guarin DL, Hadlock T. Clinician and automated assessments of ‎facial ‎function following eyelid weight placement. JAMA Facial Plast Surg 2019;21:387-92.‎

‎74‎. Bellemo V, Lim ZW, Lim G, et al. Artificial intelligence using deep learning to screen for ‎‎referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. ‎‎Lancet Digit Health 2019;1:e35-e44.‎

‎75‎. Dong L, Yang Q, Zhang RH, Wei WB. Artificial intelligence for the detection of age-‎related ‎macular degeneration in color fundus photographs: a systematic review and meta-‎analysis. ‎EClinicalMedicine 2021;35:100875.‎

‎76‎. Scruggs BA, Chan RVP, Kalpathy-Cramer J, Chiang MF, Campbell JP. Artificial ‎‎intelligence in retinopathy of prematurity diagnosis. Transl Vis Sci Technol 2020;9:5.‎

‎77‎. Bojikian KD, Lee CS, Lee AY. Finding glaucoma in color fundus ‎photographs using deep ‎learning. JAMA Ophthalmol 2019;137:1361-2.‎

‎78‎. Zheng C, Johnson TV, Garg A, Boland MV. Artificial intelligence in glaucoma. Curr Opin ‎‎Ophthalmol 2019;30:97-103.‎

‎79‎. Lim NC, Sundar G, Amrith S, Lee KO. Thyroid eye disease: a Southeast Asian experience. ‎‎Br J Ophthalmol 2015;99:512-8.‎

‎80‎. He M, Li Z, Liu C, Shi D, Tan Z. Deployment of artificial intelligence in real-world ‎‎practice: opportunity and challenge. Asia Pac J Ophthalmol (Phila) 2020;9:299-307.‎

‎81‎. Lee JG, Jun S, Cho YW, et al. Deep learning in medical imaging: general overview. ‎‎Korean J Radiol 2017;18:570-84.‎

‎82‎. He J, Baxter SL, Xu J, Zhou X, Zhang K. The practical implementation of artificial ‎intelligence ‎technologies in medicine. Nat Med 2019;25:30-6.‎

‎83‎. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med ‎‎‎2019;380:1347-58.‎

Downloads

Published

2024-08-20

How to Cite

1.
Lai KKH, Lo CSC, Wang H, Hu X, Aljufairi F, Sebastian J, Pang CP, Chong KKL. Artificial intelligence and external photographs in ophthalmology: a systematic review. Hong Kong J Ophthalmol [Internet]. 2024Aug.20 [cited 2024Nov.11];28(1). Available from: https://hkjo.hk/index.php/hkjo/article/view/384

Issue

Section

Review Articles