Automated quality assessment of individual embryologists performing ICSI using deep learning-enabled fertilization and embryo grading technology


      Data analysis is a crucial part of an effective in-vitro fertilization (IVF) quality assessment (QA) program. Routine review of identified key performance indicators (KPIs) are important to ensure proper laboratory functioning and, perhaps more importantly, to identify potential problems to permit timely corrections. Fertilization assessment is the primary outcome used to measure embryology staff proficiency with Intracytoplasmic sperm injection (ICSI). However, tracking the developmental fate of ICSI derived embryos may provide a more complete picture of how well this procedure is being performed. Current quality assessments require manual examination and recording of fertilization status and embryo developmental scores. These processes are labor-intensive and highly subjective in nature. Convolutional neural networks (CNNs) have been used to assess fertilization and blastocyst development with accuracies rivaling highly trained embryologists. The objective of this study was to compare quality assessments of staff performing ICSI using standard manual grading measurements and fully automated AI-trained image measurements.


      Retrospective cohort study using a deep convolutional neural network 1. The deep neural network (AI) was trained to evaluate embryos at day 1 and day 5 post insemination.

      Materials and Methods

      In our study, performances of 7 individual embryologists were calculated through manually analyzing the developmental outcome rates with the outcome rates measured automatically by an AI system through morphological analyses. The AI system developed to evaluate fertilization and blastocyst development was utilized for this work. We compared the rates of fertilization, blastocyst development, and high-quality blastocyst (HQB) development in a total of 947 embryos that were divided between the 7 embryologists. To evaluate the difference between the two analysis methods, we performed a Wilcoxon matched-pairs signed rank test and a coefficient of variation (%CV) analysis.


      The Wilcoxon tests revealed that the two approaches performed with negligible differences (P>0.05) for all three rate estimations (Fertilization, Blastocysts, and HQB). The medians of difference for estimations of fertilization, blastocysts, and HQB were -1.3% (P>0.31), 1.8% (P>0.09), and -3.6% (P>0.18), respectively. The %CV estimations also showed that the difference between manual and AI-generated estimations for each embryologist in all three rates was low. The median of %CV between the two approaches in measuring the rates of fertilization, blastocysts, and HQB were 1.9%, 3.4%, and 10.9%, respectively.


      This study is the first to describe the use of artificial intelligence to monitor individual embryologists performing ICSI in a clinical setting. The extremely low coefficient of variation between the manual and AI-based QA assessment methods demonstrate the high accuracy of the automated AI system. This study demonstrates an alternative method for monitoring KPIs in the IVF laboratory without the need for manual assistance.


      1. I. Dimitriadis, C. L. Bormann, P. Thirumalaraju, M. Kanakasabapathy, R. Gupta, R. Pooniwala, I. Souter, J. Y. Hsu, S. T. Rice, P. Bhowmick and H. Shafiee, Fertility and sterility, 2019, 111, e21.