A mobile-optimized artificial intelligence system for gestational age and fetal malpresentation assessment | Communications Medicine - Nature.com

Despite sizeable advancement successful maternal healthcare successful caller decades, maternal and perinatal deaths stay precocious with 295,000 maternal deaths during and pursuing gestation and 2.4 cardinal neonatal deaths each year. The bulk of these deaths hap successful low-to-middle-income countries (LMICs)^1,2,3. The deficiency of antenatal attraction and constricted entree to facilities that tin supply lifesaving attraction for the mother, fetus and newborn lend to inequities successful prime of attraction and outcomes successful these regions^4,5.

Obstetric ultrasound is an important constituent of prime antenatal care. The WHO recommends 1 regular aboriginal ultrasound scan for each large women, but up to 50% of women successful processing countries person nary ultrasound screening during pregnancy⁶. Fetal ultrasounds tin beryllium utilized to estimation gestational property (GA), which is captious successful scheduling and readying for screening tests passim gestation and interventions for gestation complications specified arsenic preeclampsia and preterm labor. Fetal ultrasounds aboriginal successful gestation tin besides beryllium utilized to diagnose fetal malpresentation, which affects up to 3–4% of pregnancies astatine word and is associated with trauma-related wounded during birth, perinatal mortality, and maternal morbidity^7,8,9,10,11.

Though ultrasound devices person traditionally been costly, the caller commercialized availability of low-cost, battery-powered handheld devices could greatly grow access^12,13,14. However, existent ultrasound grooming programs necessitate months of supervised valuation arsenic good arsenic indefinite continuing acquisition visits for prime assurance^{13,14,15,16,17,18,19}. GA estimation and diagnosis of fetal malpresentation necessitate adept mentation of anatomical imagery during the ultrasound acquisition process. GA estimation via objective modular biometry²⁰ requires expertly locating fetal anatomical structures and manually measuring their carnal sizes successful precisely collected images (head circumference, abdominal circumference, femur length, among others). To code these barriers, anterior studies person introduced a protocol wherever fetal ultrasounds tin beryllium acquired by minimally trained operators via a “blind sweep” protocol, consisting of six predefined freehand sweeps implicit the abdomen^{21,22,23,24,25,26,27}. While blind-sweep protocols simplify the ultrasound acquisition process, caller methods are required for interpreting the resulting imagery. AI-based mentation whitethorn supply a promising absorption for generating automated objective estimates from blind-sweep video sequences.

In this study, we utilized 2 prospectively collected fetal ultrasound datasets to estimation gestational property and fetal malpresentation portion demonstrating cardinal considerations for usage by novice users successful LMICs: (a) validating that it is imaginable to physique blind-sweep GA and fetal malpresentation models that tally successful real-time connected mobile devices; (b) evaluating generalization of these models to minimally trained ultrasound operators and low-cost ultrasound devices; (c) describing a modified 2-sweep blind-sweep protocol to simplify novice acquisition; (d) adding feedback scores to supply real-time accusation connected expanse quality.

Blind-sweep procedure

Blind-sweep ultrasounds consisted of a fixed fig of predefined freehand ultrasound sweeps implicit the gravid abdomen. Certified sonographers completed up to 15 sweeps. Novice operators (“novices”), with 8 h of blind-sweep ultrasound acquisition training, completed six sweeps. Evaluation of some sonographers and novices was constricted to a acceptable of six sweeps—three vertical and 3 horizontal sweeps (Fig. 1b).

Fig. 1: Development of an artificial quality strategy to get and construe blind-sweep ultrasound for antenatal diagnostics.

a Datasets were curated from sites successful Zambia and the USA and see ultrasound acquired by sonographers and midwives. Ground information for gestational property was derived from the archetypal exam arsenic portion of objective practice. An artificial quality (AI) strategy was trained to place gestational property and fetal malpresentation and was evaluated by comparing the accuracy of AI predictions with the accuracy of objective modular procedures. The AI strategy was developed utilizing lone sonographer blind-sweep data, and its generalization to novice users was tested connected midwife data. Design of the AI strategy considered suitability for deployment successful low-to-middle-income countries successful 3 ways: first, the strategy interpreted ultrasound from low-cost portable ultrasound devices; second, adjacent real-time mentation is disposable offline connected mobile telephone devices; and finally, the AI strategy produces feedback scores that tin beryllium utilized to supply feedback to users. b Blind-sweep ultrasound acquisition procedure. The process tin beryllium performed by novices with a fewer hours of ultrasound training. While the implicit protocol involves six sweeps, a acceptable of 2 sweeps (M and R) were recovered to beryllium capable for maintaining the accuracy of gestational property estimation.

Fetal property instrumentality learning inaugural (FAMLI) and novice idiosyncratic survey datasets

Data were analyzed from the Fetal Age Machine Learning Initiative cohort, which collected ultrasound information from survey sites astatine Chapel Hill, NC (USA), and the Novice User Study collected from Lusaka, Zambia (Fig. 1a)²⁷. The extremity of this prospectively collected dataset was to alteration the improvement of exertion to estimation gestational age²⁸. Data postulation occurred betwixt September 2018 and June 2021. All survey participants provided written informed consent, and the probe was approved by the UNC organization reappraisal committee (IRB #18-1848) and the biomedical probe morals committee astatine the University of Zambia. Blind-sweep information were collected with modular ultrasound devices (SonoSite M-Turbo oregon GE Voluson) arsenic good arsenic a low-cost portable ultrasound instrumentality (ButterflyIQ). Studies included modular objective assessments of GA²⁰ and fetal malpresentation performed by a trained sonographer utilizing a modular ultrasound device.

Algorithm development

We developed 2 heavy learning neural web models to foretell GA and fetal malpresentation. Our models generated diagnostic predictions straight from ultrasound video: sequences of representation pixel values were the input and an estimation of the objective quantity of involvement was the output. The GA exemplary produced an estimation of age, measured successful days, for each blind-sweep video sequence. The GA exemplary additionally provided an estimation of its assurance successful the estimation for a fixed video sequence. No intermediate fetal biometric measurements were required during grooming oregon generated during inference. The fetal malpresentation exemplary predicted a probability people betwixt 0.0 and 1.0 for whether the fetus is successful noncephalic presentation. See Supplementary Materials for a method treatment and details regarding exemplary development.

In the USA, the crushed information GA was determined for each subordinate based connected the “best obstetric estimate,” arsenic portion of regular objective care, utilizing procedures recommended by the American College of Obstetricians and Gynecologists (ACOG)²⁹. The champion obstetric estimation combines accusation from the past menstrual play (LMP), GA derived from assisted reproductive exertion (if applicable), and fetal ultrasound anatomic measurements. In Zambia, lone the archetypal fetal ultrasound was utilized to find the crushed information GA arsenic the LMP successful this mounting was considered little reliable arsenic patients often presented for attraction aboriginal successful pregnancy.

The GA exemplary was trained connected sonographer-acquired unsighted sweeps (up to 15 sweeps per patient) arsenic good arsenic sonographer-acquired “fly-to” videos that seizure 5 to 10 seconds earlier the sonographer has acquired modular fetal biometry images. The fetal malpresentation exemplary was lone trained connected unsighted sweeps. For each grooming acceptable case, fetal malpresentation was specified arsenic 1 of 4 imaginable values by a sonographer (cephalic, breech, transverse, oblique), and dichotomized to “cephalic” vs “noncephalic”. This dichotomization is clinically justified since cephalic cases are considered mean portion each noncephalic cases necessitate further aesculapian attention.

Our investigation cohort included each large women successful the FAMLI and Novice User Study datasets who had the indispensable crushed information accusation for gestational property and fetal presumption from September 2018 to January 2021. Study participants were assigned astatine random to 1 of 3 dataset splits: train, tune, oregon test. We utilized the pursuing proportions: 60% train/20% tune/20% trial for survey participants who did not person novice sweeps, and 10% tune/90% trial for participants who received novice sweeps. The tuning acceptable was utilized for optimizing instrumentality learning grooming hyperparameters and selecting a classification threshold probability for the fetal malpresentation model. This threshold was chosen to output adjacent noncephalic specificity and sensitivity connected the tuning set, blinded to the trial sets. None of the blind-sweep information collected by the novices were utilized for training.

Cases consisted of aggregate blind-sweep videos, and our models generated predictions independently for each video series wrong the case. For the GA model, each unsighted expanse was divided into aggregate video sequences. For the fetal malpresentation model, video sequences corresponded to a azygous implicit unsighted sweep. We past aggregated the predictions to make a azygous case-level estimation for either GA oregon fetal malpresentation (described further successful the Mobile Device Inference conception successful supplementary materials).

Evaluation

The valuation was performed connected the FAMLI (sonographer-acquired) and Novice User Study (novice-acquired) datasets. Test sets consisted of patients autarkic of those utilized for AI improvement (Fig. 1a). For our GA exemplary evaluation, the superior FAMLI trial acceptable comprised 407 women successful 657 survey visits successful the USA. A 2nd trial set, “Novice User Study” included 114 participants successful 140 survey visits successful Zambia. Novice blind-sweep studies were exclusively performed astatine Zambian sites. Sweeps collected with modular ultrasound devices were disposable for 406 of 407 participants successful the sonographer-acquired trial set, and 112 of 114 participants successful the novice-acquired trial set. Sweeps collected with the low-cost instrumentality were disposable for 104 of 407 participants successful the sonographer-acquired trial set, and 56 of 114 participants successful the novice-acquired trial set. Analyzable information from the low-cost instrumentality became disposable aboriginal during the study, and this radical of patients is typical of the afloat diligent set. We randomly selected 1 survey sojourn per diligent for each investigation radical to debar combining correlated measurements from the aforesaid patient. For our fetal malpresentation model, the trial acceptable included 613 patients from the sonographer-acquired and novice-acquired datasets, resulting successful 65 instances of noncephalic presumption (10.6%). For each patient, the past survey sojourn of the 3rd trimester was included. Of note, determination are much patients successful the malpresentation exemplary trial acceptable since the crushed information is not babelike connected a anterior visit. The disposition of survey participants are summarized successful STARD diagrams (Supplementary Fig. 1) and Supplementary Table 1.

Reporting summary

Further accusation connected probe plan is disposable successful the Nature Research Reporting Summary linked to this article.

Mobile-device-optimized AI gestational property and fetal malpresentation estimation

We calculated the mean quality successful implicit mistake betwixt the GA exemplary estimation and estimated gestational property arsenic determined by modular fetal biometry measurements utilizing imaging from accepted ultrasound devices operated by sonographers²⁰. The notation crushed information GA was established based connected an archetypal diligent sojourn arsenic described supra successful Methods. When conducting pairwise statistical comparisons betwixt unsighted expanse and modular fetal biometry implicit errors, we established an a priori criterion for non-inferiority which was confirmed if the blind-sweep mean implicit mistake (MAE) was little than 1.0 time greater than the modular fetal biometry’s MAE. Statistical estimates and comparisons were computed aft randomly selecting 1 survey sojourn per diligent for each investigation group, to debar combining correlated measurements from the aforesaid patient.

We conducted a supplemental investigation of GA exemplary prediction mistake with mixed effects regression connected each trial data, combining sonographer-acquired and novice-acquired trial sets. Fixed effect presumption accounted for the crushed information GA, the benignant of ultrasound instrumentality utilized (standard vs. debased cost), and the grooming level of the ultrasound relation (sonographer vs. novice). All diligent studies were included successful the analysis, and random effects presumption accounted for intra-patient and intra-study effects.

GA investigation results are summarized successful Table 1. The MAE for the GA exemplary estimation with unsighted sweeps collected by sonographers utilizing modular ultrasound devices was importantly little than the MAE for the modular fetal biometry estimates (mean quality −1.4 ± 4.5 days, 95% CI −1.8, −0.9 days). There was a inclination toward expanding mistake for a unsighted expanse and modular fetal biometry procedures with the gestational week (Fig. 2a).

Fig. 2: Gestational property estimation.

n = 407 survey participants, unsighted sweeps performed by adept sonographers. a Blind-sweep process and modular fetal biometry process implicit mistake versus crushed information gestational property (4-week windows). Box indicates 25th, 50th, and 75th percentile implicit error, and whiskers bespeak 5th and 95th percentile implicit error. b Error distributions for blind-sweep process and modular fetal biometry procedure. c Paired errors for a unsighted expanse and modular fetal biometry estimates successful the aforesaid survey visit. The errors of the 2 methods grounds correlation, but the worst-case errors for the blind-sweep process person a little magnitude than the modular fetal biometry method. d Video series feedback-score calibration connected the trial sets. The realized exemplary estimation mistake connected held-out video sequences decreases arsenic the model’s feedback people increases. A thresholded feedback people whitethorn beryllium utilized arsenic a idiosyncratic feedback awesome to redo low-quality unsighted sweeps. Box indicates 25th, 50th, and 75th percentile of implicit errors, and whiskers bespeak the 5th and 95th percentile implicit error.

The accuracy of the fetal malpresentation exemplary for predicting noncephalic fetal presumption from third-trimester unsighted sweeps was assessed utilizing a notation modular determined by sonographers equipped with accepted ultrasound imagery (described above). We selected the latest survey sojourn successful the 3rd trimester for each patient. Data from sweeps performed by the sonographers and novices were analyzed separately. We evaluated the fetal malpresentation model’s country nether the receiver operating curve (AUC-ROC) connected the trial acceptable successful summation to noncephalic sensitivity and specificity.

The fetal malpresentation exemplary attained an AUC-ROC of 0.977 (95% CI 0.949, 1.00), sensitivity of 0.938 (95% CI 0.848, 0.983), and specificity of 0.973 (95% CI 0.955, 0.985) (Table 2 and Fig. 3).

Fig. 3: Fetal malpresentation estimation.

n = 623 survey participants. Receiver operating diagnostic (ROC) curves for fetal malpresentation estimation. Crosses bespeak the predefined operating constituent selected from the tuning dataset. a ROC examination based connected the benignant of device: low-cost and standard. b ROC examination based connected the benignant of ultrasound operator: novices and sonographers.

Generalization of GA and malpresentation estimation to novices

Our models were trained connected up to 15 unsighted sweeps per survey performed by sonographers. No novice-acquired unsighted sweeps were utilized to bid our models. We assessed GA exemplary generalization to unsighted sweeps performed by novice operators that performed six sweeps. We compared the MAE betwixt novice-performed blind-sweep AI estimates and the modular fetal biometry. For the malpresentation model, we reported the AUC-ROC for unsighted sweeps performed by novices, on with the sensitivity and specificity astatine the aforesaid operating constituent utilized for evaluating unsighted sweeps performed by sonographers.

In this novice-acquired dataset, the quality successful MAE betwixt blind-sweep AI estimates and the modular fetal biometry was −0.6 days (95% CI −1.7, 0.5), indicating that sweeps performed by novice operators supply a non-inferior GA estimation compared to the modular fetal biometry. Table 1 provides novice blind-sweep show analyzed by ultrasound instrumentality type. The mixed effects regression mistake investigation did not bespeak a important relation betwixt GA mistake magnitude and the benignant of relation conducting the unsighted expanse (P = 0.119).

Fetal malpresentation utilizing novice-acquired unsighted sweeps was compared to the sonographer’s determination connected 189 participants (21 malpresentations), and AUC-ROC was 0.992 (95% CI 0.983, 1.0). On the preselected operating point, sensitivity was 1.0 (95% CI 0.839, 1.0) and specificity was 0.952 (95% CI 0.908, 0.979).

Performance of low-cost ultrasound instrumentality successful GA and fetal malpresentation estimation

GA exemplary estimation utilizing unsighted sweeps acquired with the low-cost ultrasound instrumentality were compared against the objective modular connected the combined novice-acquired and sonographer-acquired trial sets. We utilized the aforesaid a priori criterion for non-inferiority arsenic described above, 1.0 day. For the malpresentation model, we reported AUC-ROC by benignant of ultrasound instrumentality on with sensitivity and specificity astatine the aforesaid operating constituent discussed above.

GA exemplary estimation utilizing unsighted sweeps acquired with the low-cost ultrasound instrumentality were compared against the modular fetal biometry estimates connected the combined novice-acquired and sonographer-acquired trial sets. The blind-sweep AI strategy had MAE of 3.98 ± 3.54 days versus 4.17 ± 3.74 days for modular fetal biometry (mean quality −0.21 ± 4.21, 95% CI −0.87, 0.44) which meets the criterion for non-inferiority.

Paired GA estimates for unsighted sweeps acquired with some a modular ultrasound instrumentality and the low-cost instrumentality were disposable for immoderate survey participants successful the combined trial acceptable (N = 155 participants). The MAE quality betwixt unsighted sweeps performed with the low-cost and modular devices was 0.45 days (95% CI, 0.0, 0.9). The mixed effects regression showed that usage of the low-cost instrumentality was associated with accrued mistake magnitude (P = 0.001), though the estimated effect was lone 0.67 days.

Fetal malpresentation estimation utilizing unsighted sweeps acquired with the low-cost ultrasound instrumentality were compared against the sonographer’s determination connected the combined novice-acquired and sonographer-acquired trial sets (213 participants, 29 malpresentations). The blind-sweep AI strategy had AUC-ROC of 0.97 (95% CI 0.944, 0.997). At the preselected operating point, sensitivity was 0.931 (95% CI 0.772, 0.992) and specificity was 0.94 (95% CI 0.896, 0.970).

Simplified expanse evaluation

Protocols consisting of less sweeps than the modular 6 sweeps (Fig. 1b) whitethorn simplify objective deployment. We selected M and R expanse types arsenic the champion performing acceptable of 2 sweeps connected the tuning acceptable and evaluated this reduced protocol connected the trial sets.

On trial acceptable sweeps performed by sonographers, the reduced protocol of conscionable the M and R expanse types (Fig. 1b) was capable for maintaining the non-inferiority of the blind-sweep protocol comparative to the modular fetal biometry estimates (MAE quality 95% CI: [−1.5, −0.69] days). The reduced protocol was capable for maintaining non-inferiority of unsighted sweeps comparative to modular fetal biometry connected trial acceptable examinations performed by novices (MAE quality 95% CI: [−1.19, 0.67] days). On average, the reduced protocol tin beryllium completed successful 20.1 seconds, arsenic extrapolated from videos collected from novices (see Supplementary Table 2). MAE crossed subgroups utilizing the reduced protocol are provided successful Table 1 (last row).

Feedback-score evaluation

Our GA exemplary provided a feedback people to measure the suitability of a video series for GA estimation. The GA exemplary computed the feedback people for 24-frame video sequences (about 1 2nd successful length) and truthful provided a semi-continuous feedback awesome crossed the duration of a emblematic 10-s agelong unsighted sweep. The feedback people took the signifier of an inverse variance estimation and tin beryllium utilized to value and aggregate GA predictions crossed blind-sweep video sequences during a survey visit. All GA results were computed utilizing this inverse variance weighting aggregation method. More details are provided successful “Methods”.

As expected, video sequences with precocious feedback people had debased MAE erstwhile compared against crushed information GA, and debased feedback-score video sequences had precocious MAE compared against crushed information GA. Figure 2d indicates the calibration of the feedback people connected the held-out trial datasets. Supplementary Fig. 2c shows illustration blind-sweep video frames from precocious and debased feedback-score video sequences. The feedback people qualitatively aligns with the grade to which the fetus is disposable successful the video clip, with the precocious feedback people near and center-left examples showing the fetal abdomen and caput (respectively). In contrast, the fetus is not disposable successful the debased feedback-score examples (center-right and right).

Run-time valuation connected mobile phones

Our blind-sweep AI models were designed to alteration adjacent real-time inference connected modern mobile phones oregon modular computers, to alteration elimination of waiting clip during the objective introspection procedure. We measured some GA and fetal malpresentation exemplary run-time show utilizing 10-s agelong blind-sweep videos, which were chosen to lucifer the mean magnitude of novice unsighted sweeps (Supplementary Table 2). Videos were streamed to an Android trial exertion moving connected Google Pixel 3 and 4, Samsung Galaxy S10, and Xiaomi Mi 9 phones (examples of Android phones that tin beryllium purchased refurbished for little than $250 USD). Videos were processed by the GA and fetal malpresentation models simultaneously, with some models executed successful the aforesaid TensorFlow Lite run-time environment. All indispensable representation preprocessing operations were besides included successful the benchmark.

Our results indicated that combined diagnoses for some models are disposable betwixt 0.2 and 1.0 s connected mean aft the completion of a unsighted expanse connected devices with a graphics processing portion (GPU), and betwixt 1.5 and 2.5 s connected mean aft completion connected devices with neural web acceleration libraries for modular CPU processors. See Table 3 for implicit benchmark results.

In this study, we demonstrated however our end-to-end blind-sweep mobile-device-optimized AI models tin assistance novices successful LMICs successful acquiring blind-sweep ultrasounds to estimation 2 important obstetric measurements: GA and fetal malpresentation. While determination person been aggregate GA models projected successful the past, ours is the archetypal to picture an end-to-end strategy focusing connected usage successful LMIC settings. Three anterior studies person described utilizing heavy learning connected azygous video frames to either straight estimation GA oregon estimation caput circumference that is past utilized successful fetal biometry formulas^23,24,25,27. One anterior survey describes utilizing the FAMLI dataset to estimation GA done heavy learning of blind-sweep ultrasound videos²⁷. One anterior survey describes estimation of fetal malpresentation utilizing AI to archetypal observe fetal anatomy, followed by applying a objective decision-making matrix²⁶. Our models execute arsenic good oregon amended than antecedently described models. Our GA exemplary estimation was non-inferior to modular fetal biometry estimates and our fetal malpresentation exemplary had precocious sensitivity and specificity. Both models besides had akin show crossed sonographer and novice-acquired ultrasounds.

We recovered that aboriginal successful pregnancy, determination was little deterioration of GA exemplary estimation accuracy compared to the objective modular fetal biometry. Our models utilize the full ultrasound video arsenic opposed to lone accounting for isolated biometric measurements (e.g., caput circumference, femur length..) per the modular fetal biometry. This holistic attack whitethorn relationship for the accrued accuracy aboriginal successful the pregnancy, erstwhile the correlation betwixt GA and carnal size of the fetus is little pronounced. This whitethorn beryllium particularly adjuvant successful providing much close estimated GA successful LMIC settings wherever entree to ultrasound successful aboriginal gestation is rare³⁰.

While pairwise comparisons betwixt accepted devices and low-cost devices suggest that accepted devices whitethorn execute somewhat better, GA exemplary estimation from low-cost devices was non-inferior to modular fetal biometry estimation. This suggests that saltation successful instrumentality show does not effect successful clinically important differences successful GA exemplary performance. For our fetal malpresentation model, show was akin betwixt low-cost and accepted devices.

We focused connected improving idiosyncratic acquisition and simplifying ultrasound acquisition since immoderate of the astir susceptible patients are successful geographically distant areas without Internet access. While we initially evaluated connected unsighted sweeps consisting of six sweeps, we recovered that our GA exemplary performed likewise utilizing lone 2 sweeps. The compatibility of the GA exemplary with this simplified 2-sweep protocol suggests that we whitethorn beryllium capable to simplify acquisition complexity for novices. Our GA exemplary generates a real-time feedback people that provides accusation connected ultrasound video prime and reliability for usage successful our AI models. In a objective setting, this feedback people tin perchance notify the ultrasound relation to redo a poorly performed sweep. Both the GA and malpresentation models on with the video prime feedback people person been optimized to tally connected affordable mobile devices and bash not necessitate Internet access.

One regulation of this survey is the tiny illustration size, which makes it hard to measure each subgroup individually: novices, sonographers, and ultrasound instrumentality type. Our dataset included precise fewer videos for GA little than 12 weeks and greater than 37 weeks truthful we cannot guarantee the AI models generalize for these groups. In addition, we lone had a constricted fig of noncephalic presentations resulting successful wide assurance intervals. We program to validate our findings connected a larger cohort to code these limitations. These aboriginal studies volition besides see predictions for different captious maternal fetal diagnostics and gestation hazard stratification.

While designing the AI models, we addressed obstacles that whitethorn beryllium encountered successful low-resourced settings wherever distant attraction is often delivered done novices with constricted training. Overall, tools specified arsenic the ones assessed successful this survey tin perchance assistance successful upleveling the capabilities of some installation and novices successful providing much precocious antenatal care. Additionally, the underlying techniques and exertion could beryllium applied and studied successful different ultrasound-based objective workflows. The prospective objective valuation volition beryllium important to measure real-world effectiveness, and adaptations whitethorn beryllium needed to integrate tools specified arsenic this into real-world workflows.