BACKGROUND:

Despite the standardization of care, formula feeding varied across sites of the Ohio Perinatal Quality Collaborative (OPQC). We used orchestrated testing (OT) to learn from this variation and improve nonpharmacologic care of infants with neonatal abstinence syndrome (NAS) requiring pharmacologic treatment in Ohio.

METHODS:

To test the impact of formula on length of stay (LOS), treatment failure, and weight loss among infants hospitalized with NAS, we compared caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design. During October 2015 to June 2016, OPQC sites joined 1 of 4 OT groups. We used response plots to examine the effect of each factor and control charts to track formula use and LOS. We used the OT results to revise the nonpharmacologic bundle and implemented it during 2017.

RESULTS:

Forty-seven sites caring for 546 NAS infants self-selected into the 4 OT groups. Response plots revealed the benefit of high-calorie formula (HCF) on weight loss, treatment failure, and LOS. The nonpharmacologic treatment bundle was updated to recommend HCF when breastfeeding was not possible. During implementation, HCF use increased, and LOS decreased from 17.1 to 16.4 days across the OPQC.

CONCLUSIONS:

OT revealed that HCF was associated with shorter LOS in OPQC sites. Implementation of a revised nonpharmacologic care bundle was followed by additional LOS improvement in Ohio. Despite some challenges in the implementation of OT, our findings support its usefulness for learning in improvement networks.

The incidence of neonatal abstinence syndrome (NAS), the withdrawal symptoms experienced by infants exposed to opiates in utero, has increased dramatically in Ohio and nationwide.1,2 

The Ohio Perinatal Quality Collaborative (OPQC) conducted a multifaceted quality improvement (QI) initiative to standardize treatment and improve outcomes among Ohio infants with NAS from January 2014 to June 2015, resulting in a decrease in length of stay (LOS) from 18.3 to 17 days for those infants requiring pharmacologic treatment.3  One component of this initiative was the standardization of nonpharmacologic treatment. Centers were encouraged to apply a “bundle” of practices that included the promotion of maternal involvement, breastfeeding (if the mother is stable in a treatment program), swaddling, skin-to-skin care, and reduced stimulation. For infants who were not breastfeeding, use of low-lactose formula (LLF) was encouraged on the basis of pilot work4,5  and the rationale that LLF may improve gastrointestinal withdrawal symptoms. Use of 22-calorie formula was encouraged on the basis of expert opinion,68  knowledge that NAS produces a hypermetabolic state with exaggerated weight loss,810  and the rationale that high-calorie formula (HCF) may increase resilience to withdrawal symptoms.11 

Despite a lack of published data on best practices for nutritional care of infants with NAS, the OPQC aimed to standardize nonpharmacologic care, including feeding practices, because standardization, in and of itself, has been shown to improve outcomes.12,13  Throughout the initial QI effort, there was an increase in overall compliance with the nonpharmacologic bundle, but significant variation in LLF and HCF use remained, with <60% of infants receiving LLF and <20% receiving HCF. Also, a survey of OPQC teams after the initial project revealed continued uncertainty regarding the use of LLF and HCF as a part of nonpharmacologic support. This uncertainty and continued variation provided an opportunity for increased learning to determine if further standardizing formula type would be a useful change strategy to achieve the original goal of a 20% reduction in LOS.

In the second phase, the OPQC used orchestrated testing (OT) to assess the effect of HCF and LLF as part of nonpharmacologic NAS management in a coordinated, large-scale plan-do-study-act (PDSA) cycle. The aim of this phase was to reduce LOS among pharmacologically treated infants with NAS from 17 days (LOS after the first phase of work) to 14.7 days, representing a total reduction of 20% from the start of the NAS improvement work in Ohio. Our goal for this QI report is to describe the OPQC’s experience with OT and subsequent improvement work.

OT is a QI method involving coordinated wide-scale testing of change ideas. It provides a framework to learn from variation and identify best practices.1416  OT includes 2 phases: (1) coordinated testing (OT phase) and (2) replication of findings (implementation phase).14,15  During October 2015 to June 2016, we used OT to compare caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design resulting in 4 OT groups (Table 1). Each center selected 1 of the 4 OT groups to join. The OPQC physician lead at each site was encouraged to discuss the project with their entire NICU team and to select 1 of the 4 OT groups on the basis of which formula the staff felt they could use reliably. In some cases, centers selected a group that was congruent with their current practice, and in other cases, centers joined a group that required them to make changes in the typical formula used. Analysis was conducted in July 2016 and August 2016, with preliminary results presented in September 2016 and final results presented in December 2016. After the OT results were shared, we encouraged all OPQC sites to implement the revised nonpharmacologic bundle from January 2017 to December 2017 (implementation phase) and continued monitoring process and outcome measures.

TABLE 1

OT Groups

Group 1Group 2Group 3Group 4
11 Centers5 Centers20 Centers11 Centers
94 Patients74 Patients333 Patients54 Patients
OT factors     
 Designated formula     
  Low lactose Yes No Yes No 
  High calorie Yes Yes No No 
 Formula compliance, n (%) 72 (77) 38 (51) 252 (76) 30 (56) 
 Unit characteristics, n (%)     
  Level of care     
  Level 2 26 (28) 14 (19) 54 (16) 53 (98) 
  Level 3 68 (72) 60 (81) 260 (78) 1 (2) 
 Inborn 67 (71) 36 (49) 257 (77) 54 (100) 
 Mother in treatment program 62 (66) 53 (74) 213 (65) 27 (52) 
 Mother using heroin 35 (41) 27 (40) 135 (43) 20 (38) 
 Mother using benzodiazepines 4 (5) 9 (13) 44 (14) 4 (8) 
 Maternal smoking 69 (86) 45 (83) 247 (85) 39 (83) 
 Breastfeeding 31 (33) 29 (39) 109 (33) 13 (24) 
 Morphine treatment protocola 6 (55) 2 (40) 17 (85) 6 (55) 
 Methadone treatment protocola 5 (45) 3 (60) 3 (15) 5 (45) 
Group 1Group 2Group 3Group 4
11 Centers5 Centers20 Centers11 Centers
94 Patients74 Patients333 Patients54 Patients
OT factors     
 Designated formula     
  Low lactose Yes No Yes No 
  High calorie Yes Yes No No 
 Formula compliance, n (%) 72 (77) 38 (51) 252 (76) 30 (56) 
 Unit characteristics, n (%)     
  Level of care     
  Level 2 26 (28) 14 (19) 54 (16) 53 (98) 
  Level 3 68 (72) 60 (81) 260 (78) 1 (2) 
 Inborn 67 (71) 36 (49) 257 (77) 54 (100) 
 Mother in treatment program 62 (66) 53 (74) 213 (65) 27 (52) 
 Mother using heroin 35 (41) 27 (40) 135 (43) 20 (38) 
 Mother using benzodiazepines 4 (5) 9 (13) 44 (14) 4 (8) 
 Maternal smoking 69 (86) 45 (83) 247 (85) 39 (83) 
 Breastfeeding 31 (33) 29 (39) 109 (33) 13 (24) 
 Morphine treatment protocola 6 (55) 2 (40) 17 (85) 6 (55) 
 Methadone treatment protocola 5 (45) 3 (60) 3 (15) 5 (45) 
a

Number of centers, not number of patients.

The Cincinnati Children’s Hospital Institutional Review Board reviewed the OPQC NAS QI project and determined it did not meet the definition of human subjects’ research. Participating centers were encouraged to assess whether they required local institutional review board review.

Founded in 2007, OPQC is a state-based perinatal quality collaborative with a mission to reduce preterm births and improve outcomes for infants in Ohio.17  Fifty-three of 54 centers that participated in the initial OPQC NAS project participated in the OT project, and 1 new center joined.3  These centers represented all level 3 NICUs and 89% of level 2 NICUs in the state at the time.

The OT interventions were limited to infants born ≥37 weeks’ gestation with NAS who required pharmacologic treatment and were receiving formula solely or to supplement breastfeeding. During the OT period, 2985 infants who were not pharmacologically treated and 65 infants who were exclusively breastfed were not included as part of the PDSA. Centers started group-specific formula when initiating pharmacologic treatment. Centers using HCF (groups 1–2) were instructed to fortify term formula to 22 kcal/oz. Group-specific formulas were discontinued when the infant was weaned off opiates for 24 hours. Centers could deviate from group-specific formulas on a case-by-case basis. Use of a particular formula brand was not stipulated (note that the lactose content of LLFs ranges significantly across brands).

OPQC faculty spoke with each center to underscore alignment of the OT method with the PDSA improvement approach, confirm their willingness to participate in 1 of the 4 groups, and ensure they understood expectations to (1) continue their group formula practice without deviating during the project, (2) maintain other NAS care practices without change, (3) collect and submit data regularly, and (4) obtain buy-in from the entire care team. We used the Institute for Healthcare Improvement Breakthrough Series approach to support the project.18  During the OT phase, we held monthly group-specific webinars and quarterly webinars for all participants. In the implementation phase, we held only quarterly webinars. Reliable implementation of key practices and measurement of compliance are critical components of OT.15  Centers used process mapping, review of failures, and targeted PDSA cycles to increase adherence to their designated formula.19  Each center was responsible for establishing a process to support implementation of their chosen formula at high reliability based on their initial center-specific process baseline (eg, whether they adopted a new formula practice or maintained the current practice but improved reliability).

Centers uniformly collected data on treatment practices and outcomes using standard forms and submitted the data to the OPQC monthly through a secure Internet portal. The primary outcome was LOS, measured as the total LOS at the OPQC hospital. More granular detail on LOS was available during OT; therefore, for the response plot analysis only (see below), LOS was calculated as the total LOS at any center (which included the LOS at the OPQC center plus the LOS at the transferring center, if applicable). Secondary outcomes for OT learning were treatment failure (failing opiate wean, requiring dose escalation or secondary medication) and the percentage of infants with >10% weight loss during the first 7 days of life. Although we examined length of treatment, we present only LOS results because the impact of the OT interventions on LOS represents the sum of the impact on length of treatment and on other aspects of the hospital stay.

Process measures were used to examine the proportion of infants for whom HCF and LLF were used during pharmacologic treatment. During the active OT phase, we plotted on control charts the percentage of infants for whom HCF (or LLF) was used ≥90% of the time during pharmacologic treatment. For infants solely formula-fed, this was counted as the number of infants receiving HCF (or LLF), starting within 1 day of the start of pharmacologic treatment and continuing for at least 90% of the rest of the length of pharmacologic treatment. For infants who were also breastfeeding, this was counted as the number of infants receiving HCF (or LLF) for 90% of the total time receiving formula during the length of pharmacologic treatment to account for days they may have been exclusively breastfed. The denominator included all formula-fed infants receiving pharmacologic treatment. During the implementation phase, data collection was simplified, and sites were asked to report on the formula type used most frequently (defined as at least half of the time) while an infant was receiving pharmacologic treatment. Therefore, once the collaborative transitioned to the implementation phase, to harmonize the different data collection approaches in the OT and implementation phases, process measures were redefined as the proportion of formula-fed, pharmacologically treated infants for whom HCF (or LLF) was used ≥50% of the time (for data collected during OT) or most frequently (for data collected in the implementation phase) during pharmacologic treatment.

Readmission rates are often included as a balancing measure for QI initiatives focused on LOS reduction; however, the OPQC did not have access to these data and did not track a balancing measure in this project.

OT Phase

We excluded centers without a standard inpatient pharmacologic wean (n = 6) or those unwilling to commit to an OT group (n = 1) from analysis. We also excluded 14 infants with missing data on feeding and pharmacologic treatment received.

We used statistical process control to examine each group’s progress. Patient-level data on daily formula use during pharmacologic treatment were not collected during the first phase of improvement work but were collected with the onset of OT; thus, we could not compute baseline measures of HCF and LLF use with the more detailed operational definition. The effects of HCF and LLF use were studied by using response plots. Response plots reveal the magnitude of the differences in the levels of calorie and lactose content (ie, high versus standard) without considering statistical significance. Average outcome measures are plotted on the y-axis for the 4 combinations of each factor and level (ie, the 4 groups). When the effect of the factors is independent, the lines on the plot are parallel. When the lines are not parallel, the factors have some dependence on each other.14  Because centers varied in using their assigned formula, analyses were performed by (1) assigning all infants of a center to the intended group (by design) and (2) assigning each infant on the basis of the formula received (as treated). In this article, we present only the by-design analysis because it is the more conservative analysis.

To address imbalances in known confounders across groups, we conducted a post hoc analysis using multivariable regression models to create adjusted response plots. We used directed acyclic graphs20  to describe the causal pathways leading to outcomes and determine the minimal sufficient adjustment set of confounders to use in regression models.21  Through application of causal inference rules, we identified breastfeeding and the percentage of infants with <10% weight loss within 7 days of birth (weight loss) as the minimum adjustment set. We included these factors as independent variables in the regression models in which LOS or treatment failure was specified as the outcome. The minimum adjustment set included breastfeeding and birth weight for the regression model in which weight loss was specified as the outcome. We also conducted analyses that included the level of care, inborn status, and hospital methadone or morphine protocol along with the minimum adjustment set. Results were similar with this broader set of variables, so only models using the directed acyclic graph–informed minimum adjustment set were used. LOS and treatment failure models included center as a random effect to adjust for within-site clustering (correlation) of effects. We examined the interaction between HCF and LLF by including the appropriate term in the regression models and tested its statistical significance.

Implementation Phase

The OPQC modified formula feeding recommendations for the nonpharmacologic treatment bundle according to the OT findings and promoted the new bundle during the implementation phase (January 2017 and December 2017). Control charts were used to track changes in HCF and LLF use and LOS during the 1-year implementation phase. With p-charts, we plotted the percentage of infants for whom HCF or LLF was used ≥50% of the time or most frequently during pharmacologic treatment. Because of the skewed distribution of LOS, we used log-LOS as the dependent variable in a generalized linear mixed model, accounting for clustering by site and serial autocorrelation to calculate adjusted means and their control limits. We then back-transformed the adjusted means and control limits to their natural scale to construct X-bar charts, as we had done in a previous OPQC NAS analysis.3  We show only the X-bar charts because the log-LOS S-charts did not reveal special cause variation. For LOS, the 12 months before the start of OT were used as the baseline, and the centerline was fixed and carried forward. We created control charts for the aggregate and 4 OT groups. Special cause was identified by using standard control chart rules.22 

In Table 1, the 4 self-selected formula groups are described. Group 3 (low lactose, standard calorie) included 20 sites and contributed the largest number of patients (n = 333; 60% of patients in the OT phase). Group 4 (standard calorie, standard lactose) included 11 sites (mostly level 2 hospitals) and contributed the smallest number of patients (n = 54; 10% of patients in the OT phase). Except for group 3, in which 85% of sites were using morphine as pharmacologic treatment, other groups were balanced between using methadone or morphine as the primary pharmacologic treatment. Formula compliance was lower in groups not using LLF.

Unadjusted plots for by-design analyses suggest that the combination of LLF and HCF is associated with lower LOS (Fig 1A), less treatment failure (Fig 2A), and less weight loss (Fig 3A) than other combinations. However, after adjusting for breastfeeding and weight loss, HCF was associated with reduced LOS (Fig 1B) and treatment failure (Fig 2B), regardless of lactose content. The biggest change after adjustment was in group 2 (HCF, standard lactose). After adjusting for breastfeeding and birth weight, there were similar magnitudes of reduction in weight loss between the adjusted and unadjusted analyses with HCF, regardless of lactose content (Fig 3B). In the unadjusted response plot in Fig 1A, when HCF was used, LOS was shorter when using LLF versus standard lactose. When low-calorie formula was used, LOS was slightly longer when using LLF versus standard lactose. The difference in LOS between standard formula and HCF was small when regular lactose was used but reduced LOS by 2.7 days when the LLF was used. However, after adjusting for confounders (Fig 1B), the use of HCF decreased LOS by ∼2.8 to 3.1 days, regardless of lactose content (the lines are parallel). Regression analysis results are included in Supplemental Tables 24. The interaction terms in the regression models were not significant and were not included in the minimum adjusted set models, so the lines on the response plots are parallel.

FIGURE 1

Response plots for LOS (average LOS for infants treated pharmacologically). The plots reveal differences in LOS according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and weight loss.

FIGURE 1

Response plots for LOS (average LOS for infants treated pharmacologically). The plots reveal differences in LOS according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and weight loss.

Close modal
FIGURE 2

Response plots for treatment failure (either dose escalation or failed wean or secondary medication). The plots reveal differences in treatment failure according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and weight loss.

FIGURE 2

Response plots for treatment failure (either dose escalation or failed wean or secondary medication). The plots reveal differences in treatment failure according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and weight loss.

Close modal
FIGURE 3

Response plots for weight loss. The plots reveal differences in the percentage of infants with >10% weight loss from birth weight during the first 7 days of life according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and birth weight.

FIGURE 3

Response plots for weight loss. The plots reveal differences in the percentage of infants with >10% weight loss from birth weight during the first 7 days of life according to formula calorie and lactose content. A, By-design analysis. B, Adjusted analysis accounting for hospital, breastfeeding, and birth weight.

Close modal

On the basis of these findings, the OPQC NAS nonpharmacologic treatment bundle was updated to recommend the use of a term HCF (22 kcal/oz) in infants who were not breastfed. Use of LLF was at the discretion of individual centers. Across the OPQC, the mean HCF use, based on the aggregate control chart, increased after the conclusion of the OT phase to 41.1% then increased further during the implementation phase to 49.4% (Fig 4A). Rates of HCF use were higher and remained stable throughout the OT and implementation phases in groups 1 and 2 (Fig 4 B and C), although group 2 showed a trend toward decreased HCF use in the implementation phase. Group 3 (Fig 4D), which was designated to use standard-calorie formula during OT, showed an increase in HCF use from 13.6% to 35.9% during the implementation phase, with the first increase beginning at the conclusion of the OT phase. Group 4 (Fig 4E) showed a trend toward increased HCF use, with the last 6 data points above the group mean. On the basis of the aggregate control chart, LLF use across the entire collaborative also increased from 72.1% to 78.8% in the implementation phase (Fig 5A). During the OT phase, rates of LLF use were higher in groups 1 (Fig 5B) and 3 (Fig 5D), which were designated to use LLF, although rates appeared to vary more (Fig 5B, group 1) and trend downward (Fig 5D, group 3) during the implementation phase. Group 4 (Fig 5E) showed an increase in LLF at the conclusion of OT, and group 2 (Fig 5C) showed increased LLF use during the implementation phase.

FIGURE 4

OPQC NAS project (22-kcal formula): changes in use of HCF during the OT and implementation phases. The p-charts reveal the percentage of infants receiving HCF by month. A, Entire OPQC. B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

FIGURE 4

OPQC NAS project (22-kcal formula): changes in use of HCF during the OT and implementation phases. The p-charts reveal the percentage of infants receiving HCF by month. A, Entire OPQC. B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

Close modal
FIGURE 5

OPQC NAS project (LLF): changes in the use of LLF during the OT and implementation phases. The p-charts reveal the percentage of infants receiving LLF by month. A, Entire OPQC. B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

FIGURE 5

OPQC NAS project (LLF): changes in the use of LLF during the OT and implementation phases. The p-charts reveal the percentage of infants receiving LLF by month. A, Entire OPQC. B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

Close modal

The collaborative aggregate did not reveal a decrease in LOS during the OT phase, but LOS decreased during the implementation phase from 17.1 to 16.4 days (Fig 6A). We observed a group-specific decrease in LOS in group 2 (Fig 6C), and group 3 (Fig 6D) showed a trend toward a decreasing LOS in the implementation phase, with 9 of the last 11 points below the centerline.

FIGURE 6

OPQC NAS project (average LOS for infants treated pharmacologically): changes in average LOS during the OT and implementation phases. X-bar charts reveal average LOS (geometric mean) by month. A, Entire OPQC (shift in February 2017). B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose) (shift in April 2016). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

FIGURE 6

OPQC NAS project (average LOS for infants treated pharmacologically): changes in average LOS during the OT and implementation phases. X-bar charts reveal average LOS (geometric mean) by month. A, Entire OPQC (shift in February 2017). B, Group 1 (LLF and HCF). C, Group 2 (HCF and standard lactose) (shift in April 2016). D, Group 3 (LLF and standard calorie). E, Group 4 (standard calorie and lactose).

Close modal

OT enabled the OPQC to learn about the effects of a key element of the nonpharmacologic bundle. The OT phase suggested that HCF was associated with less treatment failure, less weight loss, and shorter LOS. These results were used to improve the OPQC’s standardized nonpharmacologic care bundle. Although use of HCF and LLF remained <90% by the end of this phase, LOS decreased from 17 to 16.3 days. Across both phases of the OPQC improvement work, there has been a total LOS reduction of 2 days. Although shy of the goal of a 20% reduction set in the project aim statement, this finding is similar to the improvement achieved by the Vermont Oxford Network quality collaborative.23 

OT allowed the OPQC to examine the impact of formula feeding across a range of conditions in 54 centers, a key strength of the approach. Although we saw a smaller increase in HCF than expected during the implementation phase, we saw a temporal link between increases in HCF and improved outcomes and evidence of special cause variation in LOS after the revised nonpharmacologic bundle was released. We saw the largest reduction in LOS in group 2 during the OT phase, which would be expected if HCF is associated with improved outcomes. Evidence would be clearer for the temporal link between HCF use and reduction in LOS if we also saw group-specific reductions in LOS during the OT phase in both groups implementing HCF (reduction seen only in group 2 and not in group 1) and in the implementation phase in groups 3 and 4. However, group 1 was already experiencing shorter LOS during the OT phase, which could be explained by their high use of HCF throughout OT. In addition, the increases in HCF in groups 3 and 4 (from near 0 to <50%) may not have been large enough to translate to a visible effect on the group-specific average LOS but may have been significant enough to contribute to a distributional change in HCF use across the collaborative that translated to improvement in the aggregate LOS. Replication in other systems will determine if the OPQC’s learnings translate to similar improvements outside of Ohio.

Evidence supporting rooming-in, skin-to-skin contact, use of cuddler programs, and breastfeeding for infants with NAS is growing2429 ; however, it remains unclear if formula type makes a difference in the management of infants with NAS who cannot breastfeed or need additional supplementation. Aside from a small feasibility trial (N = 47) by Bogen et al30  that revealed that HCF is safe and feasible and results in larger daily weight gains among infants with prenatal methadone exposure, there are little data to guide practice. The OPQC included HCF and LLF feeds in the nonpharmacologic bundle on the basis of the rationale that LLF may improve infants’ gastrointestinal withdrawal symptoms and that HCF may increase resilience to withdrawal symptoms, thereby resulting in fewer patients with symptoms, fewer treatment failures, and shorter LOS. Because this was a QI effort, it was not designed to generate evidence regarding the efficacy of formula feeding or the biological mechanisms of the action of formula type in NAS care. Large prospective studies are needed to conclusively identify the best approach to formula feeding when breast milk is not consistently available.

The OPQC demonstrated that OT was feasible and provided a powerful way to capitalize on learning across a large statewide QI collaborative. In 2015, Piazza et al16  used OT to reduce central line–associated bloodstream infections in partnership with 17 children’s hospitals. These hospitals identified infection prevention practices that contributed to reducing infection rates and achieved even greater improvements by implementing the findings more broadly.16,31  The OPQC’s experience using OT with 54 centers adds credibility to this approach. With the growth of state-based perinatal and pediatric chronic disease networks,32  broader use of OT may be a means to more efficient and effective improvement.

There were many lessons learned in using large-scale OT in a multicenter collaborative setting. Despite broad agreement from participating OPQC sites to adhere to the OT approach at the outset of the project and despite the OPQC’s attention to coaching teams to build processes using the tenets of reliability science, we did not see uniform adoption of group-specific formulas. Groups 1 and 3 had group-specific compliance rates of >75%, which was comparable to what was seen in the SLUG Bug (Standardizing Line Care Under Guideline Recommendations) OT effort.16  However, groups 2 and 4, which were assigned to use standard lactose formulas, had compliance rates of 50% to 60%. Although OPQC faculty emphasized the importance of adherence to selected formulas on individual calls with site lead physicians and although lead physicians were instructed to obtain buy-in for the formula group choice with the entire NICU staff, the inability to achieve high reliability in all settings may result from a lack of buy-in (particularly related to sites in the standard lactose groups using LLF) and/or difficulty in achieving widespread agreement across a large NICU staff. Sufficient time must be invested in obtaining full agreement both from the QI team and the broader unit staff up front.

We also did not see reliable implementation of the improved nonpharmacologic bundle in the implementation phase. Increases in HCF were more modest than expected, and LLF increased more than expected. This may suggest that, even in a mature network such as the OPQC, participating sites remain skeptical about the credibility of learning by using an OT approach and the findings did not successfully change center’s preconceived notions. Similarly, the SLUG Bug project revealed that only 50% of centers that were previously using clean tubing change techniques switched to sterile tubing changes on the basis of the OT findings.31  On the basis of both of these OT reports, it is clear that there is still a need to improve how OT results are interpreted and used to inform practice by frontline clinicians. Other networks that choose to use this method should focus more on establishing enthusiasm for, and commitment to, the approach up front and on ensuring that participants understand the findings.

Although OT is a powerful way to learn from coordinated large-scale testing, the design cannot fully control for all factors that drive LOS, treatment failure, and weight loss. Lack of random assignment to the testing groups may have resulted in imbalances of important factors across the formula groups, potentially confounding the observed differences between groups. Similarly, although the implementation phase revealed a general improvement in LOS that was concurrent with roll out of the revised nonpharmacologic bundle and a modest increase in HCF use, it did not include a concurrent control arm, and in our analysis, we could not exclude the impact of other system changes that may have occurred during the same time frame. It is always possible that other system changes (eg, formula contract changes) were responsible for the increase in HCF and LLF use and the observed reduction in LOS. One must be cautious in interpreting the decreases in LOS seen during the implementation phase as a direct effect of the increase in HCF use.

Over 4 years, the OPQC decreased LOS by 2 days for the ∼40% of infants exposed to opioid who were pharmacologically treated. The OT approach allowed the OPQC to learn from practice variation and further improve nonpharmacologic care, contributing, at least in part, to this accomplishment. We demonstrated that coordinated wide-scale testing of changes by using OT is a feasible and effective way to learn and drive improvement.

The following were participating OPQC NAS sites: Akron Children’s Hospital Medical Center, Akron Children’s St Elizabeth/Mahoning Valley Hospital, Akron Children’s St Joseph Warren, Akron Children’s Hospital at Summa Health System, Akron General Medical Center, Aultman Hospital, Bethesda North Hospital, Cincinnati Children’s Hospital Medical Center, Cleveland Clinic, Dayton Children’s Hospital Medical Center, Fairview Hospital, Good Samaritan Hospital, Hillcrest Hospital, Mercy Children’s Hospital, Mercy Anderson Hospital, MetroHealth Medical Center, Miami Valley Hospital, Mount Carmel East Hospital, Mount Carmel West Hospital, ProMedica Toledo Children’s Hospital NICU, ProMedica Toledo Hospital Newborn Nursery, Nationwide Children’s Hospital, Nationwide Children’s Hospital NICU at Mount Carmel St Ann’s, Nationwide Doctors Hospital, Nationwide Dublin Methodist Hospital, Nationwide Grant Hospital, Nationwide Riverside Methodist Hospital, Nationwide Ohio State University Wexner NICU, The Ohio State University Newborn Nursery, University of Cincinnati Medical Center/University Hospital, University Hospitals Cleveland Medical Center, Rainbow Babies and Children’s Hospital, Adena Regional Medical Center, Atrium Medical Center, University Hospitals Elyria Medical Center, Fort Hamilton Hospital, Genesis HealthCare System, Good Samaritan Hospital Dayton, Kettering Medical Center, Licking Memorial Hospital, Lima Memorial Health System, Marion General, Mercy Health West, Mercy Hospital Fairfield, Mercy Medical Center–Canton, Mercy Regional Medical Center (Lorain), ProMedica Bay Park Hospital, Soin Medical Center, Southern Ohio Medical Center, Southview Medical Center, Springfield Regional Medical Center, St Rita’s Medical Center, The Christ Hospital, Trumbull Memorial Hospital, and Upper Valley Medical Center.

Dr Kaplan led the conceptualization of this work, was the project lead for implementation, was the primary writer of the manuscript, and critically revised the manuscript; Mr Kuhnell conducted the original analyses and contributed to the writing and critical revisions of the manuscript; Ms Ford helped conceptualize this work, was a leader in the implementation of the project as described, and contributed to critical revisions of the manuscript; Drs Crowley, Lannon, McClead, Wexelblatt, and Walsh helped conceptualize this work, served as core faculty in the project implementation as described, and critically revised the manuscript; Mr Provost helped conceptualize the work, assisted in interpretation of the analyses, and critically revised the manuscript; Dr Macaluso helped conceptualized this work (particularly the analytic plan), supervised the original analyses, and contributed to the writing and critical revisions of the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: The Ohio Perinatal Quality Collaborative is funded by the Ohio Department of Medicaid through the Medicaid Technical Assistance and Policy Program and administered by the Ohio Colleges of Medicine Government Resource Center. The funders had no role in data analysis or manuscript development.

     
  • HCF

    high-calorie formula

  •  
  • LLF

    low-lactose formula

  •  
  • LOS

    length of stay

  •  
  • NAS

    neonatal abstinence syndrome

  •  
  • OPQC

    Ohio Perinatal Quality Collaborative

  •  
  • OT

    orchestrated testing

  •  
  • PDSA

    plan-do-study-act

  •  
  • QI

    quality improvement

1
Massatti
R
,
Falb
M
,
Yors
A
,
Starr
S
.
Neonatal Abstinence Syndrome and Drug Use Among Pregnant Women in Ohio, 2004–2011
.
Columbus, OH
:
Ohio Department of Mental Health and Addiction Services
;
2013
2
Tolia
VN
,
Patrick
SW
,
Bennett
MM
, et al
.
Increasing incidence of the neonatal abstinence syndrome in US neonatal ICUs
.
N Engl J Med
.
2015
;
372
(
22
):
2118
2126
3
Walsh
MC
,
Crowley
M
,
Wexelblatt
S
, et al;
Ohio Perinatal Quality Collaborative
.
Ohio Perinatal Quality Collaborative improves care of neonatal narcotic abstinence syndrome
.
Pediatrics
.
2018
;
141
(
4
):
e20170900
4
Hall
ES
,
Wexelblatt
SL
,
Crowley
M
, et al
;
OCHNAS Consortium
.
A multicenter cohort study of treatments and hospital outcomes in neonatal abstinence syndrome
.
Pediatrics
.
2014
;
134
(
2
).
5
Hall
ES
,
Wexelblatt
SL
,
Crowley
M
, et al
;
OCHNAS Consortium
.
Implementation of a neonatal abstinence syndrome weaning protocol: a multicenter cohort study
.
Pediatrics
.
2015
;
136
(
4
).
6
Finnegan
LP
,
Connaughton
JF
 Jr.
,
Kron
RE
,
Emich
JP
.
Neonatal abstinence syndrome: assessment and management
.
Addict Dis
.
1975
;
2
(
1–2
):
141
158
7
Jansson
LM
,
Velez
M
,
Harrow
C
.
Methadone maintenance and lactation: a review of the literature and current management guidelines
.
J Hum Lact
.
2004
;
20
(
1
):
62
71
8
Weinberger
SM
,
Kandall
SR
,
Doberczak
TM
,
Thornton
JC
,
Bernstein
J
.
Early weight-change patterns in neonatal abstinence
.
Am J Dis Child
.
1986
;
140
(
8
):
829
832
9
Dryden
C
,
Young
D
,
Campbell
N
,
Mactier
H
.
Postnatal weight loss in substitute methadone-exposed infants: implications for the management of breast feeding
.
Arch Dis Child Fetal Neonatal Ed
.
2012
;
97
(
3
):
F214
F216
10
Martinez
A
,
Kastner
B
,
Taeusch
HW
.
Hyperphagia in neonates withdrawing from methadone
.
Arch Dis Child Fetal Neonatal Ed
.
1999
;
80
(
3
):
F178
F182
11
Greene
CM
,
Goodman
MH
.
Neonatal abstinence syndrome: strategies for care of the drug-exposed infant
.
Neonatal Netw
.
2003
;
22
(
4
):
15
25
12
Balakrishnan
M
,
Raghavan
A
,
Suresh
GK
.
Eliminating undesirable variation in neonatal practice: balancing standardization and customization
.
Clin Perinatol
.
2017
;
44
(
3
):
529
540
13
Resar
R
.
Why we need to learn standardisation
.
Aust Fam Physician
.
2005
;
34
(
1–2
):
67
68
14
Moen
RD
,
Nolan
TW
,
Provost
LP
.
Quality Improvement Through Planned Experimentation
, 2nd ed.
New York, NY
:
McGraw-Hill
;
1999
15
Pallotto
EK
,
Chuo
J
,
Piazza
AJ
, et al
.
Orchestrated testing
.
Am J Med Qual
.
2017
;
32
(
1
):
87
92
16
Piazza
AJ
,
Brozanski
B
,
Provost
L
, et al
.
SLUG bug: quality improvement with orchestrated testing leads to NICU CLABSI reduction
.
Pediatrics
.
2016
;
137
(
1
):
e20143642
17
Ohio Perinatal Quality Collaborative
.
About OPQC. Available at: https://www.opqc.net/about. Accessed March 5, 2019
18
Institute for Healthcare Improvement
.
The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement
.
Boston, MA
:
Institute for Healthcare Improvement
;
2003
19
Langley
GL
,
Moen
RD
,
Nolan
KM
,
Nolan
TW
,
Norman
CL
,
Provost
LP
.
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
, 2nd ed.
San Francisco
:
Jossey-Bass
;
2009
20
Pearl
J
.
Causality: Models, Reasoning, and Inference
, 2nd ed.
New York, NY
:
Cambridge University Press
;
2009
21
Textor
J
,
Hardt
J
,
Knüppel
S
.
DAGitty: a graphical tool for analyzing causal diagrams
.
Epidemiology
.
2011
;
22
(
5
):
745
22
Provost
LP
,
Murray
SK
.
The Health Care Data Guide: Learning from Data for Improvement
, 1st ed.
San Francisco, CA
:
Jossey-Bass
;
2011
23
Patrick
SW
,
Schumacher
RE
,
Horbar
JD
, et al
.
Improving care for neonatal abstinence syndrome
.
Pediatrics
.
2016
;
137
(
5
):
e20153835
24
Bagley
SM
,
Wachman
EM
,
Holland
E
,
Brogly
SB
.
Review of the assessment and management of neonatal abstinence syndrome
.
Addict Sci Clin Pract
.
2014
;
9
(
1
):
19
25
Grossman
MR
,
Berkwitt
AK
,
Osborn
RR
, et al
.
An initiative to improve the quality of care of infants with neonatal abstinence syndrome
.
Pediatrics
.
2017
;
139
(
6
):
e20163360
26
Holmes
AV
,
Atwood
EC
,
Whalen
B
, et al
.
Rooming-in to treat neonatal abstinence syndrome: improved family-centered care at lower cost
.
Pediatrics
.
2016
;
137
(
6
):
e20152929
27
Howard
MB
,
Schiff
DM
,
Penwill
N
, et al
.
Impact of parental presence at infants’ bedside on neonatal abstinence syndrome
.
Hosp Pediatr
.
2017
;
7
(
2
):
63
69
28
MacMillan
KDL
,
Rendon
CP
,
Verma
K
,
Riblet
N
,
Washer
DB
,
Volpe Holmes
A
.
Association of rooming-in with outcomes for neonatal abstinence syndrome: a systematic review and meta-analysis
.
JAMA Pediatr
.
2018
;
172
(
4
):
345
351
29
Wachman
EM
,
Grossman
M
,
Schiff
DM
, et al
.
Quality improvement initiative to improve inpatient outcomes for neonatal abstinence syndrome
.
J Perinatol
.
2018
;
38
(
8
):
1114
1122
30
Bogen
DL
,
Hanusa
BH
,
Baker
R
,
Medoff-Cooper
B
,
Cohlan
B
.
Randomized clinical trial of standard- versus high-calorie formula for methadone-exposed infants: a feasibility study
.
Hosp Pediatr
.
2018
;
8
(
1
):
7
14
31
Pallotto
EK
,
Piazza
AJ
,
Smith
JR
, et al
.
Sustaining SLUG bug CLABSI reduction: does sterile tubing change technique really work?
Pediatrics
.
2017
;
140
(
4
):
e20163178
32
Britto
MT
,
Fuller
SC
,
Kaplan
HC
, et al
.
Using a network organisational architecture to support the development of Learning Healthcare Systems
.
BMJ Qual Saf
.
2018
;
27
(
11
):
937
946

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Wexelblatt is on the speaker bureau for Abbott Nutrition with a specific focus on neonatal abstinence syndrome; the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose. The Ohio Perinatal Quality Collaborative neonatal abstinence syndrome project is funded by the Ohio Department of Medicaid and administered by the Ohio Colleges of Medicine Government Resource Center. The views expressed in this publication are solely those of the authors and do not represent the views of the state of Ohio or federal Medicaid programs.

Supplementary data