BACKGROUND:

Interventions to address vaccine hesitancy and increase vaccine acceptance are needed. This study sought to determine if a Web-based, social media intervention increases early childhood immunization.

METHODS:

A 3-arm, randomized controlled trial was conducted in Colorado from September 2013 to July 2016. Participants were pregnant women, randomly assigned (3:2:1) to a Web site with vaccine information and interactive social media components (VSM), a Web site with vaccine information (VI), or usual care (UC). Vaccination was assessed in infants of participants from birth to age 200 days. The primary outcome was days undervaccinated, measured as a continuous and dichotomous variable.

RESULTS:

Infants of 888 participants were managed for 200 days. By using a nonparametric rank-based analysis, mean ranks for days undervaccinated were significantly lower in the VSM arm versus UC (P = .02) but not statistically different between the VI and UC (P = .08) or between VSM and VI arms (P = .63). The proportions of infants up-to-date at age 200 days were 92.5, 91.3, and 86.6 in the VSM, VI, and UC arms, respectively. Infants in the VSM arm were more likely to be up-to-date than infants in the UC arm (odds ratio [OR] = 1.92; 95% confidence interval [CI], 1.07–3.47). Up-to-date status was not statistically different between VI and UC arms (OR = 1.62; 95% CI, 0.87–3.00) or between the VSM and VI arms (OR = 1.19, 95% CI, 0.70–2.03).

CONCLUSIONS:

Providing Web-based vaccine information with social media applications during pregnancy can positively influence parental vaccine behaviors.

What’s Known on This Subject:

Many parents with concerns about childhood vaccines use the Internet and social media for vaccine information. The effectiveness of using Web-based vaccine information and social media to increase parental vaccine acceptance has not been evaluated.

What This Study Adds:

By using a randomized control trial design, we found that a Web-based, social media intervention administered outside of the physician’s office can effectively improve childhood vaccine acceptance among pregnant women.

Between 10% to 15% of parents choose to delay or refuse 1 or more recommended vaccines for their children.1 This decision leaves children and their communities vulnerable to vaccine-preventable diseases.2 Parents who are hesitant to vaccinate their children also have complex information-seeking behaviors. They often start to weigh the risks and benefits of vaccination during pregnancy, seek information from many sources, and express interest in receiving vaccine information before routine well-child visits.3,5 Although physicians are trusted sources of health information for parents, vaccine-hesitant parents are inclined to distrust traditional sources of scientific authority and report using the Internet to gather information on vaccines.6,8 

Regardless of whether they are hesitant about childhood vaccination, parents who use the Internet to educate themselves must sift through vast amounts of vaccine information. Web sites range from government-sponsored, pro-vaccine resources that carefully present factual information to staunchly antivaccination Web sites that use anecdotes and social media to disseminate misinformation.9 Along this spectrum, numerous parenting message boards and blogs vigorously discuss vaccine-related topics. Exposure to antivaccine messages through social media appears to intensify parents’ worries and lower their intentions to vaccinate.10 

At the same time, social media may have the potential to allay parental vaccine concerns and improve immunization rates. An expert-moderated, interactive vaccine Web site could provide parents with a forum to voice their opinions, ask questions, and interact with other concerned parents and vaccine experts.11 This type of dynamic online environment could help build trust and combat misinformation. At present, the impact of using social media to improve vaccine acceptance is not known.12 In addition, we are not aware of any interventions targeting vaccine hesitancy that have effectively changed parental vaccination behavior in the United States.12,14 As part of the Colorado Vaccine Social Media study, we conducted a randomized controlled trial (RCT) to evaluate the effectiveness of Web-based vaccine information and social media interventions to increase vaccine acceptance.

Between September 2013 and July 2016, we conducted a single-center RCT of vaccine information and social media interventions designed to reduce undervaccination among infants of women recruited during pregnancy. Our primary outcome was days undervaccinated from birth to age 200 days. We hypothesized that infants of women exposed to interventions during pregnancy will have less vaccine delay than infants receiving usual pediatric care.

Participants were randomly assigned to 1 of 3 groups: a Web site with vaccine information and interactive social media components (VSM); Web site with vaccine information (VI); or usual care only (UC). The Kaiser Permanente Colorado (KPCO) institutional review board approved this study.

All participants were members of the KPCO health plan, a nonprofit managed care organization serving ∼628 000 individuals. Each year, ∼5000 pregnant women and ∼130 000 children receive health care at KPCO clinics.

Recruitment was conducted in 6-week waves between September 2013 and October 2015. At the beginning of each wave, we used electronic health records to identify pregnant women in the third trimester of pregnancy (13–6 weeks from delivery). Women had to be age 18 years or older, English speaking, have Internet access, and be enrolled in the KPCO health plan. Pregnant women were ineligible if they had a diagnosis for fetal death, miscarriage, or congenital anomaly. Eligible women received a combination of postcards, e-mails, and phone calls to elicit participation. Informed consent was obtained online by using a secure encryption program.

After consent, participants were administered a baseline survey to assess demographics and Internet use. Participants were also administered the Parent Attitudes and Childhood Vaccines (PACV) screening survey, which is a validated, 15-item instrument that assesses vaccine hesitancy on a scale of 0 to 100.15 Consistent with previous studies, participants scoring ≥50 were classified as “vaccine hesitant,” whereas participants with scores <50 were “nonhesitant.” To ensure balance across study arms, randomization was conducted independently within the 2 strata of hesitancy. Because only a small fraction of Web site visitors actively engage in social media activities,16 we used a randomization allocation ratio of 3:2:1 across the VSM/VI/UC arms to facilitate interaction. Randomization was done by an unblinded statistician using the SAS/STAT (SAS Institute, Inc, Cary, NC) procedure Proc Plan. Although the participants and study team were not blinded to study arm assignment, the study team was blinded to participants’ hesitancy status.

To enhance security and prevent contamination, participants randomly assigned to the VSM and VI arms were required to create a login and password for the Web site. Infants of participants were managed for 200 days after birth to assess vaccination status. To reflect how a Web-based resource would be used in practice, individuals in the VSM and VI arms were given access to the Web site but were not required to visit it.

Separate interventions were developed for the VSM and VI arms. The theoretical basis for the VSM intervention was the multidirectional communication model,17 a social marketing strategy with 3 components. Component 1 is a standard, top-down process in which Web site developers create and present content to users. Component 2 is a bottom-up process that allows users to create content and interact with Web site developers. Component 3 is a side-to-side process in which users can interact with each other and share information. This model is intended to empower users by allowing them to become active participants in the communication process, thereby eliciting positive health behavior changes.18 In contrast to the VSM intervention, the VI intervention only included the top-down component of the model.

The interventions were designed and pilot tested by using an adapted mental-models approach that included focus groups, individual interviews, surveys, and usability testing with parents and pregnant women.19 Details of this process have been described previously.20 In brief, our study team first developed the factual vaccine content, guided by the Health Belief Model and Theory of Planned Behavior.21,22 We sought to present content that accurately represented the risks and benefits of vaccination, including information on vaccine-preventable diseases, vaccine safety, vaccine laws, the recommended immunization schedule, vaccine ingredients, vaccine development, and basic immunology. Information was labeled and arranged into short, easy-to-read sections, guided by best practices in risk communication and Web site design.23,24 Sources of information were carefully referenced and hyperlinked to help convey transparency and credibility.25 The information was focused on encouraging parents to receive recommended vaccines on time. Participants in the VSM and VI arms had access to the same base vaccine content.

In addition to vaccine content, participants in the VSM arm had access to social media technologies that included a blog, discussion forum, chat room, and “Ask a Question” portal through which participants could directly ask our experts questions about vaccination. These technologies were designed to facilitate engagement and reinforce the factual content. Experts included a pediatrician, a vaccine safety researcher, and a risk communication specialist. Each month, the research team created 1 to 2 blog posts covering topics such as new vaccine safety research, vaccine-preventable disease outbreaks, changes in immunization policy, and the importance of adhering to the recommended immunization schedule. Posts were either text or audio (podcasts), and participants could contribute comments and ask questions. Each month, we hosted online chat sessions in which participants could engage in real-time conversations with experts. Participants were also encouraged to submit questions privately through e-mail; the team provided personalized responses within 2 business days. All participants in the VSM arm received monthly newsletters to encourage Web site participation and highlight new Web site content.

All interactive components were moderated to prevent bullying, disclosure of personal identifying health information, and abusive language. Responses to comments and questions adhered to a consistent communication framework designed to convey dedication, expertise, and honesty.23,24 Intervention details (including the Hoffman’s template for intervention description and replication checklist and guide and screenshots of the intervention Web sites) are included in the Supplemental Information.26 

Routine pediatric preventive care was available to participants in all study arms. At KPCO, structured well-child visits are scheduled at 2 weeks and 2, 4, 6, and 12 months of age.27 Most immunizations are administered at these routinely scheduled, 20-minute health supervision visits. It is standard practice at KPCO to provide a previsit informational sheet listing the vaccines recommended at that visit as well as Vaccine Information Statements.28 

Vaccination Status: Days Undervaccinated and Up-to-Date Status

Immunization data for infants were extracted from the electronic health record. We assessed vaccination status over the first 200 days of age to cover a a majority of the recommended infant vaccines and minimize loss to follow-up. We assessed the following 6 vaccines recommended by the Advisory Committee on Immunization Practices: hepatitis B; rotavirus; diphtheria-tetanus-acellular pertussis; Haemophilus influenzae type b; pneumococcal conjugate vaccine; and polio. Our primary outcome was days undervaccinated, a continuous metric that measures differences between the time when vaccine doses were actually administered and when the doses should have been administered according to the Advisory Committee on Immunization Practices schedule.29,30 For example, the first dose of diphtheria-tetanus-acellular pertussis is due at age 2 months but is not considered late until age 92 days. Days undervaccinated for this dose would begin accruing on day 93. Of note, infants who did not receive the birth dose of hepatitis B vaccine were not considered delayed; days undervaccinated for the first dose of hepatitis B vaccine started accruing at age 93 days as with the other recommended 2-month vaccines.

Days undervaccinated was analyzed both as a continuous measure and as a dichotomous variable (up-to-date with no delays, yes or no). The dichotomous variable of days undervaccinated was labeled as up-to-date vaccination status, representing a clinically meaningful measure for providers. Infants with 0 cumulative days undervaccinated at age 200 days were considered up-to-date. As a subanalysis, we assessed up-to-date status for measles-mumps-rubella (MMR) vaccine among infants with at least 489 days of follow-up, when days undervaccinated for the first dose of MMR begins to accrue.

Statistical Methods

The study was powered to detect a clinically meaningful odds ratio (OR) of 1.8–2.2 for up-to-date vaccination status between the study arms. For this effect size, we required 900 participants on the basis of an anticipated baseline vaccine hesitancy of 20%, a 3:2:1 allocation ratio, and a 2-sided α of .05. An a priori P value of <.05 was considered statistically significant. We conducted a modified intent-to-treat analysis by keeping the study arm assignment but excluding infants without outcome data from the analysis. Infants of participants were excluded if they disenrolled from KPCO after birth, enrolled after age 60 days, were not continuously enrolled during their follow-up period, or were not using KPCO for primary health services. These exclusions help ensure complete ascertainment of vaccination data. Participants were also excluded if they requested to be removed from the study or experienced a fetal demise or death of the child. Although we screened infants for documented contraindications to vaccines, premature infants were not excluded because they are to receive vaccines according to the recommended schedule.31 

Days undervaccinated and up-to-date vaccination status were assessed from birth to age 200 days. Because of the skewed distribution of days undervaccinated, we used a nonparametric analysis and rank transformation approach.32 We ranked the days undervaccinated for all infants and then compared the mean ranks across study arms using 1-way analysis of variance. Up-to-date vaccination status was analyzed by using logistic regression to estimate ORs and associated 95% confidence intervals (CIs). Logistic regression was also used to assess MMR status among children with at least 489 days of follow-up. Data were analyzed with SAS 9.4 software (SAS Institute, Inc, Cary, NC).

A total of 1093 pregnant women were recruited into the study (Fig 1). By using a 3:2:1 randomization ratio, 542 participants were randomly assigned to VSM, 371 were assigned to VI, and 180 were assigned to UC. Baseline characteristics were evenly distributed across study arms (Table 1). Mean maternal age at enrollment was 31.6 years, and a majority of the population was white (86.9%) and college educated (82.8%). At enrollment, 14.1% of the population was classified as vaccine hesitant on the basis of the PACV screener, and >62% of participants reported using the Internet for health information at least weekly. Median vaccine hesitancy scores were 13, 17, and 15 for the VSM, VI, and UC arms, respectively (P = .44).

FIGURE 1

Consolidated Standards of Reporting Trials flow diagram.

FIGURE 1

Consolidated Standards of Reporting Trials flow diagram.

Close modal
TABLE 1

Baseline Characteristics of Study Participants by Study Arm

CharacteristicaTotal (n = 1093)VSM (n = 542)VI (n = 371)UC (n = 180)
Hesitancy by PACVb     
 Median (IQR) 17 (26.0) 13.0 (26.0) 17.0 (34.0) 15.0 (31.5) 
Mother’s age at enrollment     
 Mean (SD), y 31.6 (4.3) 31.6 (4.4) 31.5 (4.3) 31.4 (4.1) 
No. of children, n (%)c,d     
 Pregnant with first child 518 (47.4) 246 (45.4) 189 (50.9) 83 (46.1) 
 Have previous child or children 573 (52.4) 296 (54.6) 181 (48.8) 96 (53.3) 
Race, n (%)c,d     
 White 950 (86.9) 476 (87.8) 315 (84.9) 159 (88.3) 
 Othere 138 (12.6) 64 (11.8) 54 (14.6) 20 (11.1) 
Household income, $, n (%)c,d     
 ≤80 000  440 (40.3) 208 (38.4) 157 (42.3) 75 (41.7) 
 >80 000 597 (54.6) 306 (56.5) 194 (52.3) 97 (53.9) 
Education, n (%)c,d     
 Some college or less 186 (17.0) 93 (17.2) 63 (17.0) 30 (16.7) 
 College and higher 905 (82.8) 449 (82.8) 307 (82.8) 149 (82.8) 
Use of Internet for health, n (%)c,d     
 Less than every week 410 (37.5) 210 (38.8) 140 (37.7) 60 (33.3) 
 Every week or more 681 (62.3) 332 (61.3) 230 (62.0) 119 (66.1) 
CharacteristicaTotal (n = 1093)VSM (n = 542)VI (n = 371)UC (n = 180)
Hesitancy by PACVb     
 Median (IQR) 17 (26.0) 13.0 (26.0) 17.0 (34.0) 15.0 (31.5) 
Mother’s age at enrollment     
 Mean (SD), y 31.6 (4.3) 31.6 (4.4) 31.5 (4.3) 31.4 (4.1) 
No. of children, n (%)c,d     
 Pregnant with first child 518 (47.4) 246 (45.4) 189 (50.9) 83 (46.1) 
 Have previous child or children 573 (52.4) 296 (54.6) 181 (48.8) 96 (53.3) 
Race, n (%)c,d     
 White 950 (86.9) 476 (87.8) 315 (84.9) 159 (88.3) 
 Othere 138 (12.6) 64 (11.8) 54 (14.6) 20 (11.1) 
Household income, $, n (%)c,d     
 ≤80 000  440 (40.3) 208 (38.4) 157 (42.3) 75 (41.7) 
 >80 000 597 (54.6) 306 (56.5) 194 (52.3) 97 (53.9) 
Education, n (%)c,d     
 Some college or less 186 (17.0) 93 (17.2) 63 (17.0) 30 (16.7) 
 College and higher 905 (82.8) 449 (82.8) 307 (82.8) 149 (82.8) 
Use of Internet for health, n (%)c,d     
 Less than every week 410 (37.5) 210 (38.8) 140 (37.7) 60 (33.3) 
 Every week or more 681 (62.3) 332 (61.3) 230 (62.0) 119 (66.1) 

IQR, interquartile range.

a

None of the characteristics were statistically different across study arms (P >.05).

b

Assessed by using the PACV screening survey.

c

Percentages represent column percentages.

d

Numbers do not equal column total because of missing data.

e

Includes African American, American Indian, Asian American, Pacific Islander, and multiracial.

A lack of outcome data led to the exclusion of 205 infants (18.8%); infants were excluded because they were disenrolled from KPCO after birth (n = 16), enrolled after age 60 days (n = 21), had incomplete follow-up because of loss of insurance (n = 159), were not using KPCO for primary care services (n = 5), or had a fetal demise (n = 4) (Fig 1). There were no infants with documented contraindications to vaccines. Loss to follow-up ranged from 17.2% to 19.9% across study arms. Among participants lost to follow-up, median vaccine hesitancy scores were not significantly different across the arms (P = .97).

Of 739 participants in the VSM and VI arms with 200 days of follow-up, 259 (35.0%) visited the Web sites at least once, with a mean of 1.8 (SD = 1.7) and range of 1 to 15 visits. Of 75 vaccine-hesitant participants, 33 (44.0%) visited the Web sites compared with 226 (34.0%) of the 664 nonhesitant participants. Over the study period, the VSM Web site offered 59 blog entries and 31 chat sessions. Participants in the VSM arm (n = 442) contributed 90 comments and questions. A majority of the interaction was between participants and the research team rather than between participants.

Mean ranks for days undervaccinated were 438.5, 443.0, and 465.4 for the VSM, VI, and UC arms, respectively. Infants in the VSM arm had a lower mean rank for days undervaccinated than infants in the UC arm (difference = −26.9; P value = .02; Table 2). Mean ranks did not differ significantly between the VI and UC arms or the VSM and VI arms.

TABLE 2

Days Undervaccinated, Mean Ranks for Days Undervaccinated and Difference in the Mean Ranks Between Study Arms (n = 888)

Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
 5th 50th 95th     
VSM (n = 442) 155 438.46 VSM versus UC −26.91 .02 
VI (n = 297) 107 443.03 VI versus UC −22.34 .08 
UC (n = 149) 411 465.37 VSM versus VI −4.57 .63 
Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
 5th 50th 95th     
VSM (n = 442) 155 438.46 VSM versus UC −26.91 .02 
VI (n = 297) 107 443.03 VI versus UC −22.34 .08 
UC (n = 149) 411 465.37 VSM versus VI −4.57 .63 
a

Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.

b

Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.

The proportion of infants up-to-date at the end of follow-up were 92.5, 91.3, and 86.6 for the VSM, VI, and UC arms, respectively. Infants in the VSM arm were more likely to be up-to-date at age 200 days than infants in the UC arm (OR = 1.92; 95% CI, 1.07–3.47; Table 3). Up-to-date status did not differ significantly between the VI and UC arms or the VSM and VI arms. The interaction between study arm and baseline vaccine hesitancy status was not statistically significant (P = .52). Among all infants enrolled from birth to age 200 days in KPCO (n = 8877) during the study period, the rate of up-to-date status was 86.3%, suggesting that the UC infant population was representative of the overall KPCO infant population.

TABLE 3

Proportion of Infants Up-to-Date for Vaccination Status and OR Estimates for Up-to-Date Vaccination Status Between Study Arms (n = 888)

Study ArmProportion of Infants Up-to-Date (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VSM (n = 442) 92.53 VSM versus UC 1.92 (1.07–3.47) .03 
VI (n = 297) 91.25 VI versus UC 1.62 (0.87–3.00) .13 
UC (n = 149) 86.58 VSM versus VI 1.19 (0.70–2.03) .52 
Study ArmProportion of Infants Up-to-Date (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VSM (n = 442) 92.53 VSM versus UC 1.92 (1.07–3.47) .03 
VI (n = 297) 91.25 VI versus UC 1.62 (0.87–3.00) .13 
UC (n = 149) 86.58 VSM versus VI 1.19 (0.70–2.03) .52 

For the MMR subanalysis, there were 776 (71%) infants with at least 489 days of continuous follow-up. The proportion of infants who received MMR by the end of follow-up were 95.6, 95.5, and 91.8 for the VSM, VI, and UC arms, respectively. Although none of the study arm comparisons were statistically significant, infants in the VSM and VI arms were ∼2 times more likely to have received MMR than infants in the UC arm (Table 4).

TABLE 4

Proportion of Children Up-to-Date on First Dose of MMR Vaccine and OR Estimates for Up-to-Date MMR Vaccination Status Between Study Arms (n = 776)

Study ArmProportion of Children Up-to-Date on First Dose of MMR (%)Study Arm ComparisonsOR for Up-to-Date MMR Vaccination Status (95% CI)P
VSM (n = 389) 95.63 VSM versus UC 1.95 (0.87–4.39) .10 
VI (n = 265) 95.47 VI versus UC 1.88 (0.79–4.49) .15 
UC (n = 122) 91.80 VSM versus VI 1.04 (0.49–2.21) .92 
Study ArmProportion of Children Up-to-Date on First Dose of MMR (%)Study Arm ComparisonsOR for Up-to-Date MMR Vaccination Status (95% CI)P
VSM (n = 389) 95.63 VSM versus UC 1.95 (0.87–4.39) .10 
VI (n = 265) 95.47 VI versus UC 1.88 (0.79–4.49) .15 
UC (n = 122) 91.80 VSM versus VI 1.04 (0.49–2.21) .92 

This RCT of a Web-based vaccine information and social media intervention had a positive impact on early childhood immunization. Pregnant women exposed to the VSM arm were more likely to vaccinate their infants on time than participants receiving UC. These results suggest that interactive, informational interventions administered outside of the physician’s office can improve vaccine acceptance.

The authors of previous research have shown that the timing of vaccine information receipt is important to parents.5,7 Traditionally, vaccine information is provided to parents at well-child visits, although some parents make their vaccination decisions during pregnancy. In the absence of accurate information during pregnancy, parents may tend to rely on the Internet, which may expose them to misinformation.33 We found that providing accurate online information with interactive technologies during pregnancy has a positive impact on infant-vaccine acceptance.

Providing parents with information debunking vaccination falsehoods, such as the link between the MMR vaccination and autism, can cause vaccine-hesitant parents to become more entrenched in their antivaccination views and reduce their intentions to vaccinate.34 However, this backfire effect is likely modified by additional factors, such as the source, wording, tone, and timing of information.35 Our intervention demonstrated that parental vaccine behaviors can be positively influenced with a carefully timed, interactive, informational online resource administered by their health care organization.

Although our VSM arm was designed to foster interaction between parents, almost all of the interaction was between parents and the research team. Parents who engaged in the social media applications were primarily interested in asking our experts questions to address their specific vaccine concerns. They did not appear to be interested in forming an ongoing, vaccine-focused online community with other parents enrolled in the KPCO health plan. Given that only 1% of digital health social network members actively contributed to the interaction,16 it is possible that more participant-to-participant engagement would be observed if the intervention was scaled across the entire health plan or made publically available.

Web-based interventions are low-cost and broadly accessible approaches to deliver important public health messages.36 However, the VSM arm in our trial required significant resources to administer. A multidisciplinary, expert staff developed and reviewed new content, moderated chat room discussions, answered complicated questions related to the vaccination schedule, and addressed vaccine safety rumors as they surfaced. Therefore, it is unlikely that single clinicians or clinics would have the means to manage their own social media interventions. This could be mitigated by creating a national, centralized social media vaccine resource, but additional research would need to determine if it would be trusted and used by parents. To help with these implementation decisions, a formal cost-effectiveness analysis of the VSM intervention is underway.

Social media technologies, Web site design preferences, and online information-gathering practices are constantly evolving. Such changes pose challenges to Web-based interventions. For example, our study period spanned more than 5 years from the development of the interventions through participant follow-up, data collection, and analysis. Over this time, newer social media platforms became increasingly popular among our target demographic population, including Twitter, Snapchat, and Instagram.37 Although it is not known if these platforms could be used to effectively address vaccine hesitancy, it is possible that our Web site appeared increasingly outdated and less appealing as the trial progressed. Therefore, future applications of our interventions would have to stay abreast of emerging technologies to continue to attract each new generation of parents.

This study had several limitations. The trial was conducted in a single, integrated health care system in Colorado, where the baseline vaccine hesitancy rate was 14.1%. Although this rate is similar to other investigations, there were only 99 hesitant participants in the analysis.15,38 As a result, we had limited statistical power to assess the interaction between study arm and vaccine hesitancy status.

Because this intervention was implemented as a pragmatic trial, we chose not to conduct a per-protocol analysis. Over the course of the trial, we gave participants in the VSM and VI arms unlimited access to the Web site, but they were not required to visit it. Of participants, ∼35% visited the Web site at least once, and hesitant participants were more likely to access the Web site than nonhesitant participants. This implies that a per-protocol analysis in which researchers examine an association between Web site exposure and immunization outcomes would be biased by baseline hesitancy.

The overall loss to follow-up rate in the trial was 18.8%, which is largely attributable to parents who did not use KPCO health insurance for pediatric care after their children were born. Although this may have affected the trial’s generalizability, the rate of loss to follow-up did not differ significantly across the study arms. In addition, the rate of vaccine hesitancy among those lost to follow-up was similar across the arms, suggesting that excluding these individuals did not affect the internal validity of the results.

Despite these limitations, the results of this RCT demonstrate that Web-based vaccine information with social media technologies can positively influence parental vaccine decisions. As a complement to routine well-child care, the information appears to be effective when presented to parents before their children are born.

     
  • CI

    confidence interval

  •  
  • KPCO

    Kaiser Permanente Colorado

  •  
  • MMR

    measles-mumps-rubella

  •  
  • OR

    odds ratio

  •  
  • PACV

    Parent Attitudes and Childhood Vaccines

  •  
  • RCT

    randomized controlled trial

  •  
  • UC

    usual care

  •  
  • VI

    Web site with vaccine information only

  •  
  • VSM

    Web site with vaccine information and interactive social media component

Dr Glanz conceptualized the study design, obtained the funding, provided input for the statistical analysis, interpreted the results, and drafted the manuscript; Ms Wagner provided input on the study design, helped draft the study protocol, and contributed to the first draft of the manuscript; Dr Narwaney provided input on the study design, managed the data, conducted the statistical analysis, and helped draft the manuscript and protocol; Ms Kraus helped conceptualize the intervention design, managed the data, and contributed to the first draft of the manuscript; Dr Shoup helped conceptualize the intervention, helped manage the data, and provided input on the first draft of the manuscript; Dr Xu designed the analytic plan, interpreted the statistical results, and provided input on the first draft of the manuscript; Dr O’Leary helped conceptualize the study design, interpreted the statistical results, and provided input on the first draft of the manuscript; Dr Omer helped conceptualize the intervention design, interpreted the statistical results, and provided input on the first draft of the manuscript; Dr Gleason helped manage the data, administer the intervention, and interpret the results; Dr Daley helped conceptualize the study design, provided input for the statistical analyses, and contributed to the first draft of the manuscript; and all authors reviewed and revised the manuscript and approved the final manuscript as submitted.

This trial has been registered at www.clinicaltrials.gov (identifier NCT01873040).

FUNDING: Supported by a research grant from the Agency for Healthcare Research and Quality (R01HS021492).

We thank Chris Boyd, BS, of KPCO Institute for Health Research for his technical assistance in developing the Web-based intervention; John F. Steiner, MD, of KPCO Institute for Health Research for his helpful comments on the article; Douglas J. Opel, MD, MPH, of the University of Washington for his assistance in implementing the PACV survey and his input on the article; and Kristin Goddard, MPH, of Kaiser Permanente Northern California for her help with developing and administering the interventions.

1
Dempsey
AF
,
Schaffer
S
,
Singer
D
,
Butchart
A
,
Davis
M
,
Freed
GL
.
Alternative vaccination schedule preferences among parents of young children.
Pediatrics
.
2011
;
128
(
5
):
848
856
[PubMed]
2
Phadke
VK
,
Bednarczyk
RA
,
Salmon
DA
,
Omer
SB
.
Association between vaccine refusal and vaccine-preventable diseases in the United States: a review of measles and pertussis.
JAMA
.
2016
;
315
(
11
):
1149
1158
[PubMed]
3
Návar
AM
,
Halsey
NA
,
Carter
TC
,
Montgomery
MP
,
Salmon
DA
.
Prenatal immunization education the pediatric prenatal visit and routine obstetric care.
Am J Prev Med
.
2007
;
33
(
3
):
211
213
[PubMed]
4
Glanz
JM
,
Wagner
NM
,
Narwaney
KJ
, et al
.
A mixed methods study of parental vaccine decision making and parent-provider trust.
Acad Pediatr
.
2013
;
13
(
5
):
481
488
[PubMed]
5
Lieu
TA
,
Zikmund-Fisher
BJ
,
Chou
C
,
Ray
GT
,
Wittenberg
E
.
Parents’ perspectives on how to improve the childhood vaccination process.
Clin Pediatr (Phila)
.
2017
;
56
(
3
):
238
246
[PubMed]
6
Freed
GL
,
Clark
SJ
,
Butchart
AT
,
Singer
DC
,
Davis
MM
.
Sources and perceived credibility of vaccine-safety information for parents.
Pediatrics
.
2011
;
127
(
suppl 1
):
S107
S112
[PubMed]
7
Jones
AM
,
Omer
SB
,
Bednarczyk
RA
,
Halsey
NA
,
Moulton
LH
,
Salmon
DA
.
Parents’ source of vaccine information and impact on vaccine attitudes, beliefs, and nonmedical exemptions.
Adv Prev Med
.
2012
;
2012
:
932741
8
World Health Organization (WHO)
.
Report of the SAGE Working Group on Vaccine Hesitancy
.
Geneva, Switzerland
:
WHO
;
2014
9
Grant
L
,
Hausman
BL
,
Cashion
M
,
Lucchesi
N
,
Patel
K
,
Roberts
J
.
Vaccination persuasion online: a qualitative study of two provaccine and two vaccine-skeptical websites.
J Med Internet Res
.
2015
;
17
(
5
):
e133
[PubMed]
10
Betsch
C
,
Renkewitz
F
,
Betsch
T
,
Ulshöfer
C
.
The influence of vaccine-critical websites on perceiving vaccination risks.
J Health Psychol
.
2010
;
15
(
3
):
446
455
[PubMed]
11
Witteman
HO
,
Zikmund-Fisher
BJ
.
The defining characteristics of Web 2.0 and their potential influence in the online vaccination debate.
Vaccine
.
2012
;
30
(
25
):
3734
3740
[PubMed]
12
Sadaf
A
,
Richards
JL
,
Glanz
J
,
Salmon
DA
,
Omer
SB
.
A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy.
Vaccine
.
2013
;
31
(
40
):
4293
4304
[PubMed]
13
European Centre for Disease Prevention and Control (ECDC)
.
Catalogue of Interventions Addressing Vaccine Hesitancy
.
Stockholm, Germany
:
ECDC
;
2017
14
Kaufman
J
,
Synnot
A
,
Ryan
R
, et al
.
Face to face interventions for informing or educating parents about early childhood vaccination.
Cochrane Database Syst Rev
.
2013
; (
5
):
CD010038
[PubMed]
15
Opel
DJ
,
Taylor
JA
,
Zhou
C
,
Catz
S
,
Myaing
M
,
Mangione-Smith
R
.
The relationship between parent attitudes about childhood vaccines survey scores and future child immunization status: a validation study.
JAMA Pediatr
.
2013
;
167
(
11
):
1065
1071
[PubMed]
16
van Mierlo
T
.
The 1% rule in four digital health social networks: an observational study.
J Med Internet Res
.
2014
;
16
(
2
):
e33
[PubMed]
17
Thackeray
R
,
Neiger
BL
.
A multidirectional communication model: implications for social marketing practice.
Health Promot Pract
.
2009
;
10
(
2
):
171
175
[PubMed]
18
Yang
Q
.
Are social networking sites making health behavior change interventions more effective? A meta-analytic review.
J Health Commun
.
2017
;
22
(
3
):
223
233
[PubMed]
19
Morgan
MG
,
Fischoff
B
,
Bostrom
A
,
Altman
CJ
.
Risk Communication: A Mental Models Approach
.
New York, NY
:
Cambridge University Press
;
2002
20
Shoup
JA
,
Wagner
NM
,
Kraus
CR
,
Narwaney
KJ
,
Goddard
KS
,
Glanz
JM
.
Development of an interactive social media tool for parents with concerns about vaccines.
Health Educ Behav
.
2015
;
42
(
3
):
302
312
[PubMed]
21
Skinner
CS
,
Tiro
J
,
Champion
VL
. The health belief model. In:
Glanz
K
,
RImer
BK
,
Viswanath
K
, eds.
Health Behavior: Theory, Research, and Practice
. 5th ed.
San Francisco, CA
:
John Wiley & Sons
;
2015
:
75
94
22
Montano
DE
,
Kasprzyk
D
. Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In:
Glanz
K
,
Rimer
BK
,
Viswanath
K
, eds.
Health Behavior: Theory, Research, and Practice
. 5th ed.
San Francisco, CA
:
John Wiley & Sons
;
2015
:
95
124
23
Covello
VT
.
Best practices in public health risk and crisis communication.
J Health Commun
.
2003
;
8
(
suppl 1
):
5
8; discussion 148–151
[PubMed]
24
Fischhoff
B
,
Brewer
NT
,
Downs
JS
, eds.
Communicating Risks and Benefits: An Evidence Based User’s Guide
.
Silver Springs, MD
:
US Department of Health and Human Services, Food and Drug administration
;
2012
. Available at: www.fda.gov/downloads/AboutFDA/ReportsManualsForms/Reports/UCM268069.pdf. Accessed November 28, 2016
25
Flanagin
AJ
,
Metzger
MJ
.
The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information.
New Media Soc
.
2007
;
9
(
2
):
319
342
26
Hoffmann
TC
,
Glasziou
PP
,
Boutron
I
, et al
.
Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide.
BMJ
.
2014
;
348
:
g1687
[PubMed]
27
Bergman
DA
,
Beck
A
,
Rahm
AK
.
The use of internet-based technology to tailor well-child care encounters.
Pediatrics
.
2009
;
124
(
1
). Available at: www.pediatrics.org/cgi/content/full/124/1/e37
[PubMed]
28
Edlich
RF
,
Martin
ML
,
Foley
ML
, et al
.
Vaccine information statements. Revolutionary but neglected educational advances in healthcare in the United States.
J Long Term Eff Med Implants
.
2005
;
15
(
1
):
91
114
[PubMed]
29
Glanz
JM
,
Newcomer
SR
,
Narwaney
KJ
, et al
.
A population-based cohort study of undervaccination in 8 managed care organizations across the United States.
JAMA Pediatr
.
2013
;
167
(
3
):
274
281
[PubMed]
30
Luman
ET
,
Barker
LE
,
Shaw
KM
,
McCauley
MM
,
Buehler
JW
,
Pickering
LK
.
Timeliness of childhood vaccinations in the United States: days undervaccinated and number of vaccines delayed.
JAMA
.
2005
;
293
(
10
):
1204
1211
[PubMed]
31
Hamborsky
J
,
Kroger
A
,
Wolfe
C
, eds.
Epidemiology and Prevention of Vaccine-Preventable Diseases: The Pink Book: Course Textbook
. 13th ed.
Washington, DC
:
Public Health Foundation
;
2015
32
Conover
WJ
,
Iman
RL
.
Rank transformations as a bridge between parametric and nonparametric statistics.
Am Stat
.
1981
;
35
(
3
):
124
129
33
Kata
A
.
A postmodern Pandora’s box: anti-vaccination misinformation on the Internet.
Vaccine
.
2010
;
28
(
7
):
1709
1716
[PubMed]
34
Nyhan
B
,
Reifler
J
,
Richey
S
,
Freed
GL
.
Effective messages in vaccine promotion: a randomized trial.
Pediatrics
.
2014
;
133
(
4
). Available at: www.pediatrics.org/cgi/content/full/133/4/e835
[PubMed]
35
Lewandowsky
S
,
Ecker
UK
,
Seifert
CM
,
Schwarz
N
,
Cook
J
.
Misinformation and its correction continued influence and successful debiasing.
Psychol Sci Public Interest
.
2012
;
13
(
3
):
106
131
[PubMed]
36
Bennett
GG
,
Glasgow
RE
.
The delivery of public health interventions via the Internet: actualizing their potential.
Annu Rev Public Health
.
2009
;
30
:
273
292
[PubMed]
37
Duggan
M
. Mobile messaging and social media 2015. Pew Research Center. 2015. Available at: www.pewinternet.org/2015/08/19/mobile-messaging-and-social-media-2015/. Accessed November 28, 2016
38
Williams
SE
,
Rothman
RL
,
Offit
PA
,
Schaffner
W
,
Sullivan
M
,
Edwards
KM
.
A randomized trial to increase acceptance of childhood vaccines by vaccine-hesitant parents: a pilot study.
Acad Pediatr
.
2013
;
13
(
5
):
475
480
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data