Research on the Efficacy and Feasibility of Essentials for Parenting Toddlers and Preschoolers
SUPPORTING STATEMENT: PART B
OMB No. 0920-XXXX
Submitted by:
Department of Health and Human Services
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
Division of Violence Prevention
Government Project Officers:
Beverly Fortson, Ph.D., Science Officer
[email protected] 770-488-1284
Colby Lokey, M.S., Project Officer
[email protected] 770-488-3785
May 8, 2015
B. Collections of Information Employing statistical procedures
1. Respondent Universe and Sampling Methods
2. Procedures for the Collection of Information
3. Methods to Maximize Response Rates and Deal with Non-Response
4. Test of Procedures or Methods to be Undertaken
5. Individuals consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Information
B. Collections of Information Employing Statistical Procedures
In summary, contractors from Westat will recruit and enroll 200 eligible parents to participate in an 18 week intervention and evaluation. Because the Essentials for Parenting Toddlers and Preschoolers (EFP) resource is web-based, recruitment will be done entirely through digital outreach, both through advertising such as with Facebook, and publisher outreach through parent-oriented blogs and message boards. Interested parents will be screened through an online system, and parents who are interested and eligible will be contacted by study staff for consent and enrollment.
Parents will be enrolled through block randomization into two groups, one that is guided through a study-specific version of the EFP website modules in a highly directed order, and one that is free to navigate the study-specific EFP website as they wish. Each block will consist of 20 parents randomized into conditions. Parents will start the study as they are enrolled over the course of approximately one week’s time per block.
Participation will last for 18 weeks for each parent, during which they will explore the website and complete assigned weekly assessments. An automated management system custom designed for the study will provide parents with access to their assigned website content, and deliver scheduled assessments through the website. Reminders about assigned content and assessments will be individualized to the parent’s study schedule, and sent automatically by text or email.
Because the target population for this study is parents with young children who use the Internet, recruitment will be accomplished through digital outreach. Below are the details of the proposed recruitment plan for the current study.
Sample and Screening. The primary audience for recruitment is mothers and fathers between ages 18-45 whose oldest child is 2-4 years old, who speak English, have Internet access at home, and who are diverse in geography, socioeconomic status, and race/ethnicity. We will seek a total of 400 parents to ensure enrollment of a diverse group of 200.
A brief online screening questionnaire (see Attachments I1 and J) will be used to capture key demographic eligibility information and ensure that parents are available to complete the study during the field period. Parents who do not meet the screening criteria (e.g., they are over age 45, their oldest child is 5 or older) will be informed immediately of their ineligibility and thanked for their interest. The following criteria, as mentioned above, will be used for eligibility screening:
Parent is between ages 18-45 years.
Parent is the biological, adoptive, or step-parent of least one child, the oldest of whom is between ages 2-4 years.
Parent has internet access at home.
Parent is willing to commit to intervention/assessment procedures, and
Parent speaks and reads English.
Once eligibility is established, the following additional demographic information will be sought to ensure diversity of participants enrolled in the study and to inform recruitment:
Parent race
Parent ethnicity
Household income relative to poverty
Geographical region of the country
Parent gender
Parents will be assigned to intervention condition based on randomization of blocks of 20 parents. Within each block of 20 parents, enrollees will be randomly assigned to either the Natural Navigation (NN) or Guided Navigation (GN) arm. If a parent is randomized to the GN arm, the randomization will also determine the order in which modules are presented. Within each block randomization, 10 parents will be assigned to NN, 10 to GN, and within the GN arm, two parents will be assigned to each possible module presentation order based on a Latin square design of module orders. Assignment to condition will be done after a parent consents to participation (see Attachment E: Participant Phone Consent Script) and is officially enrolled in the study. Parents will be informed of their assignment during the enrollment phone call and told to expect an email or text message containing login information. The Post-Screening Introductory Email for Participants and the Post-Enrollment Follow-up Email for Participants can be found in Attachments F and Attachment G, respectively.
Online Recruitment. There are two methods that will be used to recruit participants. The first is advertising through Facebook (see example ads in Attachment H), and the second is by reaching out to social media sites to ask them to promote the study (see sample text for publishers in Attachment H). For the Facebook recruitment efforts, we will use targeted web ads on Facebook to recruit parents of children ages 2-4 and drive traffic to the online recruitment and screening site. For outreach, we will engage online publishers and parenting message board administrators in the United States, asking them to post on their web platform about the study recruitment effort in order to reach parents of toddlers/preschoolers and direct them to the online recruitment and screening site. These approaches are described in more detail below.
Facebook Advertising. Through the use of Facebook’s promoted post ad service, Westat will use ads that include approved study recruitment messages to individuals within the Facebook network who have identified themselves as parents. We will refine this further to ensure that parents whom Facebook’s data have identified as having a child age 0-3 or 4-12 years old are targeted. We will target this audience in every state across the U.S. to reach all regions of the country. At the outset of the campaign we will not target specific income levels, but as the participants begin to enroll, we will work closely with the rest of the study team to see where additional refinements are needed. For example, if we are seeing that interested parents are high income, we can refocus our targeting to lower income audiences in Facebook. We will monitor the advertisement performance data on a daily basis to ensure that we are targeting effectively and reaching our eligibility and diversity goals.
Publisher Outreach. There are a number of parenting blogs and websites whose audiences include the parents of toddlers and preschoolers, such as 5 minutes for mom, busy dad blog, reddit.com/r/mommit, reddit.com/r/daddit, ahaparenting.com and others. We can engage with the editors and moderators of these sites and request a post about the study on their websites, and/or sharing information about the study via social media. In the case of reddit and other such parenting message boards, we would engage the community of parents to ensure a positive reception and directly post information about the study.
Publisher outreach messaging will focus on the basics of the study and include links to the information and screening website. Each correspondence will be tailored to the specific publisher and will focus on information about the study that they can share with their audiences. The key messages communicated to publishers will be about the nature and goals of the study, the time commitment involved, the incentives provided to compensate for time spent on the study, and the potential of the study to help parents of young children more broadly. If publishers request pre-scripted content to post about the study recruitment effort, we will provide such content.
Data collection will be conducted by qualified individuals employed by the contractor, Westat. Data will be collected from parents using online assessments. The following steps will be implemented by CDC to safeguard the objectivity of the evaluation: 1) all data collectors will receive human subjects training; and 2) CDC will hold weekly or bi-weekly conference calls with the contractors to provide oversight and discuss data collection procedures.
As the Westat principal investigator on the Westat IRB protocol, Dr. Nancy Weinfield is responsible for overseeing the scientific and human subjects integrity of the study.
Parents will complete 18 weekly online assessments, cued by on-screen messages as they log into EFP modules, as well as by email or text messages (per participant’s preference). The assessments are aggregated into four groups that reflect what is assessed and when: (1) core assessment, (2) content knowledge and usefulness assessment, (3) detailed assessment, and (4) rotating assessment.
Core Assessment (see Attachments I3 and J). For a multiple baseline design, the same measures must be completed each week. Thus, each parent will need to complete 18 core assessments. The core assessments are timed to allow the examination of change from pre-, to mid- (day 7), to post-completion (day 14) of each intervention module. To accommodate 3 assessments per module while allowing 2 weeks for parents to complete each module, the post assessment of one module is administered at the same time as the pre assessment for the next. The pre-, mid-, and post-module assessments for module 1 occur on days 22, 29, and 36 of the intervention period. The day 36 assessment occurs on the same day that the parent begins module 2, prior to their exposure to module 2 material. In this way, the day 36 assessment serves as the post-module 1 assessment and the pre-module 2 assessment.
Content Knowledge and Usefulness Assessment (see Attachments I4 and I5). We will assess module-specific knowledge twice per module—at days 1 (pre) and 14 (post) of the unit—and perceived usefulness of content once per module—at day 14 into the unit (post). Knowledge items from all modules will also be part of the week 1 and week 18 assessments. Usefulness items from all modules will also be part of the week 18 assessment. Each knowledge measure will be a 3-item study-specific true/false quiz tapping the major points of the technique that is being taught in the module the parent is working on. To illustrate, parents will be asked three questions about the features of effective commands during the Giving Directions module, and three features of age appropriate time outs in the Time Out module. Each usefulness measure will comprise 3 module-specific items tapping usefulness, everyday applicability to parenting challenges, and whether the information is sufficiently detailed to be applied.
Detailed Assessment (see Attachments I2 and J). The detailed assessment will be administered at weeks 1 and 18 and is designed to measure child externalizing behavior, parenting behaviors, parenting stress, parenting thoughts, and parent psychological symptoms with validated measures. Demographics items not asked through the screening and enrollment process will also be added to the detailed assessment.
Rotating Assessment (see Attachments I3 and J). The rotating assessments will be administered during Weeks 2-17. It would be advantageous but too burdensome on parents to administer the detailed assessment weekly to examine intervention effects. Thus, we are administering only some of the measures from the detailed assessment each week, using a planned missingness design (Graham, Taylor, & Cumsille, 2001). The measures can be grouped together in 4 blocks of 9 to 11 items as follows: (1) short forms (Lorber, Xu, Slep, Bulling, & O’Leary, 2014) of the PS over-reactivity and laxness subscales (11 items), (2) CTS-PC corporal punishment and Fragile Families Parenting Aggravation subscales (10 items), (3) Positive Attitudes toward Corporal Punishment Scale and Parental Sense of Competence Scale efficacy subscale (11 items), and (4) the Parent Cognition Scale child responsible subscale (9 items). The order of administration will be counterbalanced with a 4 × 4 Latin square to ensure an equal number of administrations of each measure block with no order confounding.
Website Usage Assessment (see Attachments I6 and J). Currently, the CDC website collects metrics via Google Analytics. Google Analytics collects user information from the website to provide reporting on usage such as page views and average time spent. These same metrics will be applied to the study-specific mock website housed and managed by Westat. However, to answer the research question, “How do parents perceive the website?” we plan to administer a System Usability Scale. The items (e.g., “I would recommend that others use this website”; “I thought the website was easy to use”) tap perceived ease of use as well as usefulness of the website. Because system usability is not module-specific, the Website usage assessment will be administered only during the Week 18 assessment.
Given the large burden on respondents due to the number of assessments conducted over a substantial amount of time, we anticipate that attrition is inevitable. If we find in early weeks that attrition is higher than anticipated, we may increase the number of respondents who can start the study in a given week from 20 per block to more in later weeks, or we may extend the recruitment of additional blocks into January 2016.
We plan to use several strategies to retain respondents through the course of the study. First, during the initial screening call, we will make sure the respondents fully understand the requirements of the study, the time commitment involved, when their participation will start, and the incentive plan. Respondents will need to affirm that they are willing and able to fulfill the study requirements before being selected for the study. We believe that this detailed discussion of study requirements will prevent later attrition. Second, we will provide tokens of appreciation after each assessment. Parents will receive a $20 incentive for their first detailed baseline assessment and will then receive a $10 incentive when each subsequent weekly assessment is completed during weeks 2-17. Respondents will receive a $30 incentive for the longer, 1-month post intervention follow-up. A bonus incentive of $40 will be provided to parents who complete each of the assessments within its designated timeframe, given the need for assessments to occur at regular intervals. Receiving these timely tokens of appreciation will help maintain interest in the study. We also believe that retention will be enhanced by making the timing of the assessments as clear as possible and compliance as simple as possible. The short message service (SMS) will send automated reminder emails or texts (depending on the respondent’s preference) when it is time for the respondent to complete each assessment. If a respondent has not completed the assessment within the specific window (such as within 3 days), the SMS will send a reminder text/email emphasizing the importance of participation. We will follow up with a telephone call to respondents who have completed at least 2 assessments but then fail to complete an assessment and do not respond to the reminders. We feel that personal contact will be effective in enhancing retention, particularly when the importance of the study and our appreciation for their willingness to participate is emphasized, as well as the commitment the respondent made to the study at the time of screening. If we are not able to reach respondents through telephone, we will send them a letter that underscores these points. Respondents who fail to complete the first two assessments are unlikely to have developed any level of emotional engagement with the study, and they will be dropped and replaced with a respondent from the reserve sample.
B.4. Test of Procedures or Methods to be Undertaken
The assessment measures have been developed with the extensive input of expert consultants both internal and external to CDC and Westat. In addition, many of the assessment measures are validated measures that have been used with parents in similar research settings and are those that have strong psychometric properties.
All instruments and procedures have been reviewed extensively by CDC. The following individuals have worked closely in developing the instrument and procedures that will be used, and will be responsible for data analysis:
Beverly Fortson, Division of Violence Prevention, CDC
Colby Lokey, Division of Violence Prevention, CDC
Nancy Weinfield, Westat
Amy Slep, NYU Family Translational Research Group
Michael Lorber, NYU Family Translational Research Group
References
Barkley, R. A., Edwards, G., Laneri, M., Flethcher, K., & Metevia, L. (2001). The efficacy of problem-solving training alone, behavior management training alone, and their combination for parent-adolescent conflict in teenagers with ADHD and ODD. Journal of Consulting and Clinical Psychology, 69, 926-941.
Baydar, N., Reid, M. J., & Webster-Stratton, C. (2003). The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start mothers. Child Development, 74, 1433-1453.
Brick, J. M., & Williams, D. (2013). Explaining rising nonresponse rates in cross-sectional surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 36-59.
Carley-Baxter, L. R., Black, M., & Twiddy, S. (2007). The impact of incentives on survey participation and reports of intimate partner and sexual violence. Section on Survey Research Methods, 2954-2961.
Chacko, A., Wymbs, B. T., Chimiklis, A., Wymbs, F. A., & Pelham, W. E. (2012). Evaluating a comprehensive strategy to improve engagement to group-based behavioral parent training for high-risk families of children with ADHD. Journal of Abnormal Child Psychology, 40, 1351-1362.
Collins, R. L., Ellickson, P. L., Hays, R. D., & McCaffrey, D. F. (2000). Effects of incentive size and timing on response rates to a follow-up wave of a longitudinal mailed survey. Evaluation Review, 24(4), 347-363.
Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64(4), 413-428.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Mail and Internet surveys: The tailored design method, third edition. New York: John Wiley and Sons.
Dumas, J. E., & Albin, J. B. (1986). Parent training outcome: Does active parental involvement matter? Behaviour Research and Therapy, 24, 227-230.
Dumas, J. E., Begle, A. M., French, B., & Pearl, A. (2010). Effects of monetary incentives on engagement in the PACE parenting program. Journal of Clinical Child and Adolescent Psychology, 39(3), 302-313.
Dumas, J. E, Nissley-Tsiopinis, J., & Moreland, A. D. (2007). From intent to enrollment, attendance, and participation in preventive parenting groups. Journal of Child and Family Studies, 16, 1-26.
Fabiano, G. A. (2007). Father participation in behavioral parent training for ADHD: Review and recommendations for increasing inclusion and engagement. Journal of Family Psychology, 21, 683-693.
Goritz, A. S. (2006). Incentives in web studies: Methodological issues and a review. International Journal of Internet Science, 1(1), 58-70.
Graham, J. W., Taylor, B. J., & Cumsille, P. E. (2001). Planned missing-data designs in analysis of change. In L. M. Collins & A. Sayer (Ed.), New methods for the analysis of change (p. 335-353). Washington, DC: American Psychological Assocation.
Guyll, M., Spoth, R., & Redmond, C. (2003). The effects of incentives and research requirements on participation rates for a community-based preventive intervention research study. Journal of Primary Prevention, 24, 25-41.
Heinrichs, N. (2006). The effects of two different incentives on recruitment rates of families into a prevention program. Journal of Primary Prevention, 27, 345-365.
James, T. L. (1997). Results of the wave I incentive experiment in the 1996 Survey of Income and Program Participation. Report to the U.S. Bureau of the Census, 834-839.
Lorber, M. F., Xu, S., Slep, A. M., Bulling, L., & O’Leary, S. G. (2014). A new look at the psychometrics of the Parenting Scale through the lens of item response theory. Journal of Clinical Child and Adolescent Psychology, 43(4), 613-626.
Lundahl, B., Risser, H. J., & Lovejoy, M. C. (2006). A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26(1), 86-104.
Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training efficacy for child externalizing behavior problems—A meta-analytic review. Journal of Child Psychology and Psychiatry, 47, 99-111.
Singer, E., & Couper, M. P. (2008). Do incentives exert undue influence on survey participation? Experimental evidence. Journal of Empirical Research on Human Research Ethics, 3(3), 49-56.
Spoth, R., Goldberg, C., & Redmond, C. (1999). Engaging families in longitudinal preventive research: Discrete time survival analysis of socioeconomic and social-emotional risk factors. Journal of Consulting and Clinical Psychology, 67, 157-163.
Stahlschmidt, M. J., Threlfall, J., Seay, K. D., Lewis, E. M., & Kohl, P. L. (2013). Recruiting fathers to parenting programs: Advice from dads and fatherhood program providers. Children and Youth Services Review, 35(10), 1734-1741.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | T. Taylor |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |