Supporting Statement B
for
Study to Explore Educational Children’s Book in Pediatric Offices
New
Technical Monitor
Denise Levis, MA, PhD
Health Communication Specialist
Prevention Research Branch
National Center on Birth Defects and Developmental Disabilities
P: (404) 498-0237
Table of Contents
B.1. Respondent Universe and Sampling Methods
B.2. Procedures for the Collection of Information
B.3. Methods to Maximize Response Rate and Address Non-Response
B.4. Test of Procedures or Methods to be Undertaken
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
B. Collection of Information Employing Statistical Methods
B.1. Respondent Universe and Sampling Methods
Through a partnership with Reach out and Read (RoR), in Spring 2012 250 of RoR’s largest pediatric clinics each received 300 copies of Amazing Me for distribution to parents of 3-year-old children during well-child visits. RoR is a nonprofit organization that promotes early literacy and school readiness in pediatric exam rooms. RoR provides children’s books to families through their network of pediatric practices, most of whom serve low-income families. In participating RoR practices, parents receive a new book during each well-child visit from 6 months to 5 years of age, and health care providers often couple the book giveaway with advice for parents about reading to young children. Distribution of Amazing Me through RoR practices was used as a vehicle to target those at higher risk for developmental delays and disabilities: children insured by Medicaid and children from families with low incomes.1 According to RoR, in the majority of their clinics, 80% of the patients seen have household incomes at or below 200% of the current federal poverty level.
Data collection consists of a web survey and focus groups conducted with parents who receive the Amazing Me book from 6 primary care practices or clinics (3 RoR and 3 non-RoR) selected to participate in the study. The six practices will be selected from a convenience sample of clinics located in in the metropolitan areas surrounding Atlanta, Georgia and Washington, DC. The locations of Atlanta and Washington, DC were selected to provide geographic diversity and because of their proximity to CDC and Westat offices. Westat staff will need to make regular visits to selected clinics and conduct the planned parent focus groups at clinics sites. Thus, proximity to Westat offices helps save project resources.
Selection of Clinics
The 3 RoR practices will be selected from among 13 RoR practices (located in the metropolitan areas surrounding Atlanta, Georgia and Washington, DC) that have not received copies of the Amazing Me book to distribute. The 3 non-RoR practices will be selected from public/community health centers that provide services to children through state programs such as the PeachCare program in Georgia and Federally Qualified Health Centers. Federally Qualified Health Centers (FQHC) are public and private non-profit health care organizations that meet certain criteria under the Medicare and Medicaid Programs (respectively, Sections 1861(aa)(4) and 1905(l)(2)(B) of the Social Security Act and receive funds under the Health Center Program (Section 330 of the Public Health Service Act). The main purpose of the FQHC Program is to enhance the provision of primary care services in underserved urban and rural communities. Selecting non-RoR practices from among these types of clinics, will help ensure we select clinics that serve populations similar to those served by RoR practices, low-income families. As stated earlier, in the majority of RoR clinics, 80% of the patients seen have household incomes at or below 200% of the current federal poverty level.
Selection of Study Participants
Statistical methods will not be used to select participants for neither the parent web survey or the focus groups. Rather, we are relying on convenience sampling approaches for several reasons. First, because study participants are being recruited from clinic settings, HIPAA restrictions do not allow health centers to release contact information for their patients, without their consent. Thus, we must select study participants from among those who receive the Amazing Me book from their providers and choose to participate in the study. Second, because the purpose of our study is to gather feedback from parents who have received and read the book, we cannot post flyers or other promotional materials to recruit participants as the Amazing Me book will be distributed only to parents/guardians of 3-year-olds, soon to be 3-year-olds, or recently turned 4-year-olds (the book’s primary target audience). In addition, we seek to gather feedback from parents who have had ample time to read the book and the opportunity to bring the book and read it to their child. This represents a small segment of the larger clinic population from which study participants are being selected. However, as part of the clinic selection process, we will gather demographic data on each clinic’s patient population, which will give us some sense of the larger sample of parents/guardians of 3-year-olds, soon to be 3-year-olds, or recently turned 4-year-olds from whom study participants will be recruited.
Given that convenience samples are being used for all of the data collection efforts we discuss in this package, our goal is to gather as much information as we can about the population from which study participants have been selected and to document and discuss any biases we might find and how these might affect our findings and the conclusions we draw.
Below we discuss the process for recruiting parents for each data collection effort.
Parent Web Survey
Participants in the parent web survey will be recruited from the 6 primary health clinics (3 RoR and 3 non-RoR) selected to participate in the study, as described above. Eligible respondents must be an adult parent or legal guardian of a 3-year-old, soon to be 3-year-old, or recently turned 4-year-old seen at one of the 6 pediatric health clinics enrolled in the study and must have received a copy of the book, Amazing Me, during their child’s appointment at the clinic. Because the Amazing Me book is currently only available in English, all survey respondents must speak English as well. Recruitment of web survey participants will continue for the duration of the 6-month study period, with the goal of obtaining at least 300 responses to the survey.
Each clinic is expected to see as many as 150 parents/guardians over the 6-month study period. Thus, the respondent universe will consist of 900 parents/guardians of a 3-year-old, soon to be 3-year-old, or recently turned 4-year-old attending the selected pediatric or family health clinics.
We expect a mix of survey respondents with regards to race/ethnicity, age, clinic setting, and geographic locations since each clinic will receive the same number of books to distribute and we will work closely with each clinic to ensure that all 900 books are distributed so that the survey has the potential to be completed by as many as 900 parents. However, given the populations served by the 6 pediatric clinics participating in the study, we expect less of a mix with regards to sex and income level. We anticipate that most respondents will be female (mothers overwhelmingly bear the responsibility of taking children to doctor’s appointments2) and most will be lower income as (according to RoR administrators) 80% of the patients seen in RoR clinics have incomes at or below 200% of the current federal poverty level and a similar population will be seen in the comparison (non-RoR) clinics. As previously stated, we are relying on convenience sampling to recruit study participants. We cannot gather any contact or background information on participants from the clinics (HIPAA regulations) thus we are not able to do a more targeted recruitment of survey respondents that would help ensure a more mixed or balanced sample. Thus, ultimate sample of survey respondents will be driven in large part by the demographics of the populations served by the 6 pediatric health clinics participating in the study and the parents who chose to participate in the survey. We will gather information on the patient demographics of each clinic as part of the clinic selection process so that we can select an overall sample of clinics that serve a mix of patient populations with regards to demographic variables such as race/ethnicity, so that we can obtain a survey sample that is mixed in this regard as well.
Parents/guardians will voluntarily access the web survey by logging onto a URL address provided on a sticker affixed to the inside cover of each Amazing Me book. The sticker will include the following information:
An invitation to provide feedback on the Amazing Me book by participating in a web survey, along with incentive information )
A unique ID for which the first 2 digits are linked to the clinic (i.e., 01=RoR clinic 1, 02=Non-RoR clinic 1, etc.).
A toll-free project number at Westat to call for questions.
Screening questions located at the start of the survey will confirm eligibility and the survey will terminate for those found ineligible due to age (i.e., less than 18 years of age), age of their child (i.e., their child is under the age of 33 months or older than 50 months) or having not received an Amazing Me book. Eligible participants will proceed to complete the survey.
Recruitment of web survey participants will continue for the duration of the 6-month study period, with the goal of obtaining at least 300 responses to the survey. We expect a mix of survey respondents with regards to race/ethnicity and age. However, given the populations served by the 6 pediatric clinics participating in the study, we expect less of a mix with regards to sex and income level. We anticipate that most respondents will be female (mothers overwhelmingly bear the responsibility of taking children to doctor’s appointments3) and most will be lower income as (according to RoR administrators) 80% of the patients seen in RoR clinics have incomes at or below 200% of the current federal poverty level and a similar population will be seen in the comparison (non-RoR) clinics. To the extent practical, an attempt will be made to obtain participants at a comparable mix of educational levels and racial/ethnic backgrounds. However, this will be driven in large part by the demographics of the populations served by the 6 pediatric health clinics. The following table presents the sample size for survey respondents in each of the 6 clinics.
Sample |
Total Reponses (N=900) |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 1. |
n=150 |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 2. |
n=150 |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 3. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 1. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 2. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 3. |
n=150 |
Parent (or Guardian) Focus Groups
We chose focus group methodology for this component of the study because of the richness of information provided. As a data collection technique, focus groups capitalize on the interaction within a group to elicit “real life” responses from participants. The structured group process frequently provides very rich, detailed data as the opinions heard from one participant may stimulate the responses of others. In addition, a skilled moderator can follow up on or probe certain tangents or views that emerge but could not be planned for or anticipated in the design of the discussion guide. This unique aspect of focus group methodology often yields new information and/or additional nuances on existing information that could not be gathered using more quantitative techniques. The group dynamic fostered by this approach is also extremely helpful in eliciting information on the prevailing norms, opinions, and values of the target audience and provides the opportunity to explore new, emerging, or divergent opinions to increase understanding of an audience. Focus groups are particularly useful during the early phases of program development when it is essential to gather more in-depth information about the target audience and/or obtain audience feedback on potential interventions. In this case, the parent focus groups will allow parents to share their experiences regarding reading to their child; describe the process of receiving Amazing Me, taking it home to read, and their opinions about the book; and demonstrate their knowledge, attitudes, and beliefs about developmental milestones. The overall goal for these focus groups will be to provide CDC with a more in-depth look at parents’ perspectives of the book and their experiences with it. This will provide CDC important information on how well the book is meeting intended goals and provide useful guidance for developing and disseminating the books in settings beyond RoR practices.
Typically, focus groups rely on purposeful sampling techniques; that is, relatively homogenous groups of people with something in common that is relevant to the topic of study. For this project, focus group participants will be recruited from among the web survey respondents who opt to participate in a more in-depth discussion about their experiences with the Amazing Me book.
Focus groups will consist of six (6) in-depth discussions with parents/guardians about their experience with the Amazing Me book. Focus group participants will be recruited from among those who completed the web survey and have voluntarily submitted their contact information for screening and recruitment into the focus groups. At the end of the web survey, respondents will be asked if they would like to participate in a 1-hour focus group. Parents/guardians who are interested in participating in the focus groups will be automatically re-directed to a separate follow-up contact survey in which they will be asked to provide their contact information (name, email address, and cell phone number) so that staff from Westat can contact them for scheduling.
Westat will contact potential focus group participants and conduct a short screening instrument to ensure eligibility. To be eligible to participate in the focus groups, participants must be an adult parent/guardian of a 3-year-old, soon to be 3-year-old, or recently turned 4-year-old seen at one of the 6 pediatric health clinics enrolled in the study; have received a copy of the book, Amazing Me, during their child’s appointment at the clinic, have read the book; do not have a child with special needs; and do not have professional experience with child psychology, child development, or health care delivery.
Eligible participants will be invited to attend the 1-hour focus group discussion and provide in-person feedback on the Amazing Me book. The project team will schedule 6 focus groups (1 for each clinic). Each group will have 9 participants, for a total of 54 respondents in the focus group phase. To ensure that nine participants are in each of the 6 focus groups, 10 participants will be recruited per group. The following table presents the design and sample for the focus groups.
Table 2: Parent Focus Group Recruitment and Sample
|
Location |
|
|
|
DC metropolitan area |
Atlanta metropolitan area |
Total |
ROR Clinic Settings |
|||
|
2 groups (n=18 parents, 9 in each group) |
1 group (n=9 parents) |
3 groups (n=27 parents, 9 in each group) |
Non-RoR Clinic Settings |
|||
|
2 groups (n=18 parents, 9 in each group) |
1 group (n=9 parents) |
3 groups (n=27 parents, 9 in each group) |
Total |
4 groups (n=36 parents, 9 in each group) |
2 groups (n=18 parents, 9 in each group) |
6 groups (N=54 parents, 9 in each group) |
B.2. Procedures for the Collection of Information
Parent Web Survey
Parent web survey data will be collected using the web survey tool, SurveyMonkey. The survey (including the screener questions and follow-up contact survey) is estimated to take 5 minutes to complete. A copy of the screening and survey instrument can be found in Attachment 3.
Parents/guardians will voluntarily access the web survey by logging onto a URL address or by calling a toll-free number provided on a sticker affixed to the inside cover of each Amazing Me book. The sticker will include the following information:
An invitation to provide feedback on the Amazing Me book by participating in a web survey, along with incentive information )
Web survey URL
A unique ID for which the first 2 digits are linked to the clinic (i.e., 01=RoR clinic 1, 02=Non-RoR clinic 1, etc.).
A toll-free project number at Westat to call for questions or to complete the survey over the phone.
Screening questions located at the start of the survey will confirm eligibility and the survey will terminate for those found ineligible due to age (i.e., less than 18 years of age), age of their child (i.e., their child is under the age of 33 months or older than 50 months) or having not received an Amazing Me book. Eligible participants will proceed to complete the survey. The web survey will be conducted in English because the Amazing Me books is currently only available in English; a Spanish language adaptation of the book is in progress.
The web survey will (1) confirm receipt of the Amazing Me book and the setting in which the book was received, (2) gather feedback about parents’ experiences taking the book home and reading it with their child(ren), and (3) assess the influence of the book on parents’ awareness, attitudes, and self-efficacy regarding monitoring developmental milestones.
Parent (or Guardian) Focus Groups
Focus groups will consist of six (6) in-depth discussions with parents/guardians about their experiences with the Amazing Me book. Participants will be recruited from among those who completed the web survey and have voluntarily submitted their contact information for screening and recruitment into the focus groups. At the end of the web survey, respondents will be asked if they would like to participate in a 1-hour focus group. Parents/guardians who are interested in participating in the focus groups will be automatically re-directed to a separate follow-up contact survey in which they will be asked to provide their contact information (name, email address, and cell phone number) so that staff from Westat can contact them for further screening and scheduling.
To be eligible to participate in the focus groups, participants must be an adult parent/guardian of a 3-year-old, soon to be 3-year-old, or recently turned 4-year-old seen at one of the 6 pediatric health clinics enrolled in the study; have received a copy of the book, Amazing Me, during their child’s appointment at the clinic, have read the book; do not have a child with special needs; and do not have professional experience with child psychology, child development, or health care delivery. The project team will schedule 6 focus groups (1 for each clinic). Each group will have 9 participants, for a total of 54 respondents in the focus group phase. To ensure that nine participants are in each of the 6 focus groups, 10 participants will be recruited per group.
Eligible participants will be invited to attend the 1-hour focus group discussion and provide in-person feedback on the Amazing Me book. Participants will be asked to arrive at the focus group facility 30 minutes prior to the start of the focus group and reminder calls will be made the day before and the day of each group. Participants will be given a consent form to sign (see Attachment 7). Then the participants in each group will be gathered in a room with a trained moderator and a notetaker. The moderator will explain the study, inform the group of audiotaping, and lead a discussion using the moderator’s guide (see Attachment 6). The moderator’s guide will gather information from parents about (1) their experiences reading the Amazing Me book to their child; (2) the process of receiving Amazing Me, taking it home to read, and their opinions about the book; and (3) their knowledge, attitudes, and beliefs about developmental milestones.
Each focus group will last approximately 60 minutes. Responses will be collected by audiotape and notes taken by the notetaker. After each group, the tapes will be transcribed for qualitative analysis.
B.3. Methods to Maximize Response Rates and Deal With Nonresponse
The web survey will not use a probability sample to select respondents. Except for offering financial inducement for participation, the project team does not plan to use any special procedures to convert those who, after being informed about research objectives and participation (from their providers and via the stickers located on the inside cover of the Amazing Me books), decline to participate in the web survey. However, the project team will send reminder emails to clinics to continue disseminating the books to parents and informing them of the web survey. Also, the web survey will be available for the duration of the study period (6 months) as well as an additional month, (total of 28 weeks) to allow parents/guardians who receive the Amazing Me book adequate time to participate.
For the focus groups, the project team will make efforts to schedule the groups at times and locations that are convenient for respondents. For example, evening hours are often most convenient for collecting information from parents and guardians who work and to the extent possible. In addition, we will seek focus group locations at or near each of the clinics participating in the study, to reduce travel burden for participants. Westat staff will make reminder calls the day before and day of each focus group.
B. 4. Test of Procedures and Methods to be Undertaken
The parent web survey was programmed and tested with 6 individuals who met the screening criteria for the larger web survey; this was conducted in order to ensure that all questions were understood as intended and to assess the timing of the survey. The final parent web survey instrument and screenshots can be found in Attachments 3 and 3a.
Survey responses will be summarized by question, using frequency tables to present aggregate response to each question and descriptive statistics (e.g., mean, range, etc.) when appropriate. If the response (sample) is large enough, we will also compare survey responses between RoR and non-RoR clinics for statistically significant differences in responses. For example, respondent feedback on the book’s content or their experiences receiving the Amazing Me book could be compared between those who received the book in RoR and non-RoR clinic settings to determine if a statistically significant difference exists (at the conventional 5% level) between the two groups. As previously discussed, RoR is a nonprofit organization that promotes early literacy and school readiness in pediatric exam rooms. In participating RoR practices, parents receive a new book during each well-child visit from 6 months to 5 years of age, and health care providers often couple the book giveaway with advice for parents about reading to young children. Although non-RoR practices will be selected to match RoR practices in terms of the income levels of families they serve and clinic size (i.e., they see at least 100-150 families for 3-year old well child visits over a 6-month period), non-RoR clinics will have no prior experience in distributing children’s books to their patients. As such, parents in non-RoR clinics will have no such prior experience either. One important question for this study is if parents in non-RoR practices will be receptive to receiving the Amazing Me book as part of their child’s well-child exam and if overall they are receptive to the book. Thus, a comparison of parent feedback between RoR and non-RoR settings will help CDC can make an informed decision about whether to expand distribution of Amazing Me beyond RoR practices.
The qualitative research being conducted through the parent focus groups will follow standard focus group discussion procedures. The focus group discussion guide was tested in a pilot group discussion with 4 individuals who met the screening criteria for participating in the study. This pilot testing was conducted to ensure that the questions were understood as intended by the target audience and to estimate time requirements. The final parent focus group screener and discussion guide can be found in Attachments 5 and 6, respectively.
At the conclusion of the focus groups, notes and audio recordings from the focus groups will be analyzed for common themes and divergent viewpoints among and between audiences. Qualitative analytical software, such as NVivo will be used to facilitate the analysis. As previously discussed, focus group research is qualitative rather than quantitative. As a result, the value of focus group findings is dependent on researchers examining results systematically. Although the procedures used to analyze focus group data are not standardized, there are recommended analysis processes in place. Westat researchers use a notes and transcript-based analysis process similar to that recommended by Krueger4. First, all data will be organized and codes developed after reviewing the data for salient themes discussed across the groups. Outliers, responses given by only one or two participants, will be coded as well. The analysis process then commences with the following three steps:
1) A systematic content analysis was performed that involves the identification, labeling, and categorization of the data. This analysis reviews records from all groups (transcripts, topline reports, observer notes) to identify trends across groups.
These groups are further examined through thematic analysis to determine if certain themes may emerge from the patterns. These themes may be articulated directly by the participants or identified by the study team. In addition to assisting with the identification of patterns and themes in the data, content and thematic analyses also allow large amounts of collected data to be reduced or distilled into a more easily studied and understood format.
A logical analysis was then performed by looking for patterns of difference, as well as similarities that may emerge from cross-classifying the data. This step is helpful in identifying patterns or themes that are not immediately obvious, while also taking into consideration outlier opinions.
A final report of the focus group findings will be completed within 15 months after OMB approval. Findings will be used to inform the development of clinic engagement and book distribution procedures for future CDC efforts to engage pediatric clinics in sharing the Amazing Me book with their patients.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The individuals consulted on technical and statistical issues related to data collection are listed below. The data will be analyzed by the study contractor, Westat, Inc.
CDC collaborators:
Denise Levis, MA, PhD
Health Communication Specialist
Prevention Research Branch, NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-0237
Email: [email protected]
Katie Kilker-Green, MPH, CHES
Health Communication Specialist
Learn
the Signs. Act Early. Campaign
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-3872
Email: [email protected]
Rebecca B. Wolf, MA
Team
Leader, Learn the Signs. Act Early. Campaign
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30329
Phone: 404-498-3951
Email [email protected]
Jessica Drennan, MSW
ORISE Fellow
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-3832
Email: [email protected]
Westat Collaborators:
Erika Reed-Gross, MHS
Senior Study Director
Westat
Rachel Gaddes, MPH
Task Leader
Westat
Erika Bonilla, MS
Task Leader
Westat
Daifeng Han, MS
Senior Statistician
Westat
1 Boyle, C.A., Boulet, S., Schieve, L., Cohen, R.A., Blumberg, S.J., Yeargin-Allsopp, M., Visser, S., Kogan, M.D. 2011 Trends in the Prevalence of Developmental Disabilities in US Children, 1997–2008. Pediatrics. 2011 Available at http://pediatrics.aappublications.org/content/early/2011/05/19/peds.2010-2989.full.pdf+html
2 Kaiser Family Foundation. 2005. Women and Health Care: A National Profile. Available at http://www.kff.org/womenshealth/upload/women-and-health-care-a-national-profile-key-findings-from-the-kaiser-women-s-health-survey.pdf
3 Kaiser Family Foundation. 2005. Women and Health Care: A National Profile. Available at http://www.kff.org/womenshealth/upload/women-and-health-care-a-national-profile-key-findings-from-the-kaiser-women-s-health-survey.pdf
4 Krueger, R.A. (1994) Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage Publications.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | CDC User |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |