Supporting Statement A
for
Study to Explore Educational Children’s Book in Pediatric Offices
New
Technical Monitor
Denise Levis, MA, PhD
Health Communication Specialist
Prevention Research Branch
National Center on Birth Defects and Developmental Disabilities
P: (404) 498-0237
Table of Contents
Justification
A.1. Circumstances Making the Collection of Information Necessary
A.2. Purpose and Use of Information Collection
A.3. Use of Improved Information Technology and Burden Reduction
A.4. Efforts to Identify Duplication and Use of Similar Information
A.5. Impact on Small Businesses or Other Small Entities
A.6. Consequences of Collecting the Information Less Frequently
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A.9. Explanation of Any Payment or Gift to Respondents
A.10. Assurance of Confidentiality Provided to Respondents
A.11. Justification for Sensitive Questions
A.12. Estimates of Annualized Burden Hours and Costs
A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
A.14. Annualized Cost to the Government
A.15. Explanation for Program Changes or Adjustments
A.16. Plans for Tabulation and Publication and Project Time Schedule
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions
List of Attachments
Attachment 1: Applicable Laws or Regulations
Attachment 2: 60-Day Federal Register Notice
Attachment 3: Amazing Me Parent Web Survey Screener/Web Survey(Word)
Attachment 3a: Amazing Me Web Survey Screener/Web Survey (screenshots)
Attachment 4: Amazing Me Parent Follow-up Contact Survey (Word)
Attachment 4a: Amazing Me Parent Follow-Up Contact Survey (screenshots)
Attachment 5: Amazing Me Parent Focus Group Screener
Attachment 6: Amazing Me Parent Focus Group Moderator’s Guide
Attachment 7: Amazing Me Parent Focus Group Informed Consent
Supplemental Forms:
Supplemental 1: Amazing Me IRB Approval Letter
Supplemental 2: Amazing Me IRB Amendment Form Approval
Supplemental 3: Amazing Me. It’s Busy Being 3! Book
This Information Collection Request is submitted under the classification “new” request. The length of data collection requested for OMB-PRA approval is one year. The National Center on Birth Defects and Developmental Disabilities (NCBDDD) is making this request as authorized by the Public Health Service Act, Title 42 United States Code—The Public Health and Welfare, Chapter 6A—Public Health Service, Subchapter II—General Powers and Duties, Part A—Research and Investigations (see Public Health Service Act, 42 USC Sec. 241 Attachment 1).
Developmental disabilities have reached epidemic proportions in the U.S., with approximately 17 percent of children experiencing developmental delays.1 Related disabilities are wide-ranging from mild to more severe chronic conditions such as autism, intellectual disability, or attention-deficient/hyperactivity disorder.2 Impairment in physical, learning, language, or behavior areas can have a lifetime impact on everyday activities of life for a child and into adulthood.3
Despite the fact that most children affected will show mild developmental delays by the age of two, less than half will be identified before they start school.4 As a result, opportunities for early intervention are missed and future childhood development is compromised. Interventions before school age can have a major impact on learning new skills and the need for costly measures in future years.5
Research has shown that parents can be reliable sources of information about their children’s development. Several studies have found that parents’ concerns about their children’s development are generally valid and predictive of developmental delays.6, 7, 8 These studies suggest that efforts to raise parental awareness of developmental milestones can increase the likelihood that children with developmental disabilities are identified early and connected with appropriate services and support.
Amazing Me. It’s Busy Being 3! a book for parents of 3-year olds provides an opportunity to not only increase parents’ awareness of developmental milestones but to actively engage them in the assessment and monitoring of their child’s development. The book uses an engaging and familiar format, a children’s picture book, to reach parents and encourage them to act early if they have concerns. The book is currently available in English only but a Spanish-language version is currently in adaptation.
Through a partnership with Reach out and Read (RoR), in Spring 2012 250 of RoR’s largest pediatric clinics each received 300 copies of Amazing Me for distribution to parents of 3-year-old children during well-child visits. RoR is a nonprofit organization that promotes early literacy and school readiness in pediatric exam rooms. RoR provides children’s books to families through their network of pediatric practices, most of whom serve low-income families. In participating RoR practices, parents receive a new book during each well-child visit from 6 months to 5 years of age, and health care providers often couple the book giveaway with advice for parents about reading to young children. Distribution of Amazing Me through RoR practices was used as a vehicle to target those at higher risk for developmental delays and disabilities: children insured by Medicaid and children from families with low incomes.9 According to RoR, in the majority of their clinics, 80% of the patients seen have household incomes at or below 200% of the current federal poverty level.
Preliminary data gathered from a web survey of RoR clinical staff indicates that clinical staff are not only receptive to but supportive of the Amazing Me book. Data from this research will provide information on whether the book distribution protocol outlined by RoR is a good fit for CDC children’s books with health messages for parents. This feedback from RoR clinics is extremely valuable and will help inform future distribution approaches involving RoR clinics. However, the web survey of RoR clinical staff does not provide information from the book’s target audience--parents.
Numerous studies have documented the success of children’s literature in promoting childhood literacy and encouraging reading at home, including several studies documenting the success of the RoR Program in this area. 10,11 However, researchers have only recently begun to document the influence of children’s literature on parent behaviors, beyond literacy promotion.12, 13, 14, 15 These studies examined the influence of children’s books to address topics such as nutrition and healthy eating, parenting practices, injury prevention, and money management with parents. In each study, researchers found that parents/caregivers not only read the books with their children, but were receptive to messages conveyed in the books and reported changes in their behaviors as a result. These findings certainly highlight the potential of using children’s books, such as Amazing Me, to influence parent behaviors. Yet the dearth of such studies also underscores the importance of evaluating Amazing Me to determine if children’s literature is an effective approach for reaching parents and caregivers with health related messages. Additionally, if CDC wishes to expand book distribution, it will be important to gather data on parents’ experiences receiving the Amazing Me book as part of a pediatric visit, and what kind of influence, if any, the book has had on their knowledge, attitudes, and beliefs about developmental milestones.
To this end, working closely with our contractor (Westat) and RoR, we will identify and recruit six pediatric practices (3 RoR and 3 non-RoR) to take part in a study. The study seeks to gather feedback about Amazing Me from parents/guardians of 3-year-olds, soon to be 3-year-olds, or recently turned 4-year-olds attending the selected pediatric and family health clinics. Specifically, we will gather information about 1) their experiences receiving the book as part of a pediatric visit, and 2) the influence of the book on their awareness, attitudes, and self-efficacy regarding monitoring developmental milestones. We are requesting approval for a data collection involving a web survey of parents along with follow-up focus groups with parents to gather more in-depth information about the Amazing Me book.
The data collection involves 1) a web survey (Attachments 3 and 3a) and 2) focus groups (Attachment 6) with parents/guardians of 3-year olds, soon to be 3-year-olds, or recently turned 4-year-olds recruited from six pediatric practices (3 RoR and 3 non-RoR) selected to participate in the study. Both the web survey and the focus groups will be conducted in English only as a Spanish-language version of the Amazing Me book is not yet available and is currently in adaptation.
The six pediatric practices will be selected from a convenience sample of clinics located in the metropolitan areas surrounding Atlanta, Georgia and Washington, DC. The locations of Atlanta and Washington, DC were selected to provide geographic diversity and because of their proximity to CDC and Westat offices. Westat staff will need to make regular visits to selected clinics and conduct the planned parent focus groups at clinic sites. Thus, proximity to Westat offices helps save project resources.
The 3 RoR practices will be selected from among 13 RoR practices (located in the metropolitan areas surrounding Atlanta, Georgia and Washington, DC) that have not received copies of the Amazing Me book to distribute. The 3 non-RoR practices will be selected from public/community health centers that provide services to children through state programs such as the PeachCare program in Georgia16 and Federally Qualified Health Centers located in the metropolitan areas surrounding Atlanta, Georgia and Washington, DC as well. Federally Qualified Health Centers (FQHC) are public and private non-profit health care organizations that meet certain criteria under the Medicare and Medicaid Programs (respectively, Sections 1861(aa)(4) and 1905(l)(2)(B) of the Social Security Act and receive funds under the Health Center Program (Section 330 of the Public Health Service Act).17 The main purpose of the FQHC Program is to enhance the provision of primary care services in underserved urban and rural communities. Selecting non-RoR practices from among these types of clinics, will help ensure we select clinics that serve populations similar to those served by RoR practices, low-income families. As stated earlier, in the majority of RoR clinics, 80% of the patients seen have household incomes at or below 200% of the current federal poverty level.
In addition to serving low-income populations, selected clinics must see at least 100-150 families for 3-year old well child visits over a 6-month period as each selected practice will receive 150 Amazing Me books to distribute to parents/guardians over the 6-month study period. Using the RoR program’s current distribution protocol, clinical staff will give the book to parents, in the exam room, as part of the well-child visit. One representative from each of the 6 practices (i.e., one administrative/program coordinator or one provider) will serve as our point of contact and will act as the distribution coordinator during the 6-month study period.
Parent Web Survey
Participants will be recruited for the web survey when they receive a copy of the Amazing Me book from the pediatrician during their child’s 3-year well child visit. . Parents/guardians can access the web survey by logging onto a URL address provided on a sticker affixed to the inside cover of each Amazing Me book. Respondents can access the survey software, SurveyMonkey, on any handheld mobile device that is connected to the Internet as well as a personal computer.
We anticipate that the majority of respondents will complete the survey themselves on the web. According to the Pew Internet & American Life Project conducted by the Pew Research Center, 85% of American adults use the Internet. A growing number of American adults whose household income is $30K or below (76%) and those living in urban or suburban areas (86%) are becoming users of the Internet.18 Further, as of May 2013, research showed that over half of American adults (56%) have a smartphone.19 Smartphone adoption is gaining momentum among several demographic groups, including non-whites and adults under the age of 45.20 We realize however, that there may be some respondents who do not have access to a computer and/or the Internet or who simply may prefer to do the survey over the telephone with an interviewer. In those instances, respondents have the option of calling a toll-free number that will also be provided on the sticker that contains the link to the survey. Interested participants can call this number and leave a message on a pre-recorded voicemail with instructions for calling them back at a date and time most convenient for completing the survey over the phone.
Each Amazing Me book will have a sticker affixed to the inside of the book that encourages parents to provide feedback about the book by accessing the link to the web survey or calling the tollfree number to complete the survey via telephone. The sticker will be bright in color and have a message similar to: “Tell us what you think of this book and earn a gift card for doing so!” The sticker will include the following information:
An invitation to provide feedback on the Amazing Me book by participating in a web survey
Web survey URL
A unique ID for which the first 2 digits are linked to the clinic (i.e., 01=RoR clinic 1, 02=Non-RoR clinic 1, etc.).
A Toll-free project number at Westat to call for questions or to complete the survey over the telephone
Survey ID numbers will be loaded in SurveyMonkey and can only be used once to prevent multiple survey responses from the same participant or responses from ineligible parties. The survey ID will also be used to track the number of survey responses per clinic. Respondents interested in participating in the web survey will go to the URL or call the toll-free number provided on the sticker to access the screener and web survey. Eligible respondents must be a parent or legal guardian of the 3-year-old, soon to be 3-year-old, or recently turned 4-year-old and must have received a copy of the book, Amazing Me. The web survey will take less than 5 minutes to complete. Web survey data will be collected through a secure website hosted by SurveyMonkey.SurveyMonkey is compatible on any device that has internet access, including smartphones.
The parent web survey will (1) confirm receipt of the Amazing Me book and the setting in which the book was received, (2) gather feedback about parents’ experiences taking the book home and reading it with their child(ren), and (3) assess the influence of the book on parents’ awareness, attitudes, and self-efficacy regarding monitoring developmental milestones.
Our contractor, Westat, will be responsible for setting up, programming, and maintaining the survey website for the duration of the field period. Survey data will be delivered to Westat project staff for analysis purposes and then sent to CDC (with identifiers removed) at the close of the project. Survey respondents will receive a $5 Target or Wal-Mart gift card for completing the survey. In order to receive this incentive, participants will need to provide their email address. If participants do not have an email but would still like to receive the incentive, they will have the option of providing a physical address for mailing. Participants will be asked to provide an email address or physical mailing address at the end of the survey, where the gift card can be sent. However respondents who do not wish to share this information do not have to and therefore will not receive the gift card. Web survey data will be retained for the length of the project and then destroyed under Westat’s policies and procedures for data retention and destruction. See Table 1 below for parent web survey research design.
Table A.1.A. Parent Web Survey Research Design
Target Audiences |
Total Reponses (N=900) |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 1. |
n=150 |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 2. |
n=150 |
Parents of 3-year-olds who received Amazing Me at RoR Clinic 3. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 1. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 2. |
n=150 |
Parents of 3-year-olds who received Amazing Me at Non-RoR Clinic 3. |
n=150 |
The web survey will also be used to recruit parents for the follow-up focus groups. At the end of the web survey, respondents will be asked if they would like to participate in a 1-hour focus group. Parents/guardians who are interested in participating in the focus groups will be automatically re-directed to a separate, follow-up contact survey (see Attachments 4 and 4a) in which they will be asked to provide their contact information (name, email address, and cell phone number) so that staff from Westat can contact them for scheduling. The contact information gathered in this separate survey will be stored in an independent database and not linked to responses given in the web survey. The contact information gathered will only be used for focus group screening and scheduling purposes and will be destroyed at the end of the project.
Parent (or Guardian) Focus Groups
Parent focus groups (Attachment 6) will consist of six (6) in-depth discussions about parents’ experience with the Amazing Me book. Two focus groups will be held in the metropolitan areas surrounding Atlanta, Georgia and four groups will be held in the metropolitan areas surrounding Washington, DC. The focus groups will allow participants an opportunity to (1) share their experiences regarding reading to their child; (2) describe the process of receiving Amazing Me, taking it home to read, and their opinions about the book; and (3) to demonstrate their knowledge, attitudes, and beliefs about developmental milestones. The overall goal for these focus groups will be to provide researchers with a more in-depth look at parents’ perspectives of the book and their experiences with it. This will provide CDC important information on how well the book is meeting intended goals and provide useful guidance for developing and disseminating books for other age groups. Target audiences for the parent focus groups will consist of web survey respondents who opt to participate in a more in-depth discussion about their experiences with the book.
Westat will contact potential focus group participants and conduct a short screener (Attachment 5) to ensure eligibility. Focus group participants are eligible if they indicate they 1) have a child almost 3, 3 years old, or recently turned 4; 2) do not have a child with special needs; 3) participated in the parent web survey; 4) have in fact read the book; and 5) have no professional experience with child psychology or development. Westat will work with community-based organizations (e.g., churches, libraries, etc.) to arrange for a convenient space to hold the focus groups. The focus groups will be conducted with six to nine participants in each group and will last no more than 60 minutes. Participants will be asked to arrive early to read through and sign a consent form (See Attachment 4). Participants will receive $40 as a token of appreciation for their participation. See Table 2 for the focus group research design.
Table A.1.B. Parent Focus Groups Research Design
|
Location |
|
|
|
DC metropolitan area |
Atlanta metropolitan area |
Total |
ROR Clinic Settings |
|||
|
2 groups (n=18 parents, 9 in each group) |
1 group (n=9 parents) |
3 groups (n=27 parents, 9 in each group) |
Non-RoR Clinic Settings |
|||
|
2 groups (n=18 parents, 9 in each group) |
1 group (n=9 parents) |
3 groups (n=27 parents, 9 in each group) |
Total |
4 groups (n=36 parents, 9 in each group) |
2 groups (n=18 parents, 9 in each group) |
6 groups (N=54 parents, 9 in each group) |
The focus group data will be collected via the use of trained moderators and a structured moderator’s guide to ensure that consistent data are collected across the groups (see Attachment 6). Upon completion of each focus group, audiotapes and transcripts will be used to assist with report writing. Participant identifying information will be removed from the notes and transcripts before they are analyzed. All information gathered will be securely stored and maintained for the length of the project. Findings from the parent focus groups will provide CDC with valuable feedback on, parents’ experiences with the book, whether the book increases their knowledge, attitudes and beliefs about developmental milestones, and how parents feel about the book overall ..
The data collection system includes:
Amazing Me Parent Web Survey (Attachments 3 and 3a)
Amazing Me Parent Follow-up Contact Survey (Attachments 4 and 4a)
Amazing Me Focus Group Screener (Attachment 5)
Amazing Me Focus Group Moderator’s Guide (Attachment 6)
Amazing Me Informed Consent (Attachment 7)
Items of Information to Be Collected
Web survey participants will include parents of 3-year-old, soon to be 3, or recently turned 4-year old children who received a copy of the Amazing Me book at the year 3 well child visit at one of the six clinics participating in our study. During the screening process for the web survey, eligible participants will be required to confirm that they 1) are 18 years of age or older, 2) have a child age 3, soon to be 3 years old, or recently turned 4, and 3) have in fact read the book, (See Attachment 3). Participants’ names will not be collected during the web survey. However, at the end of the web survey, participants will be asked a number of demographic items, including age range, gender, race/ethnicity, education, and income. In order to receive their $5 gift card incentive for completing the web survey, they will however need to voluntarily provide their email address (to receive the gift card via email) or name and a mailing address to receive the incentive via postal mail. This will be optional, as they can decline the incentive if they don’t want to share their email or mailing address. Also, participants who wish to receive the incentive at their physical address will be re-directed to a separate survey in which they can provide this information. For those interested in participating in a group discussion to provide more in-depth feedback on the book, they will be re-directed to a separate follow-up contact web survey that will collect their name, email, and cell phone number. Only those who indicate that they are interested in participating will be asked to provide this information for further follow-up at a later date. All contact information collected in the follow-up contact web survey will be used either to email promised incentive payments or follow-up for focus group recruitment. However, no Information in Identifiable Form (IIF) will be included in the final dataset Westat sends to CDC. The web survey questions have been carefully crafted to collect the following information:
Process by which book was received
Book experience at home
Knowledge about book
Appeal of book format and content
Demographic information
All focus group participants will also be web survey participants. At the end of the web survey, participants will be asked to indicate if they wish to participate in a group discussion to provide more in-depth feedback on the book. If so, they will provide contact information that will be stored in a separate database that Westat will use later for focus group recruitment and screening. During the screening process for the focus groups, Westat will call and/or email all participants who indicated an interest in participating during the web survey and ask a few questions to determine whether they meet the screening criteria (see Attachment 5). Parents must confirm that they 1) have a child almost 3, 3 years old, or recently turned 4; 2) do not have a child with special needs; 3) participated in the parent web survey; 4) have in fact read the book; and 5) have no professional experience with child psychology or development to be eligible for the focus groups. Contact information collected during the web survey will be used by Westat to schedule participants for the focus groups, and mail out confirmation letters/emails verifying the person’s participation and provide the exact date, time and location of the focus group. Westat will conduct one focus group with parents from each of the six clinics recruited to participate in this project. Participants will be asked to arrive at the focus group location 30 minutes prior to the start of the focus group to read through and sign a consent form (see Attachment 7). Participants will only provide their first name during the focus group discussions. The focus group discussion guide (See Attachment 6) will collect the following information:
Parents’ experiences with reading to their child in the home
Process of taking Amazing Me home, reading it, and opinions about it
Impact of Amazing Me on knowledge, attitudes, and beliefs about developmental milestones
Experience receiving the book from the pediatrician’s office
No website content directed at children under 13 years of age is involved in this information collection request.
The CDC’s NCBDDD will fund this research effort to assess the influence of the children’s book, Amazing Me in pediatric office settings where the book will be distributed to parents of 3-year-old children, soon to be 3-year olds, and recently turned 4-year olds. The findings will help CDC determine if distribution of the book through RoR/pediatric office settings is an appropriate approach for delivering information about child development to parents and if children’s literature is an appropriate approach for reaching parents and caregivers with health related messages. Though findings from previous research studies have highlighted the potential of using children’s books, such as Amazing Me, to influence parent behaviors, there has not yet been an evaluation of the Amazing Me book. In addition, there has not been an evaluation of the distribution process through the RoR system. Practices that collaborate with RoR typically tend to serve lower income families and those at higher risk for developmental delays and disabilities. If the requested data collection was not conducted, CDC would be unable to 1) evaluate the effectiveness of the Amazing Me book in reaching and raising awareness of developmental milestones in parents and actively engaging them in assessing and monitoring their child’s development and 2) evaluate the RoR book distribution model with both RoR and Non-RoR practices. The collection of this information will enable CDC to determine if this method of reaching parents through a children’s book is effective and whether more books like Amazing Me for other age groups should be developed and if the RoR vehicle is an effective means to reach low-income and at-risk families.
Why the information is being collected
The overall purpose of this data collection effort is to assist CDC in their efforts to evaluate the influence of the Amazing Me book in six pediatric practices in the metropolitan DC, and Atlanta, GA areas. Developmental disabilities have reached epidemic proportions in the U.S., with approximately 17 percent of children experiencing developmental delays.21 Research also indicates that parents can be reliable sources of information about their children’s development.22,23,24 To piggyback off this, CDC has aimed to develop a children’s book that parents can read to their children that will teach parents how to monitor developmental milestones in their 3-year-old while at the same time entertaining the child with a story. Despite the fact that most children affected with developmental disabilities will show mild developmental delays by the age of two, less than half will be identified before they start school.25,26,27 As a result, opportunities for early intervention are missed and future childhood development is compromised. CDC’s implementation of the Amazing Me book in pediatric offices provides parents who are a constant in the lives of children the tools necessary to monitor developmental milestones in their children. The hope is that potential developmental problems are identified early and can be addressed and resolved prior to children starting school and suffering from further developmental delays.
Intended use of the Information
Intended uses of the parent web survey data will be to track how the books were distributed in each of the six pediatric practices, gather parent feedback on the Amazing Me book, including what they did with the book upon receiving it from their doctor, their current knowledge of the book, and the appeal, format, and content of the book. Findings will also suggest whether disseminating through pediatric practices is an appropriate vehicle for reaching parents.
Intended uses of the parent focus group findings include the following: (1) CDC will have more information on whether receiving a children’s book with health messages reaches parents, as intended, (2) CDC will be able to evaluate if receiving a book like Amazing Me from a pediatrician’s office is more credible and therefore parents are more receptive to the information contained within, and (3) CDC will receive some feedback on the concept of developmental milestones and whether the book taught them anything new or encouraged them to seek additional information pertaining to the health and development of their child.
Impact on Privacy to Respondents: No IIF collected will be transmitted to CDC. The only IIF being collected will be an email address for respondents who wish to receive the $5 gift card incentive via email; a name and mailing address for those who wish to receive the $5 gift card incentive via postal mail; or contact information (email and/or phone number) for those who are interested in being contacted to participate in the focus groups. . Limited demographic information will be collected in the web survey. All contact information and IIF collected in the web survey for focus group purposes will be stored and maintained in a separate database that will not in any way be linked to the web survey data. The web survey data will be linked to participant ID that will be non-identifying and will not be linked to any personal information stored in the separate database for focus group participation. Respondents who wish to receive their promised gift card for completing the web survey will be asked to provide their email address. This information will be used solely for sending incentive payments and will be stripped from the data set prior to sending the data to CDC. Therefore, the proposed data collection will have little or no effect on the respondent’s privacy.
The parent web survey will be conducted electronically on the internet. All survey responses (100%) will be submitted through a secure survey website established for this project. The website will be maintained by Survey Monkey and a Westat programmer will program and maintain the survey website. Minimum information will be collected from the respondents to ensure they are eligible for the survey. Web survey is a cost-effective way of surveying a large number of people; it allows respondents the flexibility of completing the survey at a time that is most convenient and allows researchers an opportunity to learn from participants located in different parts of the country.
CDC has done some formative research prior to the development of the Amazing Me book. Findings from that research effort guided the development of the book. Since CDC has developed the Amazing Me book, no evaluation research has been conducted with target audiences. The proposed data collection is unique in that it will provide CDC with information on the effectiveness of reaching parents with health messages through a children’s book and if pediatrician practices, particularly those that work with RoR, are an appropriate vehicle for getting them with the book. The book itself will also be evaluated to determine if it reached parents as intended, what they thought of it, and if they learned anything new from reading it to their child. This data collection effort does not duplicate any past, current, or planned information collection by other federal government agencies. Findings will help inform the development of future health-related children’s books for other age groups.
There is no burden on small businesses or small entities. No small businesses will be involved in this activity. The focus groups will be completed at the convenience of the participants and will not impact the participants’ employers.
This is a one-time data collection effort, and parents will be asked to respond only once to the web survey and/or the focus group. If the requested data collection were not conducted, CDC would be unable to evaluate the Amazing Me book with parents who received it. CDC believes that parents are a reliable source for monitoring children’s developmental milestones and can assist physicians in early identification to prevent further delay once the child is school-aged. Further, without this data collection, CDC would not know for certain if disseminating through RoR practices is an appropriate vehicle for reaching lower income, more at-risk populations. The collection of this information would enable CDC to provide other pediatric offices with an evidence based book that helps teach parents how to monitor children’s development, identify potential problems, and who to contact if they suspect there is a problem.
There are no legal obstacles to reduce the burden.
This request fully complies with regulation 5 CFR 1320.5. The web survey is not designed to produce results that can be generalized to the entire study population. Instead, the web survey results will be used to gather general feedback from parents on their experience receiving the book, reading the book to their child, and their thoughts on the format/content of the book. This information will provide CDC with quantitative data on general parental feedback.
A copy of the agency’s 60-day Federal Register Notice is attached (60-day Federal Register Notice Attachment 2). The notice, as required by 5 CFR 1320.8 (d), was published on April 3, 2013 (volume 78, number 64, pages 20111 - 20112). Two non-substantive public comments were received in response to this notice.
Since September 2012, the CDC Amazing Me team has collaborated with Westat staff on the development of data collection instruments for this study.
CDC collaborators:
Denise Levis, MA, PhD
Health Communication Specialist
Prevention Research Branch, NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-0237
Email: [email protected]
Katie Kilker-Green, MPH, CHES
Health Communication Specialist
Learn
the Signs. Act Early. Campaign
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-3872
Email: [email protected]
Rebecca B. Wolf, MA
Team
Leader, Learn the Signs. Act Early. Campaign
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30329
Phone: 404-498-3951
Email [email protected]
Jessica Drennan, MSW
ORISE Fellow
NCBDDD/CDC
1600 Clifton Rd. Mail Stop E86
Atlanta, GA 30333
Phone: 404-498-3832
Email: [email protected]
Westat collaborators:
Erika Reed-Gross, MSH
Senior Study Director
Westat
1600 Research Blvd.
Rockville, MD 20850
Phone: 770-455-4897
Email: [email protected]
Erika Bonilla, MS
Task Leader
Westat
1600 Research Blvd.
Rockville, MD 20850
Phone: 301-610-4879
Email: [email protected]
Rachel Gaddes, MPH
Task Leader
Westat
1600 Research Blvd.
Rockville, MD 20850
Phone: 301-251-2289
Email: [email protected]
Reach Out and Read (RoR) collaborators:
Jackie Miller, MS
National Book Coordinator
Reach Out and Read National Center
56
Roland Street, Suite 100D
Boston, MA 02129-1243
Phone: 617-455-0600
Email: [email protected]
Anne-Marie Fitzgerald, MA
Chief Operating Officer
Reach Out and Read National Center
56
Roland Street, Suite 100D
Boston, MA 02129-1243
Phone: 617-455-0600
Email: [email protected]
A.9. Explanation of Any Payment or Gift to Respondents
To ensure that sufficient numbers participate in the parent web survey, eligible respondents will receive a $5 Target or Wal-Mart gift card for completing the parent web survey, estimated to take less than 5 minutes to complete. The parent web survey will (1) confirm receipt of the Amazing Me book and the setting in which the book was received, (2) gather feedback about parents’ experiences taking the book home and reading it with their child(ren), and (3) assess the influence of the book on parents’ awareness, attitudes, and self-efficacy regarding monitoring developmental milestones.
All survey responses (100%) will be submitted through a secure survey website established for this project. Web survey is a cost-effective way of surveying a large number of people; it allows respondents the flexibility of completing the survey at a time that is most convenient and allows researchers an opportunity to learn from participants located in different parts of the country.
For the 60 minute parent focus groups, respondents will receive $40 cash as a token of appreciation for their participation. Our target audience has small children at home; the incentive for these focus groups will help defray the cost of transportation and potential child care needs. In addition, the focus groups will be held in informal facilities (e.g., public libraries, community centers) that do not offer childcare services due to liability concerns; further supporting the need for the aforementioned incentive. Every effort is being made to utilize facilities located close to public transportation as well.
Research has consistently shown the value of offering a modest remuneration for motivating respondents to participate in a research study: “Focus groups are unique from other data-gathering processes in terms of the investment that must be made by the individual. It is therefore no surprise that a tradition has been established to provide incentive for participation. From a practical aspect, it would be next to impossible to conduct focus groups without incentives in some situations. The incentive is not a reward and not really an honorarium or salary. It is an incentive. It serves as a stimulus to attend the session. The primary function of the incentive is to get the participants to show for the focus group—and to show up on time. The incentive serves to protect the promised time slot from being preempt.”28 The IRB approval of the study (see IRB Findings and Approval Supplemental 1) included the review and approval of this level of remuneration.
A.10. Assurance of Confidentiality Provided to Respondents
This submission has been reviewed by the NCBDDD Privacy Officer and determined that the Privacy Act does not apply. The only IIF being collected will be an email address for respondents who wish to receive the $5 gift card incentive via email; a name and mailing address for those who wish to receive the $5 gift card incentive via postal mail; or contact information (email and/or phone number) for those who are interested in being contacted to participate in the focus groups. Provision of this information is optional, however, , as respondents can decline the incentive if they don’t want to share their email or mailing address. Also, participants who wish to receive the incentive at their physical address will be re-directed to a separate survey in which they can provide this information. This contact information will be kept in a database separate from survey responses.
After completing the survey using the specified login, participants who are interested in participating in the focus group will be directed to a separate embedded link. Clicking on the link will redirect them to a separate data collection form (i.e., the follow-up contact web survey) in which they will enter their name, email address, and cell phone number. This data will be collected and stored separately from the survey responses and unique identifier and will be used to contact individuals to schedule the focus groups only. This ensures that survey responses remain de-identified.
Focus groups will be audio taped and transcribed for use by the Westat research team in developing a report. All data will be maintained by Westat on project folders located on Westat’s secured server. Study data will only be accessible to Westat staff assigned to this project. Any printed data or notes will be kept in a locked, secure cabinet located in Westat’s Atlanta and Rockville offices for the duration of the study. All Westat employees have taken and signed the Confidentiality pledge. All electronic study data such as focus group audio recordings and survey data will be destroyed at the end of the study.
Participation in the web survey and/or focus groups is voluntary and participants will be advised that their responses will be treated in a secure manner and will not be linked to their names.
Institutional Review Board Approval: Westat’s Institutional Review Board (IRB) reviewed the study instruments and granted approval for the study due to minimal risk (see Attachment 6). The study was approved by the IRB on January 18, 2013. Activity is research involving identifiable human subjects, but CDC involvement does not constitute “engagement” in the research. This project is conducted under a grant or cooperative agreement and CDC employees will not interact with living individuals for research purpose and CDC will not obtain individually identifiable private information. The Contractor will be reviewed by an IRB with a Federal-wide Assurance (FWA) number.
10.1. Privacy Impact Assessment Information
The Privacy Office within the NCBDD has reviewed this submission and determined that the Privacy Act does not apply. Recruitment of participants will use unique identifiers for the web survey and only name, email address, and phone number for focus groups, which will be de-identified from the study responses.
All data (hard copy and electronic) will be stored at Westat, CDC’s selected contractor. All study materials (e.g., research notes, participant consent forms and incentive receipts) will be properly filed, maintained, and secured in a locked file cabinet. Electronic data will be kept on the project-specific network on Westat’s secure server, which is accessible only to users granted rights by the project director and in a secure location with restricted physical access to staff working on the project only.
An Informed Consent Form will be obtained from all of the participants participating in the focus group (see Attachment 7). Consent forms will be signed before the focus group begins. Project staff will be available to answer any questions that the participants may have prior to the beginning of the focus group. The trained moderator will assure participants that any comments made during the focus group will not be attributed to them by name in any of the reports resulting from this research.
Further, the participants in both the focus group as well as the web survey will be reminded that their participation is voluntary and that they may withdraw from the study at any time. Should focus group participants decide to withdraw from the study, they will still receive their promised incentive within two weeks of their participation. Web survey respondents will receive their incentive within two weeks of their completion of the web survey. Focus group respondents will be informed during the screening process that all notes and transcripts from the data collection will solely be used to write the final report. The focus group participants will receive their incentives in cash in person at the end of the group. All of the transcripts and notes from the focus groups will only be available to the project staff. In addition, this information will also be disclosed to the respondents in the informed consent form. The legal authority to collect and maintain this data is granted by Public Health Service Act, Section 301, "Research and Investigation," (42 U.S.C. 241); and Sections 304, 306 and 308(d) which discuss authority to maintain data and provide assurances of confidentiality for health research and related activities (42 U.S.C. 242 b, k, and m(d)).
A.11. Justification for Sensitive Questions
There are no items considered to be highly sensitive for respondents. Potential participants in the web survey will be asked if they received and read Amazing Me and how they were influenced by the books. Focus group questions asked will be intended to prompt information similar to that collected in the web survey, but will allow for more deeper exploration and discussion of parents’ experiences with Amazing Me, as referenced in the moderator’s guide (see Attachment 6). The focus group discussion protocol does not contain questions that ask participants to share any personal or sensitive information with the group, and participants may refuse to answer any question they wish. Our target audience includes parents of 3-year-old children at risk for developmental-behavioral disabilities. CDC wishes to get feedback on the distribution, reach, and influence of Amazing Me in pediatric office settings where the books were distributed to the parents of 3-year-old children.
A.12. Estimates of Annualized Burden Hours and Costs
Assuming all Amazing Me books are distributed, 900 parents will receive an invitation to take the web survey, which asks parents whether or not they have received the book and if they have a child who is age 3, soon to be 3, or recently turned 4 years of age (see Attachments 3 and 3a). Because we do not know how many parents will ultimately receive the books and be both interested and eligible to complete the survey, we do not have a sense of what the final study sample will be. Therefore, we have based our estimates of burden hours and costs on the sampling universe of 900 parents who could receive the Amazing Me book and therefore have the option of participating in the survey. The survey will take respondents on average 10 minutes to complete. After completing the web survey, respondents interested in receiving the $5 gift card incentive will be asked to click on an embedded survey link that will re-direct them to a separate survey in which they can provide their email or name and mailing address for receiving the incentive. In this separate follow-up contact survey, respondents will also be asked if they are interested in participating in a follow-up focus group about Amazing Me. If they are, they will be asked to provide their name, phone number, and email address so that Westat staff can contact them in the future regarding the focus groups. The follow-up contact survey will take respondents on average 1 minute to complete. The total burden for the complete web survey (web survey and follow-up contact survey) is 165 hours.
After completing the web survey, respondents will be invited to provide their contact information for participating in a follow- up focus group that will be held in their community. Using standard over-recruitment strategies, we plan to screen and recruit up to 60 survey respondents to ensure participation by at least 54 parents in the planned focus groups. Although we hope to recruit a mix of focus group respondents based on demographic variables such as age, gender, race/ethnicity, and income, this information is not collected as part of the focus group screener. Although we do collect this data as part of the web survey (which all focus group participants will have completed), we will not be able to link this data to individual focus group participants because the web survey itself does not collect any identifiable information. Identifiable information is only collected after the web survey has been completed, in a separate database, for those who wish to participate in the focus groups. Further, we chose not to collect demographic data as part of the focus group screener, as this would increase respondent burden since such data is collected as part of the web survey. Given our attempt to keep the respondent burden low and to not link identifiable information with web survey response, we recognize that the final sample of focus group participants could be biased in a number of ways, including both demographically as well as the inherent biases involved with voluntary participation in research. However, because focus group participants are being recruited from the sample of web survey respondents, we will have a demographic profile of the universe or sample from which they have been selected. This will give us some sense of how the focus group participants might be similar or different from the larger sample of web survey respondents and we can address any differences we find in the analysis and reporting of the findings. Given that convenience samples are being used for all of the data collection efforts we discuss in this package, our goal is to gather as much information as we can about the population from which study participants have been selected and to document and discuss any biases we might find and how these might affect our findings and the conclusions we draw. After sharing their contact information, potential focus group participants will be contacted by Westat staff and screened to ensure that they have read the book, do not have a child with special needs, and do not have professional experience with child psychology, child development, or health care delivery (see Attachment 5). Each screening will be conducted over the phone and will take approximately 5 minutes. The estimated response burden for the screening process is 5 hours.
The focus groups will have an average of 9 participants each and will be conducted at 6 different locations with a total of 54 participants. The informed consent (Attachment 7) will take 5 minutes to complete and the focus group discussion using the moderator’s guide (Attachment 6) will take 60 minutes to complete. All focus group activities will have a total burden of 64 hours. The total burden for both the survey and focus groups is 229 hours.
Table A. 12. A. Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Parents/ Guardians |
Web Survey |
900 |
1 |
10/60 |
150 |
Parents/ Guardians |
Follow-up Web survey |
900 |
1 |
1/60 |
15 |
Parents/ Guardians |
Focus Group Screener |
60 |
1 |
5/60 |
5 |
Parents/ Guardians |
Focus Group Informed Consent |
54 |
1 |
5/60 |
5 |
Parents/ Guardians |
Focus Group |
54 |
1 |
1 |
54 |
|
|
|
|
|
|
Total |
|
|
|
|
229 |
The annualized cost burden is shown in Table A.12.B. The mean hourly wage rate is based on the most recent (May 2011) National Occupational Employment and Wage Estimates for all occupations, published on the Bureau of Labor Statistics website which is $21.74. See http://www.bls.gov/oes/current/oes_nat.htm.
Table A. 12. B. Estimated Annualized Burden Costs
Type of Respondent |
Form Name |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Parents/ Guardians |
Web Survey |
150 |
$21.74 |
$3,261.00 |
Parents/ Guardians |
Follow-up Web Survey |
15 |
$21.74 |
$326.10 |
Parents/ Guardians |
Focus Group Screener |
5 |
$21.74 |
$108.70 |
Parents/Guardians |
Focus Group Informed Consent |
5 |
$21.74 |
$108.70 |
Parents/ Guardians |
Focus Group |
54 |
$21.74 |
$1,173.96 |
TOTAL |
$4978.46 |
|||
A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no costs to respondents associated with either capital and startup efforts or operation and maintenance of services for this project.
A.14 Annualized Cost to the Government
The average annualized cost to the Federal Government to collect this information is $120,000.85. The federal government personnel estimate is based on the cost of the Federal Project Officer and three Co-Principal Investigators who are responsible for the management and oversight of the project (See Table A.14).
Contractor costs include direct labor for development of instruments, data collection, analysis and reporting for the web survey and focus groups, other direct costs including, vendors, transcription of focus group data, and participant incentives, and indirect costs such as fringe, overhead, general and administrative fees.
Table A.14. Annualized Cost to the Government
|
Parent Web Survey and Focus Group Data Collection |
Reporting |
Total ($) |
|
Federal Government Personnel Costs |
CDC Project Officer (GS-13 at 8% of time) |
-- |
-- |
$6,840 |
CDC Co-Principal Investigator (GS- 9 at 15% of time) |
-- |
-- |
$7,437 |
|
CDC Co-Principal Investigator (GS- 13 at 8% of time) |
-- |
-- |
$6,840 |
|
|
CDC Co-Principal Investigator (GS-14 at 8% of time) |
-- |
-- |
$8,082 |
Contractor Direct Labor |
|
$23,983.70 |
$6,215.95 |
$30,119.65 |
Other Contractor Direct Costs |
Subcontractors, focus group facility rental, transcription, incentives |
$12,251.85 |
$379.30 |
$12,630.15 |
Total Contractor Indirect Costs |
Fringe, overhead, general and administrative, fee |
$38,638.60 |
$9,368.45 |
$48,052.05 |
Total Annualized Cost to Government |
$120,000.85 |
|||
A.15. Explanation for Program Changes or Adjustments
This is a new data collection; therefore, program changes and adjustments do not apply.
A.16.Plans for Tabulation and Publication and Project Time Schedule
Parent Web Survey Analysis
Survey responses will be summarized by question, using frequency tables. If the response (sample) is large enough, we will also compare survey responses between RoR and non-RoR clinics for statistically significant differences in responses. For example, respondent feedback on the book’s content or their experiences receiving the Amazing Me book could be compared between those who received the book in RoR and non-RoR clinic settings to determine if a statistically significant difference exists (at the conventional 5% level) between the two groups. As previously discussed, RoR is a nonprofit organization that promotes early literacy and school readiness in pediatric exam rooms. RoR provides children’s books to families through their network of participating pediatric practices, most of whom serve low-income families. In participating RoR practices, parents receive a new book during each well-child visit from 6 months to 5 years of age, and health care providers often couple the book giveaway with advice for parents about reading to young children. Although non-RoR practices will be selected to match RoR practices in terms of the income levels of families they serve and clinic size (i.e., they see at least 100-150 families for 3-year old well child visits over a 6-month period), non-RoR clinics will have no prior experience in distributing children’s books to their patients. As such, parents in non-RoR clinics will have no such prior experience either. One important question for this study is if parents in non-RoR practices will be receptive to receiving the Amazing Me book as part of their child’s well-child exam and if overall they are receptive to the book . Thus, a comparison of parent feedback between RoR and non-RoR settings will help CDC can make an informed decision about whether to expand distribution of Amazing Me beyond RoR practices.
Focus Group Analysis
At the conclusion of the focus groups, notes and audio recordings from the focus groups will be analyzed for common themes and divergent viewpoints among and between audiences. Qualitative analytical software, such as NVivo will be used to facilitate the analysis. Although the procedures used to analyze focus group data are not standardized, there are recommended analysis processes in place. Westat researchers use a notes and transcript-based analysis process similar to that recommended by Krueger.29 First, all data will be organized and codes developed after reviewing the data for salient themes discussed across the groups. Outliers, responses given by only one or two participants, will be coded as well. The analysis process then commences with the following steps:
1) A systematic content analysis was performed that involves the identification, labeling, and categorization of the data. This analysis reviews records from all groups (transcripts, topline reports, observer notes) to identify trends across groups.
These groups are further examined through thematic analysis to determine if certain themes may emerge from the patterns. These themes may be articulated directly by the participants or identified by the study team. In addition to assisting with the identification of patterns and themes in the data, content and thematic analyses also allow large amounts of collected data to be reduced or distilled into a more easily studied and understood format.
A logical analysis was then performed by looking for patterns of difference, as well as similarities that may emerge from cross-classifying the data. This step is helpful in identifying patterns or themes that are not immediately obvious, while also taking into consideration outlier opinions.
A final report of the focus group findings will be completed within 15 months after OMB approval. Findings will be used to inform the development of clinic engagement and book distribution procedures for future CDC efforts to engage pediatric clinics in sharing the Amazing Me book with their patients.
Recruitment of RoR and non-RoR clinics will begin within 1 month of OMB approval. Once the 3 RoR and 3 non-RoR clinics have been recruited, designated coordinators in each clinic will be trained on book distribution procedures and distribution will commence in all clinics approximately 3 months after OMB approval. Recruitment and scheduling for the parent focus groups will begin approximately 7 months after clinics have begun distributing the Amazing Me book. As previously discussed, the Parent Web Survey will be used to identify respondents interested in participating in the follow-up focus groups. The study will be completed within 12 months. See Table A16 below outlines the project time schedule by activity.
Table A.16.B. Project Time Schedule
Activity |
Time Schedule |
Parent Web Survey |
|
Recruit RoR and non-RoR clinics |
1 month after OMB approval |
Clinics begin distribution of Amazing Me book |
3 months after OMB approval |
Begin parent web survey (survey remains open for 6 months) |
3 months after OMB approval |
Final report of parent web survey findings |
13 months after OMB approval |
Parent Focus Groups |
|
Recruit and schedule parent focus groups |
9 months after OMB approval |
Conduct parent focus groups |
12 months after OMB approval |
Draft report of web survey findings |
14 months after OMB approval |
Final report of web survey findings |
14 months after OMB approval |
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable. The OMB expiration date will be displayed.
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
1 Centers for Disease Control and Prevention (2012). Data and Statistics. Available at http://www.cdc.gov/NCBDDD/autism/data.html
2 Centers for Disease Control and Prevention (2012). Facts about Developmental Disabilities. Available at http://www.cdc.gov/ncbddd/developmentaldisabilities/facts.html
3 Centers for Disease Control and Prevention (2012). Key Findings: Trends in the Prevalence of Developmental Disabilities in U.S. Children, 1997-2008. Available at http://www.cdc.gov/ncbddd/features/birthdefects-dd-keyfindings.html
4 Mackrides, P and Ryherd, S., (2011). Screening for Developmental Delay. American Family Physician. 2011 Sep 1;84(5):544-549. Available at http://www.aafp.org/afp/2011/0901/p544.html#afp20110901p544-b12.
5 Centers for Disease Control and Prevention (2012). Learn the Signs. Act Early. Available at http://www.cdc.gov/ncbddd/actearly/index.html.
6 Squires J, Nickel R, Eisert D. 1996. Early detection of developmental problems: strategies for monitoring young children in the practice setting. Journal of Developmental and Behavioral Pediatrics. 17. 420-427.
7 Glascoe, F.P., (1997). Parents' concerns about children's development: Prescreening technique or screening test. Pediatrics 1997; 99: 522-528.
8 Glascoe, F.P., (2003). Parents' Evaluation of Developmental Status: how well do parents' concerns identify children with behavioral and emotional problems? Clinical Pediatrics 2003;42:133-138.
9 Boyle, C.A., Boulet, S., Schieve, L., Cohen, R.A., Blumberg, S.J., Yeargin-Allsopp, M., Visser, S., Kogan, M.D. 2011 Trends in the Prevalence of Developmental Disabilities in US Children, 1997–2008. Pediatrics. 2011 Available at http://pediatrics.aappublications.org/content/early/2011/05/19/peds.2010-2989.full.pdf+html
10 Diener ML, Hobson-Rohrer W, Byington CL. 2012). Kindergarten Readiness and Performance of Latino Children Participating in Reach out and Read. Journal of Community Medicine and Health Education 2:133. Available at http://www.reachoutandread.org/FileRepository/Diener_etal2012.pdf
11 Needlman R., Toker K.H., Dreyer B.P., Klass P., Mendelsohn A.L. Effectiveness of a primary care intervention to support reading aloud: a multicenter evaluation. Ambulatory Pediatrics 2005; 5, p. 209–215. Available at http://www.reachoutandread.org/FileRepository/Needlman2005.pdf.
12 Blom-Hoffman, J., Wilcox, K., Dunn, L., Leff, S., Power, T. (2009). Family involvement in school-based health promotion: bringing nutrition information home. School Psychology Review. 37(4): 567-577.
13 Bauer, N, Hus, A., Sullivan, P., Szczepaniak, D., Carroll, A., Downs, S. (2012). A Pilot Study Using Children's Books to Understand Caregiver Perceptions of Parenting Practices. Journal of Developmental & Behavioral Pediatrics: June 2012 , Vol. 33, Issue 5, 423–430. Available at http://journals.lww.com/jrnldbp/Abstract/2012/06000/A_Pilot_Study_Using_Children_s_Books_to_Understand.6.aspx.
14 Reich, S. M., Penner, E. K., & Duncan, G. J. (2011). Using baby books to increase new mothers’ safety practices. Academic pediatrics, 11(1), 34-43. Available at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3043459/
15 Behal, P., Bennett, K., Crites, A., Weigel, D., (2003). Money on the Bookshelf: Using Children's Books to Reach Limited Resource Families with Money Management Education. Journal of Extension.vol. 41, No.3 Available at http://www.joe.org/joe/2003june/iw1.php
16 PeachCare provides comprehensive health care to children through the age of 18 who do not qualify for Medicaid and live in households with incomes at or below 235% of the federal poverty level.
17 Health Resources and Services Administration, What is a Health Center? Available at http://bphc.hrsa.gov/about/
18 Pew Research Center, Available at http://www.pewinternet.org/Static-Pages/Trend-Data-(Adults)/Whos-Online.aspx
19 Pew Research Center, Available http://www.pewinternet.org/Trend-Data-(Adults)/Device-Ownership.aspx
20 Pew Research Center, Available http://www.pewinternet.org/Reports/2011/Smartphones.aspx
21 Centers for Disease Control and Prevention (2012a). Data and Statistics. Available at http://www.cdc.gov/NCBDDD/autism/data.html [Accessed on June 13, 2012].
22 Squires J, Nickel R, Eisert D. 1996. Early detection of developmental problems: strategies for monitoring
young children in the practice setting. Journal of Developmental and Behavioral Pediatrics. 17. 420-427.
23 Glascoe, F.P., (1997). Parents' concerns about children's development: Prescreening technique or screening
test. Pediatrics 1997; 99: 522-528.
24 Glascoe, F.P., (2003). Parents' Evaluation of Developmental Status: how well do parents' concerns identify
children with behavioral and emotional problems? Clinical Pediatrics 2003;42:133-138.
25 Mackrides, P and Ryherd, S. (2011). Screening for Developmental Delay. American Family
Physician. 2011 Sep 1;84(5):544-549. Available at
http://www.aafp.org/afp/2011/0901/p544.html#afp20110901p544-b12 [Accessed June 18, 2012]
26 Glascoe, F.P., (2005). Screening for developmental and behavioral problems. Mental Retardation and
Developmental Disabilities Research Review 2005;11(3):173–179.
27 Brothers, K.B., Glascoe , F.P., Robertshaw, N.S., (2008). PEDS: developmental milestones—an accurate brief
tool for surveillance and screening. Clinical Pediatrics (Phila). 2008; 47(3):271–279.
28 Krueger RA, Casey MA. Focus groups. A practical guide for applied research. Thousand Oaks (CA): Sage; 2009.
29 Krueger, R.A. (1994) Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage Publications.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Erika Bonilla |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |