Evaluation of NHTSA’s Child Pedestrian Safety Curriculum
Table of Contents
SUPPORTING STATEMENT
A. JUSTIFICATION………………………………………………………………………3
1. Explain the circumstances that make the collection of information necessary..................4
2. Indicate how, by whom, and for what purpose the information is to be used ...................5
3. Describe whether, and to what extent, the collection of information involves
the use of technological collection techniques or other forms of information
technology...........................................................................................................................7
4. Describe efforts to identify duplication. Show specifically why any similar
information already available cannot be used.....................................................................7
5. If the collection of information impacts small businesses or other small entities,
describe methods used to minimize burden........................................................................8
6. Describe the consequence to Federal program or policy activities if the
collection is not conducted or is conducted less frequently................................................8
7. Explain any special circumstances that would cause the information collection
to be conducted in a manner inconsistent with the guidelines set forth in
5 CFR 1320.6.......................................................................................................................8
8. Provide a copy and identify the date and page number of publication in the Federal
Register of the agency’s notice, required by 5 CFR 1320.8 (d), soliciting comments
on the information collection prior to submission to OMB. Summarize public
comments received in response to that notice and describe actions taken by the
agency in response to these comments. Describe efforts to consult with persons
outside the agency to obtain their views............................................................................. 9
9. Explain any decision to provide any payment or gift to respondents, other
than remuneration of contractors or grantees..................................................................... 9
10. Describe any assurance of confidentiality provided to respondents.................................. 9
11. Provide additional justification for questions of a sensitive nature.................................. 10
12. Provide estimates of the hour burden of the collection of information on the
respondents........................................................................................................................10
13. Provide an estimate of the total annual cost burden to respondents or record
keepers resulting from the collection of information........................................................11
14. Provide estimates of annualized cost to the Federal government.....................................11
15. Explain the reasons for any program changes or adjustments reported in Items
13 or 14 of the OMB Form 83-1.......................................................................................11
16. For collections of information whose results will be published, outline plans
for tabulation, and publication...........................................................................................11
17. If seeking approval to not display the expiration date for OMB approval of
the information collection, explain the reasons that display would be inappropriate........12
18. Explain each exception to the certification statement identified in Item 19,
Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-1...............12
Appendices
Appendix A: Authority
Appendix B-1: 60 Day Federal Register Notice
Appendix B-2: 30 Day Federal Register Notice
Appendix C-1: Student Survey
Appendix C-2: Caregiver Survey - English
Appendix C-3: Caregiver Survey - Spanish
Appendix C-4: Instructional Staff Survey
Appendix D-1: Justification for Student Survey Questions
Appendix D-2: Justification for Caregiver Survey Questions
Appendix D-3: Justification for Instructional Staff Survey Questions
Appendix E: Parent Notification Sheet
The National Highway Traffic Safety Administration (NHTSA) was established by the Highway Safety Act of 1970 (23 U.S.C. 101) to carry out a Congressional mandate to reduce the mounting number of deaths, injuries and economic losses resulting from motor vehicle crashes on our Nation’s highways. In support of this mission, NHTSA proposes to conduct information collections to evaluate the implementation and impact of its newly developed Child Pedestrian Safety Curriculum which has been integrated into North Carolina’s Let’s Go NC pedestrian and bicycle curriculum. The pedestrian portion of Let’s Go NC is closely modeled after the NHTSA curriculum and was intended by its developers to be a localization of the NHTSA version as suggested by the NHTSA curriculum developers. Thus, evaluating the North Carolina version of the pedestrian curriculum provides the desired assessment of the NHTSA curriculum as actually implemented by a State.
NHTSA developed a new Child Pedestrian Safety Curriculum to teach and encourage safe pedestrian behaviors for students at the elementary school level (grades K-5). The overall goal of the curriculum is to aid elementary age school children in developing age appropriate traffic safety knowledge and practical pedestrian safety skills that will form the foundation for a lifelong improvement in their walking safety. The curriculum at each grade level is organized into five lessons that target key areas of pedestrian safety and are designed to meet national learning standards. The curriculum is designed to increase in difficulty as the grade-based unit advances, to require greater problem-solving opportunities, and to facilitate peer modeling and discussion. It actively involves parents/caregivers of the students through take-home materials. Following NHTSA’s suggested approach, the North Carolina Department of Transportation localized NHTSA’s curriculum to arrive at the pedestrian module of the Let’s Go NC program.
NHTSA desires to document how the pedestrian curriculum is implemented by a typical school district and whether the implementation is effective in changing knowledge, attitudes, and behaviors. An essential part of this evaluation effort will be surveying knowledge, attitudes, and self-reported behaviors of various groups of people integral to the curriculum implementation. These groups include children in grades K-5, their parents/caregivers, and instructional staff at the test schools. The study design includes surveying equivalent groups of participants at comparison schools where the curriculum was not implemented. A brief description of the proposed data collection effort for each group of participants is provided below.
NHTSA is requesting approval to conduct in-class structured oral surveys (NHTSA Form 1217) of elementary school students in all experimental and comparison schools pre- and post-curriculum implementation to assess knowledge, opinions, and self-reported experiences. Experience from previous studies of the K-5 age group showed that a one-on-one oral survey works best with this population because of their limited reading ability. The oral survey also permits the child respondent to demonstrate key search behaviors if he/she cannot verbalize them.
NHTSA is requesting approval of paper-and-pencil surveys of student caregivers (NHTSA Forms 1216 and 1216-A) and an Internet-based survey of all school instructional staff (NHTSA Form 1215) in the two experimental and two comparison schools to determine their attitudes, experiences, and self-reported behaviors related to pedestrian safety and the implementation of the curriculum. These surveys would only take place after curriculum implementation.
NHTSA is requesting approval for surveys beginning in January 2014 and concluding in July of 2014. NHTSA will administer the in-class surveys of children through the use of an oral survey conducted by adults associated with each school’s Parent Teacher Organization (PTO). The paper-and-pencil survey of caregivers (in English and Spanish) will be sent home in students’ homework folders to be given to parents/caregivers and returned with the same folder. This is an established and proved effective method of data collection already in use by the cooperating school district. The survey of all school instructional staff will be conducted via an Internet survey. This is also the established and proved effective method the school district uses to obtain information from their teacher population.
The following sections describe the justification for these proposed data collections in more detail, along with the estimates of cost and burden.
A.1. Explain the circumstances that make the collection of information necessary. Identify any Legal or administrative requirements that necessitate the collection.
Circumstances making the collection necessary
NHTSA was established by the Highway Safety Act of l970 (23 U.S.C. 101) to carry out a Congressional mandate to reduce the mounting number of deaths, injuries, and economic losses resulting from motor vehicle crashes on the Nation’s highways. As part of this statutory mandate (see Appendix A), NHTSA is authorized to conduct research as a foundation for the development of motor vehicle standards and traffic safety programs.
There are currently other child pedestrian curricula available from other safety organizations; however, NHTSA’s new Child Pedestrian Safety Curriculum is comprehensive, meets learning standards so that it does not displace core teaching topics such as reading and math, and emphasizes the importance of the development of the pedestrian safety knowledge and skills identified by previous research as necessary for reducing pedestrian crashes. Achieving the objectives of the curriculum requires implementing both the core lessons and a choice of activity options for each lesson. To determine if the curriculum is effective, it is important to perform a systematic assessment of both how the curriculum is implemented by a typical school district and the curriculum’s effectiveness in achieving the desired knowledge and behavioral changes in the target audiences.
For many types of similar courses, NHTSA will hear that the training was a “success” because hundreds of students were trained or because a large number of students passed the test with a certain competency score. However, neither of these explanations is sufficient to inform NHTSA as to what the students learned, whether they will retain the knowledge, whether behavior was affected, how often they should be trained, how they will use the information, or how safe they became based on receiving instruction. Moreover, NHTSA needs a firm basis for passing on best practices to future users of the curriculum and for making future curriculum revisions.
NHTSA is interested in implementing strong and pertinent curricula and conducting solid evaluations of those courses that can help teach students in an effective manner how to be safe in traffic and thereby reduce crashes and injuries. It is, therefore, particularly important for any child pedestrian safety curriculum NHTSA promotes to be demonstrably successful in reducing the likelihood of harm and/or injury to elementary-aged children. This study will employ both implementation and impact evaluation methodologies to examine the effectiveness of the Child Pedestrian Safety Curriculum in full support of NHTSA’s mission.
The Let’s Go NC pedestrian curriculum will be implemented in two Wilson, NC schools on or about April 1, 2014. The schools have agreed to not implement the bicycle portion of Let’s Go NC until the study is over. Two additional schools in Wilson not receiving the curriculum will serve as comparisons.
Based on the United States Census estimate for 2012, Wilson County has a population of 81,867 with 52.4% being female. The County is racially diverse with a population that is 57.5% White, 39.6% Black, and 9.8% Hispanic. The County has 14 elementary schools.
The National Traffic and Motor Vehicle Safety Act of 1966, Title 15 United States Code 1395, Section 106 (b), gives the Secretary authorization to conduct research, testing, development, and training as authorized to be carried out by subsections of this title. The Vehicle Safety Act was subsequently re-codified under Title 49 of the U.S. Code in Chapter 301, Motor Vehicle Safety. Section 30168 of Title 49, Chapter 301, gives the Secretary authorization to conduct research, testing, development, and training to carry out this chapter. Title 23 of the U.S. Code, Chapter 4, Section 403 (attached as Appendix A) gives the Secretary authorization to use funds appropriated to carry out this section to conduct research on all phases of highway safety and traffic conditions; to conduct training or education programs in cooperation with other Federal departments and agencies, States, private sector persons, highway safety personnel, and law enforcement personnel; and to conduct research on, and evaluate the effectiveness of, traffic safety countermeasures [See 23 U.S.C. 403(a)(1), 23 U.S.C. 403 (a)(4), 23 U.S.C. 403 (a)(5)].
The purpose of this information collection is to provide critical information needed by NHTSA to demonstrate effective countermeasures that meet the Agency’s mandate to improve highway traffic safety. The collected data will assist NHTSA in its ongoing responsibilities for: (a) reporting the effectiveness of program activities; (b) providing information to NHTSA’s partners involved in improving public safety; and (c) providing sound scientific reports on NHTSA’s activities to other public safety researchers.
NHTSA has awarded a contract to an evaluation firm with extensive experience in conducting program evaluations that include surveys and behavioral observations. Data from the surveys will be used to evaluate knowledge, attitudes towards, and self-reported behaviors relative to the implementation of the new child pedestrian curriculum. It will also be used to develop best practices to recommend to future curriculum users and to identify ways to improve the curriculum in any future revisions.
Data from all surveys collected at the selected intervention schools will be compared to data collected from the chosen comparison schools that did not receive the curriculum. This will permit NHTSA to assess whether the curriculum impacted knowledge, attitudes, and self-reported behaviors of students, caregivers, and instructional staff. In the future the results from these surveys can also be compared to findings in other locales that may implement the curriculum.
Any differences in knowledge, attitudes, and self-reported behaviors will be examined relative to program implementation activities at the intervention sites and any changes in observed behaviors. Without a measured difference in knowledge or attitudes, it would be difficult to attribute any observed behavioral changes to the implementation of the curriculum at the intervention sites. The addition of the comparison sites ensures that any natural changes in knowledge and attitudes that occur over time or changes promoted by other efforts not related to the curriculum are taken into consideration.
NHTSA will also document school instructional staff efforts associated with curriculum implementation. This will include efforts to prepare for implementation, the types of activities undertaken during implementation, any problems encountered, and suggestions for ways to improve the curriculum or its delivery. This will be achieved through direct observations of instructor/administrator activities, unstructured discussions with staff delivering the curriculum (fewer than nine discussions), and an Internet-based survey of all school instructional staff, including those not directly involved with teaching the curricula or assisting with its implementation since they will likely indirectly experience the effects of the program.
The results of the survey efforts will be used by NHTSA to assess the effectiveness of the curriculum and determine where refinements in the content or delivery of the curriculum could be beneficial. Demographic data collected by the surveys will identify if group differences exist in responses to these survey items. In addition, results from the survey analyses will add value to the larger research and evaluation design. NHTSA will employ a complementary, multi-pronged evaluation approach consisting of surveys (as described in detail in this justification), observations of program delivery, and observations of staged and naturalistic child pedestrian behaviors at the selected school sites. It should be noted that staged behaviors are part of the normal curriculum activities. Naturalistic observations will take place in areas near the schools and in surrounding neighborhoods from which the schools draw students. The combined results from all of the data collection efforts will provide a comprehensive assessment of the curriculum’s implementation and impact on the target populations. Results will be applied to the development of strategic initiatives and future programs aimed at reducing traffic injuries and fatalities.
The findings from this proposed collection of information will assist NHTSA in addressing the problem of child pedestrian crashes and in formulating programs and recommendations. NHTSA will use the findings to help focus current programs and activities to achieve the greatest benefit, to develop new programs to address child pedestrian safety issues, and to provide informational support to States, localities, and law enforcement agencies that will aid them in their efforts to improve child pedestrian safety.
Besides developing its own program and technical assistance activities, NHTSA will:
Disseminate the information to State and local highway safety authorities, who can use it to develop, improve and target their own child pedestrian safety programs and activities.
Disseminate the information to citizen action groups and other organizations concerned with traffic safety issues who can use it to develop, improve and target their own programs and activities.
Make reports available to the public on NHTSA’s website (www.nhtsa.dot.gov), and in NHTSA’s behavioral research electronic library (http://ntlsearch.bts.gov/repository/ntlc/nhtsa/index.shtm).
The data collected under this project will help determine how the new curriculum was implemented and whether it affected knowledge, attitudes, and behaviors of the target audience. The data will determine appropriate emphasis for future countermeasure activity. The results will be disseminated to others for research and program development activities. If the surveys were not conducted, NHTSA program efforts would lack direction due to inadequate information upon which to base program decisions, severely limiting the Agency’s effectiveness in reducing injuries and fatalities.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical or other technological collection techniques or other information technology. Also describe any consideration of using information technology to reduce burden.
The only collection effort involving advanced technology is the Internet survey of instructors and administrators. The approach will be the same as the school district uses for similar data collection activities from its staff. The district will send out an email with a link to the Internet-based survey to the staffs of the four involved schools. Staff members will voluntarily complete the survey at their convenience. The oral survey of students will include the use of on-paper recording by the surveyor. The survey of student caregivers will involve simple paper-and-pencil surveys to be filled out at a time and place of the participants’ choosing. Using electronic recording for either the student interview or the caregiver survey would not reduce the burden on the respondent and would increase complexity. Any data not collected electronically will be entered into an electronic database, and NHTSA will receive 100 percent of the results of the data collection in electronic files.
NHTSA’s Child Pedestrian Safety Curriculum is a new curriculum that includes state-of-the-art approaches to teaching children safe pedestrian behaviors. As such, no formal evaluations of this program or any closely related program have been conducted, and no legacy programs have used the same approach in the past. NHTSA did pilot test the program as part of its development process before official release, but the small-scale pilot efforts were intended to check the basic flow and usability of the curriculum and did not include the in-depth data collection activities focused on process and outcomes proposed here.
Because no data on this new curriculum exists until it is collected, no other data source can be substituted. There is no possibility of duplicating information that is currently available.
Small businesses or other small entities are not the target of the data collection activities. All collection activities will be associated with the chosen schools, and data collection activities will be integrated under their normal operating procedures. For example, parents/caregivers are often asked to fill out various forms throughout the school year that are distributed in each child’s homework folder. The caregiver surveys will be distributed using this same distribution and return channel normally employed by the schools. The same is true for school instructors and administrators using the internal email systems at the schools. All survey distributions will come from the individual school or the school district headquarters but will reference the NHTSA study.
As the national leader in traffic safety research, Congress has tasked NHTSA with providing evidence-based guidance to the States and stakeholders. Without comprehensive evaluation efforts to measure how traffic safety programs work, it will be impossible to develop effective intervention strategies and adequately interpret the value of these programmatic efforts.
In evaluating new countermeasures such as the subject child pedestrian safety curriculum, the collection of information generally occurs at two points: before implementation of the countermeasure and again after the implementation is complete. Multiple future collections may take place if NHTSA is interested in the persistence of any observed effects. The baseline and post-implementation surveys of students are necessary to determine whether observed changes in knowledge, attitudes, and behaviors can be confidently attributed to the program activities (as opposed to extraneous events or chance). The use of comparison sites allows the study to utilize only post-implementation surveys for parents/caregivers and instructional staff, which reduces the burden on these study groups. Without the administration of the surveys it would be impossible to provide evidence-based recommendations for future local, State, and national interventions of this type.
No special circumstances require the collection to be conducted in a manner inconsistent with the guidelines in 5CFR 1320.6.
Provide a copy of the Federal Register document soliciting comments on the collection of information, a summary of all public comments responding to the notice, and a description of the agency’s actions in response to the comments. Describe efforts to consult with persons outside the agency to obtain their views.
NHTSA published a notice in the Federal Register with a 60-day public comment period to announce this proposed information collection on September 5, 2013, Volume 78, Number 172, pages 54727 - 54729 (See Appendix B-1). There were no comments on the Notice.
NHTSA published a notice in the Federal Register with a 30-day public comment period to announce forwarding of the information collection request to OMB for approval on December, 26, 2013, Volume 78, Number 248, pages 78502-78503. (See Appendix B-2)
Responses to the Federal Register Notice
No comments were submitted to Docket Number NHTSA-2013-0087 in response to the 60-day Federal Register Notice.
Expert Consultation
Dunlap and Associates, Inc. and the Highway Safety Research Center at the University of North Carolina, Chapel Hill created the survey items. Prior to the survey development work, NHTSA’s program, research, communications, and regional offices provided significant input on the topics and questions.
The study will not offer any direct payment or gift for participation to respondents. However, the standard operating protocol for the participating school district for similar surveys of parents/caregivers is to offer a class-based incentive (e.g., ice cream social) to those classes that have the highest return rates. Similarly, instructional staff often are eligible to receive an incentive of some sort (e.g., classroom/school supplies) when participating in school-wide survey efforts. The study will reimburse each school, or the school district, for any incentives offered for participation in the surveys related to this project. In addition, there are no direct out-of-pocket expenses associated with participation.
The survey introductions inform respondents that participation is voluntary and that responses are anonymous. The surveys will not collect identifying information such as names, addresses, telephone numbers, employee numbers, or social security numbers. Upon completion of these surveys, it would be impossible to identify any specific individual based on his or her responses to the questions.
The surveys do not contain any questions of a sensitive nature or related to matters that are commonly considered private.
Participants will be asked to complete surveys as detailed in Table 1 below. Estimated sample sizes are provided based on the maximum number of interviews to be attempted or surveys distributed. Actual response rates will likely vary. In order to reduce burden on parents/caregivers and instructional staff, the self-administered questionnaires will only be administered after the curriculum implementation has taken place. Each household will receive a questionnaire for each resident child who is a student in the test schools to be completed by one primary caregiver selected at the discretion of the members of the household based on the survey instructions. All surveys and observations of students will take place before and after implementation. Data collection at the comparison sites will follow the same schedule.
Group |
Pre-implementation |
Post-implementation |
Total Hour Burden Estimate |
Parents/Caregivers
|
N/A |
Self-administered questionnaire; 5 minutes to complete; Sample = Primary caregivers of all enrolled K-5 students at 4 schools (N = 2,000) |
166.67 hours |
All Instructional Staff at Schools |
N/A |
Self-administered questionnaire; 5 minutes to complete; Sample = All K-5 instructors in 4 schools (N = 200) |
16.67 hours |
Participants (students) |
Fully structured oral survey; 5 minutes to complete; Sample is all students K-5 in 4 schools (N = 2,000) |
Fully structured oral survey repeated; 5 minutes to complete; Sample is all students K-5 in 4 schools (N = 2,000) |
333.33 hours |
In sum, NHTSA proposes to interview or survey up to 4,200 unique participants (2,000 students will be surveyed twice) over the life of the project and estimates a burden of 516.67 total hours. At $22.011 per hour, the total annual estimated cost associated with the burden hours is: $22.01 x 516.67 hours for a total of $11,371.91 (if all possible respondents in Table 1 are interviewed or surveyed). The above calculations apply the same $22.01 per hour rate to the children surveyed since no consensus estimates on the value of a child’s time could be located. This is likely high, and it can even be postulated that there is no burden to the child respondents since the survey is part of their normal school work and educational day. Respondents would not incur any other reporting cost from the information collection.
Provide an estimate of the total annual cost to the respondents or record keepers resulting from the collection of information.
There are no recordkeeping or reporting costs to respondents. Respondents will be contacted through the schools system’s routine contact procedures and asked about their attitudes, knowledge, and behavior regarding specific traffic safety topics pertaining to pedestrian safety and the studied curriculum. As noted in Table 1, some respondents will be contacted more than once, but there is no preparation of data required or expected of respondents above and beyond that required for normal operations. Respondents do not incur: (a) capital and startup costs, or (b) operation, maintenance, and purchase costs as a result of participating in the survey.
The costs of collecting the data, as well as supervising collection, travel to the sites, data entry and data analysis are shown by target group in Table 2. These costs are non-recurring; all data collection as described in Table 1 will be accomplished in a single Federal Government Fiscal Year.
Group |
Target Sample Size (See Table 1) |
Estimated Hours |
Annual Cost Burden |
Parents/Caregivers
|
2000 |
166.67 |
$15,624.96 |
All Teaching Staff at Schools |
200 |
16.67 |
$7,426.80 |
Participants (students) |
4000 |
333.33 |
$47,594.48 |
Total all Groups |
6200 |
516.67 |
$70,646.24 |
The total cost to the Federal Government for this study is $476,624.06 over 36 months, which amounts to an annual cost of approximately $158,874.69 per year for 3 years. In addition to the costs accounted for in Table 2, this total cost includes site/school recruitment, survey development, naturalistic observations, report writing, and other project planning and administrative costs.
The reason for the program change is this is a new survey which will increase NHTSA’s overall burden hour total by 516.67 hours over the course of one year.
Baseline data collection is scheduled to begin March 1, 2014 to accommodate the school system’s planned curriculum start date of April 1, 2014. This is the optimum time based on school district schedules. Post-implementation data collection will begin immediately after the curriculum has been delivered, likely in late April or early May of 2014. NHTSA expects to receive an interim report on March 27, 2014 detailing efforts to that date. This report will include any interim results from sampling according to the plan described in Table 1. Analyses for the student survey will compare pre- to post waves at experimental and control schools as well as comparing the overall pattern of results for the experimental and control schools. Non-parametric analyses including cross tabulations will be the main approach. More sophisticated analyses such as multi-way frequency analysis, MANOVA, and Logistic Regression will be applied if supported by the data. Caregiver and instructional staff surveys involve a single post sampling and will be analyzed primarily using non-parametric techniques. A draft final report is expected by September 27, 2015. The final report will be published upon receipt and completion of agency review, likely in late 2015.
NHTSA will display the expiration date for OMB approval.
Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions” of the OMB Form 83-I.
No exceptions to the certification are made.
1 US Department of Labor, Bureau of Labor Statistics (2013). May 2012 National Occupational Employment and Wage Estimates – Mean Hourly Wage (All Occupations). http://www.bls.gov/oes/current/oes_nat.htm
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Table of Contents |
| Author | Mary Hinch |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |