Incentives for the Adoption of the Youth@Work – Talking Safety Curriculum
Supporting Statement B
Request for Office of Management and Budget Review and Approval
for Federally Sponsored Data Collection
March 26, 2015
Rebecca J. Guerin, MA
National Institute for Occupational Safety and Health
Education and Information Dissemination Division
1090 Tusculum Ave, MS C-10
Cincinnati, OH 45226-1998
513-533-8435 (voice)
513-533-8560 (fax)
Email:[email protected]
Table of Contents
B. Collections of Information Employing Statistical Methods
Respondent Universe and Sampling Methods
Procedures for the Collection of Information
Methods to Maximize Response Rates and Deal with Nonresponse
Tests of Procedures or Methods to be Undertaken
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
B. Collections of Information Employing Statistical Methods
1. Respondent Universe and Sampling Methods
The respondent universe for this study consists of representatives of/from 13,567 U.S. public school districts (as estimated by the National Center for Education Statistics, NCES, for 2011-2012; http://nces.ed.gov/programs/digest/d13/tables/dt13_214.10.asp). Twenty-eight (28) key informant interviews will be conducted. This number is based on results from the pre-test and is consistent with qualitative research methods for achieving response saturation [Guest 2006]. Interviews will be conducted with seven (7) respondents from each of the four (4) regions of the United States (Northeast, Midwest, West, South) as defined by the U.S. Census Bureau. In each region, a sample of districts will be selected based on jurisdictional density, as defined by the National Center for Education Statistics (NCES). The categories of jurisdictional density are:
City: Located inside an urbanized area and inside a principal city.
Suburban: Located inside an urbanized area and outside a principal city.
• Town/rural: Territory inside an urban cluster/territory, or rural territory.
In the United States in 2011-12, 29 percent of students attended schools in cities, 34 percent attended schools in suburbs, and 37 percent of students attended town or rural schools [Keaton 2013]. To achieve a sample of districts that is proportional to the distribution of students in the United States, 2-3 City districts, 2-3 Suburban districts, and 2-3 Town/rural districts will be selected in each of the 4 Census regions.
Table 1 shows how the sample will be distributed:
Table 1: Location and Number of Key Informant Interviews |
||
Region |
Locale |
Target number of completed interviews |
Northeast |
City |
3 |
|
Suburb |
2 |
|
Town/Rural |
2 |
Midwest |
City |
2 |
|
Suburb |
3 |
|
Town/Rural |
2 |
West |
City |
2 |
|
Suburb |
2 |
|
Town/Rural |
3 |
South |
City |
3 |
|
Suburb |
2 |
|
Town/Rural |
2 |
Total |
|
28 |
Within each of the four regions, we will select 21 districts using random assignment. Two to three districts will be selected from each locale code (city, suburb, rural). Using the National Center for Education Statistics Local Education Agency Universe data, districts will be sorted by region and locale code and a random process will be utilized to identify the recruitment pool of seven districts within each region and locale code.
The number of districts in the screening pool is based on our experience with the pilot test. Therefore, 84 districts will be contacted for screening in order to identify respondents who will be eligible to complete the planned, 28 interviews. To achieve the same number of interviews in each region, 21 districts will be screened in order to obtain 7 eligible interviews. In order to get about the same number of interviews per locale, 7 districts each of the three locale codes (city, suburb, rural) will be screened to yield two or three interviews in each locale code and region. Exhibit 1 shows the target number of interviews and recruiting goals at each phase.
Exhibit 1: Target number of interviews and recruits.
|
Total |
In each region (4) |
In each locale within a region (3) |
Target |
28 interviews |
7 |
2 or 3 |
Screen |
84 districts |
21 |
7 |
Eligible respondents will consist of those staff who are responsible for making decisions about the curriculum delivered within their school districts for specific content area(s) (such as health education, family and consumer sciences, or others) where the Talking Safety curriculum would find a fit. The following inclusion criteria will be used in this phase of the study. Participants must:
be at least 18 years of age
have a formal position (assistant superintendent; curriculum coordinator/director; or related position) within school districts identified for this study
be responsible for making decisions in their school district about curriculum that would include workplace safety and health
Screening will begin with an introductory letter sent to the superintendents’ offices of the pool school districts, describing the goals of the study, including a brief introduction to Talking Safety curriculum, and explaining the process of conducting the interviews. The letter informs the potential key informants that they will be contacted by phone to discuss the best possible respondent for that district. Next, a screening call to the superintendent’s office will take place. We will consult with either the superintendent or, more likely, the superintendent’s assistant, to determine the best person to interview. In addition to shortening the process for identifying the respondent, this process will have the added advantage of allowing the recruiter to mention that we got the prospective interviewee’s name from the superintendent’s office. One person will be interviewed in each district.
For purposes of reaching population response saturation, which means enough responses are gathered that no new response themes are generated, these interviews will be administered to a total of 28 curriculum coordinators/directors located in selected, public school districts within the continental United States. Based on evidence from Guest, Bunce and Johnson (2006), it is posited that data saturation may occur after as few as twelve interviews.
Qualitative data analysis will be conducted to explore common themes and findings within the 28 key-informant interviews. The information from the initial six pretest interviews will be included in the analysis. It should be noted that the qualitative methods used to analyze data from open-ended questions do not lend themselves to the same sort of power analyses aimed at identifying between group differences used for quantitative research approaches. Rather, the concept of saturation is used to determine whether or not an adequate number of interviews have been conducted [Kuzel 1999]. Recommendations in the literature and prior experience of the current research team with similar qualitative data collection efforts have informed the expectation that the sample of 28 respondents should provide sufficient data to successfully carry out the research [Guest, Bunce & Johnson 2006]. The proposed sample size allows for the likely attainment of response saturation.
Data Collection Table
U.S. public school districts |
Sample |
13,567 |
28 representatives of U.S. public school districts |
2. Procedures for the Collection of Information
Participants for this data collection will be recruited with the assistance of contractors who have successfully performed similar tasks for NIOSH in the past.
Potential study participants will receive an introductory letter describing the goals of the study, including a brief introduction to Talking Safety curriculum, and explaining the process of conducting the interviews. The letter informs the potential key informants that they will be contacted to schedule an interview. These individuals also are provided contact information for NIOSH and the contractor conducting the interviews. The letter, sent via Federal Express to help highlight its importance, also will include the Talking Safety information sheet (Attachment L). Attachments C and D have the draft advance letters for respondents and superintendents, respectively.
Respondents will be contacted via telephone and email a few days after the advance materials are mailed in order to secure their participation in the interview. The aim of the initial call is to confirm the eligibility of the selected respondent, discuss the interview process and to secure participation. Based on our experience in the pilot, it is expected that three schools will need to be recruited for every completed interview. Screening materials will be sent to all schools simultaneously. Attachments E and F include the draft scripts for the screening calls to respondents and superintendents’ offices, respectively. The specific items to include during the calls include the following:
Sharing the goals for the interview.
Confirming that this person is most appropriate to interview. If not, get a recommendation of another person on district staff and begin the process again.
Stating the process for conducting the interview and how long the interview will take.
Noting how the respondent’s data will be used in the report and how it will be protected after the interview.
Securing agreement to participate in the interview.
Within each region and locales, we will interview the first qualified individual who agrees to the interview. When a target respondent agrees to be interviewed, a time is scheduled at the respondent’s convenience Attachment G includes the follow-up email to be sent thanking the respondent for agreeing to be interviewed, confirming the interview time, and providing contact information in case the respondent needs to reschedule the interview. Forty-eight hours prior to the interview, the same email will be sent again as a reminder to the respondent.
It is estimated that the interviews will take approximately 30 minutes to complete. The interviews will be conducted on the telephone and will be audio-recorded. Participants will not be compensated for these interviews. Individuals will be provided with information about the purpose of the study. They also will be told that whatever information they share will be kept private to the extent permitted by law. Data will be collected using open-ended questions in the interview guide (Attachment I: Discussion Guide) and also will include spontaneous probes to clarify or follow-up on responses to the questions in the guide. In addition to the items directly related to the study, some basic demographic information from participants regarding their position in the organization will be collected. No personally identifiable information, such as their name or birth date, will be collected.
Results from the interviews will be used to gain an understanding about how districts across the United States integrate new materials into school curricula, the extent to which districts currently include workplace health and safety materials in curricula, and whether districts would consider including the vital work readiness skills taught through the Talking Safety curriculum into current programs. NIOSH also would like to understand the challenges districts might face in including these concepts.
The interviews will begin with a consent statement (Attachment H: Interview Opening Statement [Confidentiality and Consent]). The NIOSH Human Subjects Review Board (HSRB) generally waives the documentation of informed consent in studies like this one because the research presents no more than minimal risk of harm to participants and involves no procedures for which written consent is normally required outside of the research context. Each interviewee will be asked to consent verbally before beginning the interview. They also will receive, by email, a handout that contains the name and contact information for the project officer should they have questions related to the study and the contact information for the NIOSH HSRB office should they have questions about their rights or privacy. We do not anticipate any significant risks. Participation/non-participation is entirely voluntary. Participants will be advised that there is a small risk that their participation or the information they provide will not remain confidential. The data collection will not use electronic respondent reporting as attempting to collect the data using a computer would only increase the response time and add an additional level of discomfort for the majority of the respondents. The interviewer will read the items to the respondents and their answers will be recorded on a digital audio device.
The recordings will be saved on a password protected computer and/or on CDs that will be stored in a locking file cabinet for the duration of the project (up to 5 years) in the Taft Building of National Institute for National Occupational Safety and Health (NIOSH) in Cincinnati, OH. Only the investigators will have access to the recordings once the data transcriptions are complete. The recordings will be destroyed at the end of the study. The proposed collection will have no impact on the respondent’s privacy other than their identity being known among the research team members.
3. Methods to Maximize Response Rates and Deal with Nonresponse
The contractor representatives who will be conducting the screening, recruiting and scheduling of participants will be trained by NIOSH researchers regarding the purpose of the overall study and this data collection. It is expected that this training will assist them in responding to any concerns expressed by prospective participants, as well as clarifying item meanings for participants completing the interview. After the pre-screening process is conducted to determine eligible schools districts and the suitable respondent within that district, based on past experience (an interview pre-test), a 95%-100% response rate is expected among the individuals who meet the criteria of the initial screening process. (A 100% response rate was achieved with the pre-test, but the sample size of six districts was smaller). The interview will be administered once to each respondent. It is possible that some participants who initially agree to participate in the interviews will not do so. However, NIOSH assumes, based on the pretest, that the majority (if not all) of participants will complete the interviews.
4. Tests of Procedures or Methods to be Undertaken
Six interviews were conducted as a pre-test of the recruitment methods and interview guide. These six interviews included two different school districts from each of three states (California, New York, and Oregon) where NIOSH currently has demonstration projects for implementation of the curriculum. NIOSH had no prior connection to the school districts. The selection of the school districts within these states was based on input from the external partners in these states regarding districts they would like to learn more about/screen as potential collaborators. Based on these six pretest interviews, modifications were made to the recruitment procedures and interview guide to best assure that the 28 key informant interviews would address the study questions of interest defined above. The pretest also was used to determine the length of time needed to conduct the interviews and the burden hours requested.
These reviews insured the items were understood as intended by the researchers, that no unnecessarily redundant items were retained, and that the interview could be administered within a 30 minute period. The items themselves were derived from insights provided by subject matter experts and a review of the literature.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following individuals provided peer-review regarding statistical aspects, data collection, and/or data analysis:
Liliana Rojas-Guyler, PhD, CHES
Associate Professor
Health Promotion and Education
University of Cincinnati
PO Box 210068
Teachers College 542
Cincinnati OH 45221-0068
Diane
S. Rohlman, PhD
Occupational and Environmental Health
University
of Iowa
Iowa City, IA 52242
(319) 384-4007
The following NIOSH/CDC staff will assist with overseeing the contract deliverables, and designing the data collection and analysis for this project:
Andrea Okun, DrPH
Associate Director for Global Collaborations
Office of the Director (OD)
National Institute for Occupational Safety and Health
Office of the Director
1090 Tusculum Ave
Cincinnati, OH 45226-1998
513-533-8377 (voice)
Email: [email protected]
The following individuals will collect and analyze the data for this project:
Elizabeth Glennie, PhD
Senior Education Research Analyst
Education Studies Division
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709-2194
Hobbs 358 (office)
(919) 541-6434 (phone)
(919) 541-7014 (fax)
Kristina Peterson, Ph.D.
Director, Program for Occupational Safety and Health
RTI International
3040 Cornwallis Road
P.O. Box 12194
Research Triangle Park, NC 27709-2194
Phone: (919) 485-7722
Fax: (919) 541-6604
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |