SUPPORTING STATEMENT FOR THE
2021
and 2023 NATIONAL YOUTH RISK BEHAVIOR SURVEY and 2021 ADOLESCENT
BEHAVIORS AND EXPERIENCES SURVEY
PART B
Submitted by:
Nancy Brener, PhD, Project Officer
Division of Adolescent and School Health
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB
Prevention
1600 Clifton Road, Mailstop US8-1
Atlanta, GA 30329-4027
404-718-8133 (voice); 404-718-8010 (fax)
[email protected]
Centers for
Disease Control and Prevention
Department of Health and Human
Services
October 20, 2020
TABLE OF CONTENTS
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 4
B.1 RESPONDENT UNIVERSE AND SAMPLING METHODS 4
B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION 6
B.3 METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NO RESPONSE 13
B.4 TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN 15
B.5 INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS COLLECTING AND/OR ANALYZING DATA 15
LIST OF ATTACHMENTS
Authorizing Legislation
60-Day Federal Register Notice
60-Day Federal Register Notice Comment(s)
Rationale for Survey Questions
Expert Reviewers for the 1989 Consultations
Report on the 2018 YRBS External Peer Review
Data Collection Checklist
H1. Survey Administrator Script YRBS 2021 – Booklet
H2. Survey Administrator Script YRBS 2023 – Tablet
H3. Survey Administrator/Student Video Script ABES - Web
H4. Survey Administrator/Student Video Script ABES - Web (Spanish Version)
Parental Permission Forms and Supplemental Documents
I1. Parental Permission Form and Fact Sheet (English Version)
I2. Parental Permission Form and Fact Sheet (Spanish Version)
I3. Parental Permission Form Distribution Script
I4. Parental Permission Form Reminder Notice (English Version)
I5. Parental Permission Form Reminder Notice (Spanish Version)
I6. ABES Parental Permission Form and Fact Sheet (English Version)
I7. ABES Parental Permission Form and Fact Sheet (Spanish Version)
I8. ABES Parental Permission Form Distribution Script EDL
I9. ABES Parental Permission Form Distribution Script In Person
I10. ABES Parental Permission Form Reminder Notice EDL (English Version)
I11. ABES Parental Permission Form Reminder Notice In Person (English Version)
I12. ABES Parental Permission Form Reminder Notice EDL (Spanish Version)
I13. ABES Parental Permission Form Reminder Notice In Person (Spanish Version)
IRB Approval Letters
Questionnaires
K1. Youth Risk Behavior Survey Questionnaire
K2. Adolescent Behaviors and Experience Survey Questionnaire
Recruitment Scripts for the Youth Risk Behavior Survey
L1. State-level Recruitment Script for the Youth Risk Behavior Survey
L2. District-level Recruitment Script for the Youth Risk Behavior Survey
L3. School-level Recruitment Script for the Youth Risk Behavior Survey
L4. ABES State-level Recruitment Script
L5. ABES District-level Recruitment Script
L6. ABES School-level Recruitment Script
Example Table Shells
Sampling and Weighting Plan
Data Collection Checklist Supplemental Documents
O1. Letter to Teachers in Participating Schools
O2. Make-up List and Instructions
O3. ABES Letter to Teachers in Participating Schools EDL
O4. ABES Letter to Teachers in Participating Schools In Person
Letters of Invitation
P1. Letter of Invitation to States
P2. Letter of Invitation to School Districts
P3. Letter of Invitation to School Administrators
P4. YRBS Fact Sheet for Schools – Booklet
P5. YRBS Fact Sheet for Schools – Tablet
P6. Letter to Agreeing Schools
P7. ABES Letter of Invitation to States
P8. ABES Letter of Invitation to School Districts
P9. ABES Letter of Invitation to School Administrators
P10. ABES Fact Sheet for Schools
P11. ABES Letter to Agreeing Schools EDL
P12. ABES Letter to Agreeing Schools In Person
Privacy Impact Assessment (PIA)
The universe for both the YRBS and ABES will consist of all regular public and private school students in grades 9, 10, 11, and 12 in the 50 states and the District of Columbia.
Independent samples of schools will be drawn for the ABES and each cycle of the YRBS. For all samples, the sampling frame for schools combines data files obtained from MDR Inc. (Market Data Retrieval, Inc.) and from the National Center for Education Statistics (NCES). The MDR file contains school information including enrollments, grades, race distributions within the school, district, and county, and other contact information for public and non-public schools across the nation. The NCES file includes the Common Core of Data (CCD) for public schools and the Private School Survey (PSS) for non-public schools. When combining data sources to form a sampling frame, duplicates are eliminated so that each school is represented once on the final frame.
Table B-1 displays the current distribution of schools nationally by urban status and type of school.
Distribution of Schools Nationally by Urban Status and School Type
Urban Status |
School Type |
Total |
||
Frequency Percent |
Public |
Private |
Catholic |
|
City-Large |
2835 |
739 |
236 |
3810 |
10.04% |
2.62% |
0.84% |
13.49% |
|
City-Midsize |
1165 |
333 |
90 |
1588 |
4.12% |
1.18% |
0.32% |
5.62% |
|
City-Small |
1400 |
363 |
116 |
1879 |
4.96% |
1.29% |
0.41% |
6.65% |
|
Suburb-Large |
4676 |
1355 |
279 |
6,310 |
16.55% |
4.80% |
0.99% |
22.34% |
|
Suburb-Midsize |
666 |
146 |
16 |
828 |
2.36% |
0.52% |
0.06% |
2.93% |
|
Suburb-Small |
409 |
82 |
21 |
512 |
1.45% |
0.29% |
0.07% |
1.81% |
|
Town-Fringe |
732 |
77 |
15 |
824 |
2.59% |
0.27% |
0.05% |
2.92% |
|
Town-Distant |
1673 |
166 |
52 |
1891 |
5.92% |
0.59% |
0.18% |
6.69% |
|
Town-Remote |
1148 |
89 |
33 |
1270 |
4.06% |
0.32% |
0.12% |
4.50% |
|
Rural-Fringe |
2602 |
587 |
59 |
3248 |
9.21% |
2.08% |
0.21% |
11.50% |
|
Rural-Distant |
3192 |
238 |
5 |
3435 |
11.30% |
0.84% |
0.02% |
12.16% |
|
Rural-Remote |
2580 |
70 |
4 |
2654 |
9.13% |
0.25% |
0.01% |
9.40% |
|
Total |
23078 |
4245 |
926 |
28249 |
81.69% |
15.03% |
3.28% |
100.00% |
|
Sampling or other respondent selection method used: Students will be selected using the procedures described in detail below. To briefly summarize, for the ABES and each YRBS cycle, a nationally representative sample of students will be selected using a three-stage stratified cluster sample. Primary Sampling Units (PSUs - counties, a portion of a county, or a group of counties) and Secondary Sampling Units (SSUs - schools) within sampled PSUs will be selected with probability proportional to size (PPS) selection methods. Within each selected school, one class in each grade will be selected to participate, except in high minority schools where two classes per grade will be selected. All students in selected classes are eligible to participate, with the exception of students who cannot complete the survey independently (e.g., for language or cognitive reasons). A Spanish translation of both questionnaires will be made available to any students who need it.
Expected response rates for the data collection: The average participation rates over the 16 prior cycles of YRBS are 77% for schools and 86% for students. Over the five most recent cycles, the average participation rates are 75% for schools and 84% for students. We consider the historical participation rates, with particular emphasis on the rates obtained during the five most recent cycles, to reflect the current picture of school and student participation in preparing the sample design for the 2021 and 2023 YRBS.
For the ABES, we expect lower-than-historical response rates due to COVID-19. We have assumed a 40% school response rate. In order to attain the desired student yield, the typical YRBS school sample size will be augmented for the ABES.
Actual response rates achieved during the last collection period: During the most recent cycle of the YRBS, conducted in 2019, the participation rates were 75% for schools and 80% for students.
Statistical justification for all sample sizes: The expected student sample size for the YRBS is approximately 19,144 students before nonresponse and the expected student sample size for the ABES is approximately 48,312 before nonresponse. These sample sizes are necessary to meet study precision requirements. The sample sizes are calculated by inflating the sample size that would be required under the assumptions of simple random sampling by historical design effects (to account for the complex sampling design) and participation rates to account for nonresponse at both the student and school levels. It is possible that the sample size for the fall 2021 YRBS might need to be increased if school response rates are still expected to be lower than historical values because of lingering effects of the COVID-19 pandemic.
Statistical Methodology for Stratification and Sample Selection
For the ABES and each YRBS cycle, a probability sample will be selected that will support national estimates among students in grades 9-12 overall and by grade, sex, and race/ethnicity (white, black, Hispanic). The design also will support sex-specific estimates by grade or race/ethnicity and racial/ethnic-specific estimates by grade. A detailed description of the YRBS sampling design may be found in Attachment N. The ABES sampling design is similar except that it has been augmented to account for an expected 40% school response rate.
Sampling Frame. The sampling frame for both the ABES and each YRBS cycle will stratify the 50 states and the District of Columbia by urbanicity and minority composition. The sampling frame is structured into geographically defined units, Primary Sampling Units (PSUs), defined as a county, portion of a county, or a group of contiguous counties. The stratification by minority composition will divide the PSUs into eight groups based on the percentages of blacks and Hispanics in the PSU. This is accomplished in two steps. First, each PSU is stratified into either the Hispanic stratum or black stratum based on whether there is a higher percentage of Hispanic or black enrolled students in the PSU. Each stratum is then subdivided into four strata depending on the percentage of black or Hispanic enrolled students, as appropriate, in the PSU. The eight racial/ethnic-oriented strata will each be further divided by urban status defined as being in one of the 54 largest Metropolitan Statistical Areas (MSA) versus not. In addition, the first-stage PSU sample will be implicitly stratified by geography using 5-digit zip code areas.
Selection of PSUs. For each YRBS cycle, 54 PSUs will be selected, and for the ABES, 81 PSUs will be selected. For all samples, PSUs will be selected with probability proportional to the student enrollment in the PSU within strata. The allocation of PSUs to the first-stage strata will be approximately in proportion to the total enrollment in the PSU. A proportional allocation tends to maximize the precision of overall survey estimates.
Selection of Secondary Sampling Units (SSUs). SSUs are comprised of either a single school (if the school includes each of grades 9-12) or multiple schools “linked” together. An SSU is comprised of multiple “linked” schools when the physical schools do not include all of grades 9-12. This is done to form school-based SSUs that provide coverage for all four grades in each unit. SSUs will be grouped by size as either large or small, depending upon whether they have 28 students or more per grade. For each YRBS cycle, in each selected PSU, at least three large SSUs (28 students or more per grade) will be selected with probability proportional to an aggregate enrollment measure, resulting in 162 selected SSUs (54 PSUs * 3 SSUs). In addition, from a sub-sample of 15 PSUs, one small SSU (fewer than 28 students per grade) will be randomly selected to represent those attending small schools. A total of 177 SSUs will be selected (162 large and 15 small). These 177 SSUs will include approximately 200 physical schools, to account for “linked” schools that are combined during sampling to provide the full span of the grades of interest. For the ABES, a total of 293 large SSUs and 27 small SSUs will be randomly selected.
Selection of Classes. For the ABES and each cycle of the YRBS, classes in each school are randomly selected based on two specific scientific parameters to ensure a nationally representative sample. First, classes must be selected in such a way that all students in the desired grade(s) within the school have a chance to participate. Second, all classes must be mutually exclusive so that no student is selected more than once. In each school, once we have determined the type of class or time period from which classes will be selected, we randomly select the appropriate number of intact classes within each grade. To maintain acceptable school participation rates, it is essential that each school have input into the decision regarding which classes will be sampled in their school. Examples of class sampling frames that have been used in past surveys include a required subject course such as English or all 2nd period classes. As long as the scientific sampling parameters are met, we work with each school to identify a classroom sampling frame that will work best for the school. For the fall 2021 YRBS, additional classes will be selected in each school to test the use of tablets to administer the questionnaire.
Selection of Students. As stated above, for both the YRBS and the ABES, all students in a selected classroom are eligible to participate, with the exception of students who cannot complete the survey independently (e.g., for language or cognitive reasons). As stated above, for each YRBS cycle, we will draw a sample of 54 PSUs, with 3 large SSUs (“full” schools) selected from each PSU, for a total of 162 large SSUs. Based on historical averages, a PSU will supply a sample of 336 students across all of grades 9-12 before non-response (162 SSUs * 4 grades/school * 28 students per grade). The estimated sample yield from these large schools will be 18,144 students before school and student non-response. For the ABES, we will draw a sample of 81 PSUs, for a total of 293 large SSUs. The estimated sample yield from these large schools will be 45,937 students before school and student non-response.
To provide adequate coverage of students in small schools (those with an enrollment of less than 28 students per grade) we also will select one small SSU in each of 15 subsample PSUs for each YRBS sample, therefore adding an additional 15 SSUs to the sample. From historical averages, small SSUs are expected to add 1,000 students before non-response in each YRBS sample. For the ABES, we will select one small SSU in each of 27 subsample PSUs, adding 27 SSUs to the sample. These small SSUs are expected to add 2,375 students before non-response to the ABES sample.
Refusals. School districts, schools, and students who refuse to participate in the study, and students whose parents refuse to give permission, will not be replaced in the sample. Further, any schools that participate in the ABES during spring 2021 that are also drawn into the fall 2021 YRBS sample will not be re-approached for participation in the YRBS and will be considered automatic refusals. We will record the characteristics of schools that refuse for analysis of potential study biases. Accounting for school and student nonresponse, we expect approximately 12,067 participating students in each YRBS cycle and 15,460 participating students in the ABES.
Estimation and Justification of Sample Size
The YRBS and ABES are designed to produce estimates with error margins of ±5 percent:
95 percent confidence for domains defined by grade, sex, or race/ethnicity;
95 percent confidence for domains defined by crossing grade by sex, and race/ethnicity by sex; and
90 percent confidence for domains formed by crossing grade with race/ethnicity.
During the design of the initial YRBS cycles, CDC’s contractor conducted a series of simulation studies that investigated the relationship of various weighting functions to the resulting numbers and percentages of minority students in the obtained samples. New simulation studies are performed periodically to determine opportunities for efficiency while maintaining the target yields across grade, sex, and race/ethnicity to meet the levels of precision required for CDC’s purposes. The 2021 and 2023 YRBS sample size and design will be consistent with the parameters developed for the 2017 and 2019 cycles, but the ABES sample size will be larger to account for anticipated lower-than-historical school response rates due to COVID-19. Minor design refinements are made to account for the changing demographics of the in-school population of students, primarily, the growing number of Hispanic students.
Estimation and Statistical Testing Procedures
Sample data will be weighted by the reciprocal of the probability of case selection and adjusted for non-response. The resulting weights will be trimmed to reduce mean-squared error. Next, the strata weights will be adjusted to reflect true relative enrollments. Finally, the data will be post-stratified to match national distributions of high school students by race/ethnicity and grade.
Variances will be computed using linearization methods. YRBS data are also used for trend analyses where data for successive cycles are compared using statistical testing techniques. Statistical testing methods are used also to compare subgroup prevalence rates (e.g., male versus female students) for each cross-sectional survey.
Confidence intervals vary from estimate to estimate depending upon whether the estimate is for the full population or for a subset such as a particular grade or sex. Within a grouping, confidence intervals also vary depending on the level of the estimate and the design effect associated with the measure. Based on prior YRBS cycles with similar designs and sample sizes, we can expect the following for both YRBS and ABES:
Estimates among students overall or by grade, sex, or race/ethnicity (white, black, Hispanic) will be accurate at ±5 percent at 95 percent confidence.
For racial/ethnic estimates by grade (e.g., 11th grade Hispanics), about 70% will be accurate to within ±5 percent at 90 percent confidence.1
The experience in using these data is that the levels of sampling errors involved are appropriate given the uses of the data for descriptive reporting and trend analysis.
Use of Less Frequent Than Annual Data Collection
As stated in A.6, the YRBS originally was planned and twice approved by OMB as an annual survey. Based on experience, it was determined in 1992 that it would be sufficient to address the programmatic needs of CDC and other Federal agencies to conduct the YRBS biennially. By shifting from an annual to biennial survey starting with the 1993 YRBS, burden has been reduced by half. In 2021, CDC is still conducting the YRBS only once, but since that survey is being moved to the fall because of the COVID-19 pandemic, CDC is adding the ABES in the spring to ensure data are available to monitor student health behaviors and experiences during the pandemic.
Survey Instrument
The YRBS questionnaire (Attachment K1), contains 99 items which can be roughly divided into seven categories. The first category includes five demographic questions. The remaining questions address health risk behaviors in six categories: unintentional injuries and violence; tobacco use; alcohol and other drug use; sexual behaviors that contribute to HIV infection, other sexually transmitted diseases and unintended pregnancies; unhealthy dietary behaviors; and physical inactivity. Obesity (assessed by self-reported height and weight) and other health behaviors also are assessed. The questions are all in a multiple-choice format. During the 2021 cycle, the YRBS questionnaire will be administered as a 12-page optically scannable questionnaire booklet. Additional classes will be selected to test the tablet-based, electronic administration of the questionnaire. Following the 2021 cycle, the YRBS will transition to electronic data collection to reduce burden and improve information gathering via advances in information technology. Beginning with the 2023 cycle, the YRBS will be a digitally-based self-administered questionnaire.
The ABES questionnaire (Attachment K2) is identical to the YRBS questionnaire with two exceptions. First, all YRBS questions that refer to behaviors or experiences that occurred at school, on school property, or on the way to or from school have been revised to include a response option that allows students to indicate that they have not attended school in-person during the timeframe specified in the question. Second, additional questions have been included to assess students’ behaviors and experiences specifically during the COVID-19 pandemic.
Data Collection Procedures
For the YRBS, data will be collected by a small staff of professional data collectors, specially trained to conduct the survey. The time during the school day in which the survey is administered varies by school. This decision is made in coordination with each school to ensure that the type of class or period of the day selected for sampling 1) meets the scientific sampling parameters to ensure a nationally representative sample and 2) results in the least burden/highest possible acceptability for the school. The data collector will have direct responsibility for administering the survey to students. Data collectors will follow a survey administrator script (Attachments H1 and H2).
Teachers will be asked to remain at the front or back of the classroom and not to walk around the room monitoring the aisles during survey administration because doing so could affect honest responses and compromise anonymity. Teachers also will be asked to identify students allowed to participate in the survey and to make sure non-participating students have appropriate alternative activities. The rationale for this is to increase the candor and comfort level of students. The only direct responsibility of teachers in data collection is to distribute and follow up on parental permission forms sent out prior to the scheduled date for data collection in the school. Teachers are provided with a parental permission form distribution script (Attachment I3) to follow when distributing permission forms to students.
For the ABES, teachers of selected classes will be provided with written documentation of all processes they need to follow (Attachment O3 for exclusively distance learning [EDL] schools and Attachment O4 for in-person schools), and will also be provided a link to a brief, high-quality video that can be viewed securely online using an access ID. This video will convey the same information as the written documentation but provides another, more user-friendly means to communicate teachers’ roles and responsibilities, reinforce survey protocol, and provide an overview of key steps for successful survey implementation. Teachers are provided with a parental permission form distribution script/email template (Attachment I8 for EDL schools and I9 for in-person schools) to follow when distributing permission forms to students or parents. Teachers also will be provided with sign-in information for students. For students attending schools in-person at least part-time, teachers will distribute sign-in cards with student access IDs when the students are physically attending school. For students attending school in an EDL environment, teachers will provide a classroom-level sign-in to all students in a selected class via the school’s established teacher-student communication channels. Upon sign in, all records are associated with a unique student-level ID in the backend database, but none of these student access IDs can be traced to any individual student.
Using any internet-connected device, students will log in to the ABES via a secure URL using the sign-in information provided by the teacher. Students will be instructed to find a comfortable place where they can take the survey in private. After viewing a brief video similar to the Survey Administrator Script (Attachment H3, Student Video Script), students will complete the web-based questionnaire. The interface allows respondents to move easily through the questions. Should a connection be interrupted, respondents will be able to start, stop, and return to their survey on the last-viewed screen without data loss. To eliminate the possibility of students intentionally or unintentionally using the same sign-in ID twice, once a questionnaire is submitted by the student, the record cannot be accessed again.
For both the YRBS and ABES, in general, data collection procedures have been designed to ensure that:
Protocol is followed in obtaining access to schools
Everyday school activity schedules are disrupted minimally
Administrative burden placed on teachers is minimal
Parents give informed permission to participate in the survey
Anonymity of student participation is maintained, with no punitive actions against nonparticipants
Alternative activities are provided for nonparticipants
Control over the quality of data is maintained
Student anonymity for the tablet version of the YRBS that will be tested in 2021 and rolled out in 2023 will continue to be maintained by the same processes implemented for the scannable booklets. At the start of survey administration, professionally trained YRBS data collectors will remind students that their responses will be captured anonymously (Attachment H1). Students will be instructed to hand their tablet to the data collector at the conclusion of survey administration. Following data collection at each school, student data will be uploaded to a central repository using secure protocols and erased from the tablets.
For both the YRBS and ABES, the Data Collection Checklist (Attachment G) is completed by teachers to track which students have received parental permission to participate in the data collection. The teachers receive instructions on completing the Data Collection Checklist in the “Letter to Teachers in Participating Schools” (Attachment O1 for YRBS, Attachment O3 for ABES EDL schools, and Attachment O4 for ABES in-person schools). For the YRBS, the data collector will utilize the information on the Data Collection Checklist to identify students eligible for a make-up survey administration; this information will be recorded by the data collector on the “Make-up List and Instructions” document (Attachment O2).
Obtaining Access to and Support from Schools
For both the YRBS and ABES, all initial letters of invitation will be on CDC letterhead from the Department of Health and Human Services and signed by Kathleen Ethier, PhD, Director, Division of Adolescent and School Health, National Center for HIV, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention. The procedures for gaining access to schools will have three major steps:
Notify state education agencies (SEAs) in states with sampled schools and inform states of their schools’ selection into the ABES sample or national YRBS sample. Obtain names of supportive school district contacts and general guidance on working with the selected school districts and schools in the state, and request state-level support for the survey prior to sending district invitations. State-level letters of support for the ABES will not be solicited because of time constraints.
Invite school districts in which selected schools are located to participate in the study. For Catholic schools and other private schools, invite the office comparable to the school district office (e.g., diocesan office of education). Obtain approval to invite sampled schools to participate. Verify existence of schools, grade ranges, and other information as needed. Request that the school district notify schools that they may anticipate being contacted about the survey. Request general guidance on working with the selected schools and district scheduling information.
Once cleared at the school district level, invite selected schools to participate. Verify information previously obtained about the school. Present the burden and benefits of participation in the survey. Obtain approval for participation at the school level. After a school agrees to participate, develop a customized plan for collection of data in the school (e.g., select classes and schedule survey date). Ensure that all pre-survey materials reach the school well in advance of when they are needed. Maintain contact with schools until all data collection activities have been completed.
Prior experience suggests the process of working with each state education agency, school district, and school will have unique features. Discussions with each education agency will recognize the organizational constraints and prevailing practices of the state. Scripts for use in guiding these discussions for the YRBS may be found in Attachments L1 (state-level), L2 (district-level), and L3 (school-level). Parallel scripts for the ABES may be found in Attachments L4, L5, and L6. Attachment P contains copies of letters of invitation to states (Attachment P1 for YRBS and P7 for ABES), school districts (Attachment P2 for YRBS and P8 for ABES), and school administrators (Attachment P3 for YRBS and P9 for ABES). Attachment P also contains the YRBS Fact Sheet for Schools (Attachments P4 and P5) and the ABES Fact Sheet for Schools (Attachment P10). The letters to be sent to schools once they have agreed to participate are found in Attachment P6 for YRBS, P11 for ABES ED schools, and P12 for ABES in-person schools.
Informed Consent
The parental permission form and fact sheet (Attachments I1 and I2 for YRBS and Attachments I6 and I7 for ABES) inform both the student and the parent about an important activity in which the student has the opportunity to participate. By providing adequate information about the activity, it ensures that permission will be informed. The permission form indicates that a copy of the questionnaire will be available for review by parents at their child’s school. The parental permission forms will be made available in both English and Spanish.
For both surveys, a waiver of written student assent was obtained for the participation of children because this research presents no more than minimal risk to subjects, parental permission is required for participation, the waiver will not adversely affect the rights and welfare of the students because they are free to decline to take part, and it is thought that some students may perceive their responses are not anonymous if they are required to provide stated assent and sign a consent/assent document. Students are told “Participating in this survey is voluntary and your grade in this class will not be affected, whether you answer the questions or not.” Completion of the survey implies student assent.
Quality Control
Table B.2 lists the major means of quality control for both the paper and electronic data collection methodologies. As shown, the task of collecting quality data begins with a clear and explicit study protocol, regardless of mode.
For the paper questionnaire, data collection quality control includes visual inspection and scanning of collected booklets in order to produce a single data set for analysis. For the digitally-based questionnaire, data collection quality control begins with accurate programming of the YRBS and ABES questionnaires and concludes with the regular submission of data records to a secure central repository.
Because the ultimate aim is production of high-quality data sets and reports, various quality assurance activities will be applied during the data collection phase. Subsequent to YRBS data collector training, measures must be taken to reinforce training, to assist field staff who express/exhibit difficulties completing data collection activities, and to verify compliance with data collection protocols. Similarly, teachers administering the ABES will be provided with customized technical assistance and outreach to ensure all protocols typically followed by a data collector are followed by the teachers. Also, early inspection of a preliminary data set is necessary to ensure data integrity.
Table B.2 - Major Means of Quality Control
Survey Step |
Quality Control Procedures |
Mailing to Districts and School |
|
Telephone Follow-up Contacts |
|
Previsit Logistics Verification (YRBS only) |
|
Data Collector Training and Supervision of School Visits (YRBS only) |
|
Teacher Adherence to ABES Protocols (ABES only) |
|
Questionnaire Programming and Testing (Electronic) |
|
Computer Scanning (Paper) |
|
Receipt Control (Electronic) |
|
Receipt Control (Paper) |
|
Data Review (Electronic) |
|
Data Review |
|
Merge data sets of scanned paper and electronic survey data |
|
Expected Response Rates
Historically for the YRBS, the overall response rate (product of school response rate and student response rate) has ranged from 60% to 71%, with a 16-cycle average of 66%. For the purposes of 2021 and 2023 YRBS sample design, we have conservatively assumed an overall response rate of 63%, the average over the five most recent survey cycles. The addition of a $500 token of appreciation for each participating YRBS school (as suggested by OMB in 1999) has helped maintain stable school participation rates, and the $250 token of appreciation for each participating ABES school should help with reaching expected school participation rates. The participation rates established by the YRBS are the result of the application of established procedures for maximizing school and student participation and minimizing non-response as described below. These will also be applied to the ABES, unless otherwise noted.
Methods for Maximizing Responses
To increase the likelihood of an affirmative decision to participate, we will:
(1) work through the SEA and/or state health agency to communicate state-level support
(2) indicate that the survey is sponsored by CDC and has support of Federal and state agencies
(3) for the YRBS, solicit endorsement from and convey to school districts and schools that the survey has the endorsement of many key national education and health associations, such as the American Academy of Pediatrics, American Association of School Administrators, Association of State and Territorial Health Officials, Council of Chief State School Officers, National Association of Secondary School Principals, National Association of State Boards of Education, National Catholic Educational Association, National Education Association, National PTA, and the National School Boards Association
(4) maintain a toll-free hotline to answer questions from school district and school officials, teachers, parents, and students throughout the process of recruiting schools and obtaining parental permission for student participation
(5) comply with district requirements in preparing written proposals for survey clearance
(6) convey a willingness to appear in person, if needed, to present the survey before a school board, research committee, or other local entity tasked with reviewing the survey
(7) offer schools a monetary token of appreciation of $500 for YRBS ($250 for ABES), consistent with previous recommendations from OMB, and implemented on the national YRBS since 2001.
Once a school has agreed to participate, we collaborate with each school to determine the class selection method that fits best with their school environment and meets the scientific protocol and to schedule a survey administration date that is convenient for their school calendar. As indicated in A.16, when collecting data during the spring semester, it is highly desirable to complete data collection before the final 1-2 months of school. At that point in the academic year, schools are typically focused on testing and attendance can be unstable, particularly among twelfth grade students. To further encourage participation among students in selected classes, we will recommend that schools help to advertise the survey through the principal’s newsletter, PTA meetings, and other established means of parental communication.
Methods for Handling Non-Response
We distinguish among six potential types of nonresponse problems: refusal to participate by a selected school district, school, teacher, parent, or student; and collection of incomplete information from a student. To minimize refusals at all levels--from school district to student--we will use a variety of techniques, emphasizing the importance of the survey and the value of the data gathered in allowing for the continued monitoring of factors that influence youth health. All participating districts and schools will be notified when the survey results are published and data are available for download from CDC’s website, which districts and schools may utilize in supporting grant applications.
Dealing with refusals from parents, teachers, and students require different strategies. Parental permission form reminders (Attachments I4 and I5 for YRBS, I10 and I12 for ABES EDL schools, and I11 and I13 for ABES in-person schools) will be sent to parents who have not returned parental permission forms within an agreed upon time period (e.g., 3 days); those who do not respond to the reminder will be sent a second and final reminder. The permission form will provide a telephone number at CDC that parents may call to have questions answered before agreeing to give permission for their child's participation. Permission forms will be available in English, Spanish, and other languages as required based on dominant languages spoken by parents in selected schools. Project staff will be available to answer questions from parents who remain uncertain of permission.
Teacher refusals to cooperate with the study are not expected to become a cause for concern because school leadership will already have agreed to participate. Refusals by students who have parental permission to participate are expected to be minimal. No punitive action will be taken against a nonconsenting student. Nonconsenting students will not be replaced. Data will be analyzed to determine if student nonresponse introduces any biases.
To minimize the likelihood of missing values on the questionnaire, students will be reminded within the instrument instructions and verbally by the survey administrator (for the YRBS) to review their answers prior to submitting their paper booklet or electronic questionnaire. For the ABES, students have the ability to review their responses before they submit their questionnaire. They can choose to start back at the first question and proceed all the way through, or they can jump directly to unanswered questions. Validations at the submission screen present students with a list of any unanswered questions, which they can then choose to answer or submit with them being blank.
For the optically scannable questionnaire booklets utilized for the 2021 YRBS, students will be prompted by the written survey instructions and reminded verbally through the survey administrator script to review their answers before turning it in to verify that: (1) each question they wished to answer has been answered, (2) only one oval is filled in for each question with the exception of the question on race/ethnicity, and (3) each response has been entered with a No. 2 pencil, fills the oval, and is dark. A No. 2 pencil will be provided to each survey participant to reduce the likelihood that responses will not scan properly, which would produce missing values. In addition, when completed questionnaires are visually scanned later at project headquarters, any oval that is lightly filled in will be darkened (unless they appear to be erasures) and stray marks will be erased before the forms are scanned. Missing values for an individual student on the survey will not be imputed.
For the electronic surveys , students will be prompted within the survey application instructions and reminded verbally through the survey administrator script (for YRBS questionnaires administered via tablet) to review their answers to verify that each question they wished to answer has been answered before submitting their survey responses electronically. Missing values for an individual student on the survey will not be imputed.
YRBS questionnaire items were originally tested by the NCHS laboratories. The 1993 special issue of Public Health Reports on the development of the Youth Risk Behavior Surveillance System describes the development and testing process. A limited pretest of the questionnaire on nine respondents was conducted in November 1989 by the contractor in the Prince George's County, Maryland school system in accord with OMB guidelines. The pretest was conducted to:
Quantify respondent burden
Test survey administrator instructions and procedures
Verify the overall feasibility of the survey approach
Identify needed changes in the instruments or instructions to control/reduce burden
The pilot test sharpened the articulation of certain survey questions and produced an empirical estimate of the survey burden.
The YRBS questionnaire has been used extensively in 16 prior national school-based surveys approved by OMB, as well as at the state and local levels. Further pilot testing in accord with OMB guidelines has been performed on new and potential questions.
Under OMB's prior review of the YRBS (OMB No. 0920-0493, expiration, 9/30/2019), a Notice of Action was issued that requested the study undergo an external peer review prior to submitting the next package for approval. To ensure continuous scientific rigor of the sample design, best practices for recruitment, and efficient strategies to maximize participation rates, a panel of four experts was convened in April 2018. Four experts in survey methodology, school-based data collection, and health surveys commented on the YRBS methodology and offered recommendations for improvement. Specifically, the topics of discussion were frame development and sampling design, maximizing participation, transition to a mixed mode methodology and YRBS strategy to address emerging topics. A summary of the panel's recommendations and CDC's concurrence with those recommendations can be found in Attachment F.
Statistical aspects of the study have been reviewed by the individuals listed below.
Ronaldo Iachan, PhD
ICF
530 Gaither Road, Suite 500
Rockville, Maryland 20850
Phone: (301) 572-0538
E-mail: [email protected]
Richard (Lee) Harding, MS
ICF
530 Gaither Road, Suite 500
Rockville, Maryland 20850
Phone: (301) 572-0524
E-mail: [email protected]
Within the agency, the following individual will be responsible for receiving and approving contract deliverables and will have primary responsibility for data analysis:
Nancy Brener, PhD
Team Lead, Survey Operations Team
Division
of Adolescent and School Health
Centers for Disease Control
and Prevention
1600 Clifton Road, NE
Mailstop US8-1
Atlanta, GA 30329
Phone: 404-718-8133
Email: [email protected]
The representative of the contractor responsible for conducting the planned data collection is:
Alice Roberts, MS
Project Director
ICF
530 Gaither Road, Suite 500
Rockville, Maryland 20850
Phone: (301) 572-0290
E-mail: [email protected]
1 Based on empirical results and simulations.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Windows User |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |