SUPPORTING STATEMENT FOR THE
STUDY TO EXAMINE WEB-BASED ADMINISTRATION OF THE
YOUTH RISK BEHAVIOR SURVEY
PART A
Submitted by:
Danice K. Eaton, MPH, PhD, Project Officer
Division of Adolescent and School Health
National Center for Chronic Disease Prevention and Health Promotion
4770 Buford Hwy, NE, MS K-33
Atlanta, GA 30341
770-488-6143
(voice); 770-488-6156 (fax)
[email protected]
Centers
for Disease Control and Prevention
Department of Health and Human Services
September 2007
TABLE OF CONTENTS
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
2. Purposes and Use of Information Collection
a. Survey Purposes
b. Anticipated Uses of Results
3. Use of Improved Information Technology and Burden Reduction
4. Efforts to Identify Duplication and Use of Similar Information
5. Impact on Small Businesses or Other Small Entities
6. Consequences of Collecting the Information Less Frequently
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
a. Federal Register Announcement
b. Consultations
9. Explanation of Any Payment or Gift to Respondents
10. Assurance of Confidentiality Provided to Respondents
11. Justification for Sensitive Questions
12. Estimates of Annualized Burden Hours and Costs
a. Estimated Burden Hours
b. Estimated Burden Costs
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
14. Annualized Cost to the Government
15. Explanation for Program Changes or Adjustments
16. Plans for Tabulation and Publication and Project Time Schedule
a. Tabulation Plans
b. Publication Plans
c. Time Schedule for the Project
17. Reason(s) Display of OMB Expiration Date is Inappropriate
18. Exceptions to Certification for Paperwork Reduction Act Submissions
REFERENCES
LIST OF APPENDICES
Authorizing
Legislation
60-Day
Federal Register Notice
60-Day Federal Register Notice Comment
D. Data Collection Instrument for Principals – “Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys”
E. Data Collection Instrument for Students – “Student Health Survey”
E1. “Student Health Survey” Without Skip Patterns
E2. “Student Health Survey” With Skip Patterns
F. Data Collection Instrument for Principals Supplemental Documents
F1. Letter of Invitation
F2. Consent Form
G. Data Collection Instrument for Students (“Student Health Survey”) Supplemental Documents
G1. Parental Permission Form Distribution Script
G2. Parental Permission Form and Fact Sheet (English Version)
G3. Parental Permission Form and Fact Sheet (Spanish Version)
G4. Parental Permission Form Reminder Notice (English Version)
G5. Parental Permission Form Reminder Notice (Spanish Version)
G6. Questionnaire Administration Guides
G7. Data Collector Confidentiality Agreement
H. School Recruitment Script for the “Student Health Survey”
I. School Recruitment Script for the “Student Health Survey” Supplemental Documents
F1. School Letter of Invitation and Fact Sheet
F2. Letter to Agreeing Schools
J. Data Collection Checklist for the “Student Health Survey”
K. Data Collection Checklist for the “Student Health Survey” Supplemental Documents
K1. Student Questionnaire Letter to Teachers in Participating Schools, Paper-and-Pencil or Computer Lab Conditions
K2. Student Questionnaire Letter to Teachers in Participating Schools, “On-Their-Own” Condition
K3. Make-up List and Instructions
L. 2008 Methodological Study IRB Approval Letter
M. 2008 Methodological Study Table Shells
N. Detailed Sampling and Weighting Plan for the Principal Data Collection
A.1. CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY
This statement supports a request to obtain approval for a new information collection to conduct a methodological study to assess the feasibility of administering to students a web-based questionnaire assessing risk behaviors. The information collection will be based on two questionnaires.
The first data collection will be a questionnaire (“Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys,” Appendix D) administered to a national probability sample of high school principals. This instrument will assess perceptions of the feasibility and acceptability of using web-based data collection methods for student surveys and assessments. For ease of reference, this activity is referred to as the “principal data collection.”
The second data collection will be a questionnaire called the Student Health Survey administered to a convenience sample of 9th and 10th grade students to assess how self-reported risk behavior varies across four study conditions. For ease of reference, activities associated with this instrument are referred to as the “student data collection.” Selected classrooms of students will be randomly assigned to one of four conditions to complete the “Student Health Survey.” In condition 1, students will complete a paper-and-pencil version of the questionnaire (Appendix E1) as an intact class in their regular classroom. In condition 2, students will complete a web-based version of the questionnaire (Appendix E1) without programmed skip patterns as an intact class in the school computer lab. In condition 3, students will complete a web-based version of the questionnaire (Appendix E2) with programmed skip patterns (to take fuller advantage of the benefits of web-based administration) as an intact class in the school computer lab. In condition 4, students will complete a web-based version of the questionnaire (Appendix E1) without programmed skip patterns on their own time, using any computer available to them and at the time of their choosing (i.e., “on your own” condition). The feasibility of web-based data collection will be assessed by determining whether risk behavior prevalence estimates are comparable across study conditions without incurring a significant decrease in participation rates and questionnaire completeness (number of questions with missing responses) in the web-based conditions compared to the paper-and-pencil condition.
OMB has issued guidance indicating that all ongoing and periodic data collections attempt to offer a web-based mode for responding. One ongoing data collection is the National Youth Risk Behavior Survey (YRBS) conducted by the Centers for Disease Control and Prevention. The National YRBS, currently approved under OMB Number 0920-0493 (expiration 11/07), is an ongoing survey among high school students that assesses priority health-risk behaviors among US high school students. The National YRBS was first cleared by OMB in 1990. Since 1991, the National YRBS has been conducted biennially during odd-numbered years (1991, 1993, 1995, 1997, 1999, 2001, 2003, 2005, and 2007) using a paper-and-pencil mode of data collection. To be responsive to OMB’s guidance, this study seeks to identify the feasibility of adopting a web-based mode of data collection for school-based surveys of risk behaviors such as the National YRBS.
Justification for the 2008 methodological study is the increased demand for methodological information on the sources of variation among estimates of health-risk behaviors, especially among adolescents. In May 2000, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Department of Health and Human Services (DHHS) commissioned five papers (Cowan, 2001; Fendrich & Johnson, 2001; Fowler, 2001; Harrison, 2001; Sudman, 2001) under the project “Examining Substance Abuse Data Collection Methodologies.” The five papers examined variations in coverage, sample design, weighting, and data collection methodology in three national surveys: Monitoring the Future (MTF; no OMB control number because it is administered through a grant), the National Survey on Drug Use and Health (NSDUH, formerly the National Household Survey on Drug Abuse, or NHSDA; OMB No. 0930-0110, expiration 01/07), and the National YRBS (OMB No.: 0920-0493; expiration 11/07). The review focused on issues that could affect prevalence estimates for the teenage population targeted by these three national surveys.
An apparent consensus of this paper series was that methodological studies are needed to assess the effects of various aspects of survey methodology. As a result, recommendations of panelists helped to shape two OMB-approved methodological studies. The first (OMB No.: 0920-0534, expiration 12/02), conducted in 2002, was designed to assess the effects of question wording, perceptions of privacy, and appeals for honesty on prevalence estimates. The second methodological study (OMB No.: 0920-0611, expiration 12/04), conducted in 2004, evaluated the effects of setting (home vs. school) and mode (paper-and-pencil vs. laptop computer) and the interaction of setting and mode on prevalence estimates. The results of these studies have been shared with ASPE, the agency representatives of NSDUH, and MTF, and the subcommittee of the DHHS Data Council charged with better coordination of national surveys gathering data on adolescent drug use, notably the National YRBS, MTF, and NSDUH. In addition, manuscripts describing results of the 2002 and 2004 methodological studies have been published in refereed journals and shared directly with OMB (Brener et.al., 2003; Brener et. al., 2004; Brener et. al., 2006).
The proposed new methodological study addresses concerns raised by authors of the original ASPE-commissioned papers and builds directly on the 2004 methodological study. The 2004 study sought to test Sudman’s (2001) hypothesis that differences in setting (home vs. school) might be the single most important cause of differences in survey results. More specifically, the 2004 study accepted the challenge from Fendrich and Johnson (2001) to test the greater tendency for youth to disclose health risk behaviors in school-based rather than home-based data collections through a carefully designed experimental study with consistent administration procedures across settings. In doing so, it also sought to test the effects of mode (paper-and-pencil vs. laptop computer) and of several factors that might determine whether students respond and how truthfully they respond, as suggested by Sudman (2001) and Harrison (2001), including respondents’ privacy (both actual and perceived), perceived anonymity, trust, and comfort and experience with computers. The 2004 study provided support for the hypothesis that mode does not affect prevalence estimates, but setting (school vs. home) does.
A fundamental shortcoming of the 2004 methodological study is that it was conducted under somewhat contrived conditions and, as a result, tells us little about the true feasibility of conducting ongoing risk behavior surveillance using a web-based questionnaire in schools. In the 2004 study, we brought laptops into participating schools and did not require the capacity to link to the internet. A more realistic assessment of feasibility involves working with schools to conduct a web-based questionnaire under conditions that that more closely approximate what would be expected of them when participating in an ongoing national surveillance system. A feasible scenario would involve arranging with participating schools to use their computer labs for data collection. When undertaking such a feasibility assessment, it also makes sense to look at the effects on prevalence estimates of the presence or absence of skip patterns, since the ability to program skip patterns into the questionnaire is one benefit of web-based data collection. This can be accomplished by having two groups of students complete a web-based questionnaire in a computer lab in each participating school, one with and the other without programmed skip patterns. Finally, if administration of web-based questionnaires is proven feasible, the educational community next may logically ask why—given that nearly all students have access to computers outside of school—students cannot be selected through schools but then allowed to complete questionnaires “on their own,” using a computer and at a time of their own choosing. Thus, the final “on your own” condition is included as one of the four conditions under which the student questionnaire will be administered. Results from all three web-based questionnaire conditions will be compared to the first condition, the traditional paper-and-pencil questionnaire in a regular classroom setting without skip patterns. Further, as in the 2004 methodological study, factors affecting whether students respond and how truthfully they respond, will be examined.
If students completing a web-based questionnaire in a computer lab (conditions 1 and 2) have prevalence estimates comparable to those obtained with students completing a paper-and-pencil questionnaire in the classroom (condition 1), with no appreciable loss in participation rates and data completeness, then the results from the student questionnaire will make a strong case for conducting school-based surveys of student behaviors using school computer labs. Also, if the student data collection show students in the “on-their-own” condition (condition 4) produce prevalence estimates comparable to the paper-and-pencil condition (condition 1), with no appreciable loss in participation rates and data completeness, then these results will make a case for allowing individual administration outside the classroom setting in school-based surveys.
Results from the principal data collection will complement results from the student data collection. Although the student data collection will tell us if web-based and paper-based questionnaires produce comparable prevalence estimates and participation rates, it cannot tell us the extent to which web-based data collection can or will be supported in schools nationally. Thus, the principal data collection will provide important information about the feasibility and acceptability of web-based data collections in schools nationally.
The legal justification for the study may be found in Section 301 of the Public Health Service Act (42 USC 241) (Authorizing Legislation, Appendix A).
A.2 PURPOSES AND USE OF INFORMATION COLLECTION
A.2.a Purpose of Information Collection
The purpose of the 2008 methodological study is to identify the feasibility of assessing student risk behavior participation using a web-based questionnaire. Feasibility will be assessed by employing a principal data collection and a student data collection to answer the following research questions:
To what extent do principals already have experience in administering web-based data collections in computer labs at their school?
How difficult do principals think it would be to schedule time in the school computer labs for classrooms to complete a web-based data collection?
Do
principals prefer paper-and-pencil to web-based modes for collecting
data from classrooms of students?
Do risk behavior prevalence estimates, student participation rates, and data completeness (i.e., the number of questions with missing responses) vary by student data collection condition?
Do risk behavior prevalence estimates vary by the presence or absence of programmed skip patterns on the questionnaire?
Do risk behavior prevalence estimates vary by group (i.e., classroom) or individual (i.e., “on your own”) questionnaire administration?
To what extent can the above effects be explained by students’ privacy (both actual and perceived), perceived anonymity, trust, and comfort and experience with computers?
A.2.b Use of Information Collection
The proposed methodological study will be conducted during Winter/Spring 2008. Results will inform the design of future YRBS data collections. In addition, the information generated by the 2008 methodological study will be used by several Federal agencies, including CDC. In particular, ASPE will be interested in this project. CDC, SAMHSA, and NIDA share similar interests in the study results since surveys conducted by each agency differ in both mode and setting of typical administration. The National YRBS (OMB No.: 0920-0493; expiration 11/07) and MTF are typically administered as paper-and-pencil questionnaires with groups of students in schools; the NSDUH (OMB No.: 0930-0110; expiration: 01/07) is typically administered as a computer assisted self-interview (CASI) questionnaire with individuals in households. The data resulting from the planned 2008 methodological study will contribute to ongoing discussions among these agencies and within a subcommittee of the DHHS Data Council regarding the differential prevalence estimates generated using different survey methodologies.
Within
the National Center for Chronic Disease Prevention and Health
Promotion (NCCDPHP), data from school-based risk behavior
surveillance systems are used by the the Division of Adolescent and
School Health, the Division of Cancer Prevention and Control, the
Division of Nutrition, Physical Activity, and Obesity, the Division
of Adult and Community Health, the Division of Reproductive Health,
the Office of the Director, and the Office on Smoking and Health.
Data from such surveillance systems also are used by other centers
within CDC, including the National Center for HIV/AIDS, Viral
Hepatitis, STD, and TB Prevention (NCHHSTP); the National Center for
Health Statistics (NCHS); and the National Center for Injury
Prevention and Control (NCIPC). As users of school-based risk
behavior surveillance data, each of these entities have an interest
in how the collection of the data may influence reporting of risk
behaviors.
The information also will be used by state and local governments, nongovernmental organizations, and others in the private sector that engage in student risk behavior surveillance or use data from such surveillance systems.
A.3 USE OF IMPROVED INFORMATION TECHNOLOGY AND BURDEN REDUCTION
Respondents who participate in the principal data collection will be offered the option of responding either on optically-scannable questionnaire booklets or via a web-based questionnaire. We plan to keep careful track of the mode of response they choose, especially as they relate to their responses on the questionnaire. The principal data required for the study cannot be accessed from currently existing automated databases. During questionnaire design, every effort has been made to limit respondent burden. There are no legal or technical barriers to the use of improved information technology to reduce burden.
To reduce burden associated with the student data collection, student data will be collected either on optically-scannable questionnaire booklets, a method that is currently used with other large scale data collection efforts among students such as the National YRBS, MTF, or, using a web-based questionnaire similar to the CASI administration format currently used by NSDUH. A quarter of the selected student respondents will be offered paper-and-pencil questionnaires (condition 1) and three quarters will be offered a web-based questionnaire (conditions 2, 3, and 4). The student data required for the study cannot be accessed from currently existing automated databases. During questionnaire design, every effort has been made to limit respondent burden. There are no legal or technical barriers to the use of improved information technology to reduce burden.
A.4 EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION
CDC conducts ongoing searches of all major educational and health-related electronic databases, reviews related literature, consults with experts in behavioral epidemiology and survey research, and maintains continuing communications with Federal agencies with related missions. These efforts have identified no previous, current, or planned comprehensive effort to conduct a methodological study assessing the feasibility of administering a web-based questionnaire to students, including examining the effect of questionnaire mode (paper-and-pencil versus web-based), group vs. individual administration, and skip patterns on the prevalence of risk behaviors, including: unintentional injury and violence, tobacco use, alcohol and drug use, sexual behaviors, unhealthy dietary behaviors, and physical activity among high school students. Further, CDC consulted specifically with those responsible for the MTF and NSDUH surveys, and confirmed that no similar studies had been conducted or were going to be conducted. Ongoing searches and consultations with experts also revealed no evidence that a similar study of principals’ perceptions of the feasibility and desirability of student web-based data collection was conducted, is currently ongoing, or has been planned.
A.5 IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES
The planned data collection does not involve small businesses or other small entities.
A.6. CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY
This is a one time data collection. Without this study, CDC will lack information about the feasibility of implementing a web-based data collection option for the YRBS in future years. CDC would also lack information about the variability of results obtained by mode and setting.
A.7. SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINES OF 5 CFR 1320.5
The data collection will be implemented in a manner consistent with 5 CFR 1320.6. No special circumstances are applicable to this proposed study.
A.8. COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE THE AGENCY
A.8.a Federal Register Announcement
CDC published a Federal Register notice of the proposed data collection on March 6, 2007 (Vol. 72, Number 43, Page 9949-9950) (Appendix B). One comment requesting a copy of the study protocol and questionnaires was received (Appendix C).
A.8.b Consultations
The plans for the OMB-approved 2002 and 2004 methodological studies and for the proposed 2008 methodological study emerged from the previously discussed ASPE project “Examining Substance Abuse Data Collection Methodologies,” which resulted in five published papers on the effects of variations in coverage, sample design, weighting, and data collection methodology in three national surveys on prevalence estimates among teenagers. The DHHS Data Council reviewed these papers and formed a subcommittee charged with better coordination of surveys gathering data on drug use among adolescents, notably the National YRBS, MTF, and NSDUH. The subcommittee has approximately 15 members, including the key representatives of CDC, NIDA, and SAMHSA responsible for the National YRBS, MTF, and NSDUH. Since its formation, this subcommittee typically has met 3 times per year. The members of the subcommittee outside of CDC who were most closely involved in discussions of the 2004 and this proposed 2008 methodological studies are listed below:
Jim Colliver, Ph.D.
Statistician
SAMHSA
1 Choke Cherry Road
Room 7-1015
Rockville, MD 20857
Phone: (240) 276-1252
Email: [email protected]
Joe Gfroerer
Division Director
Office of Applied Studies
SAMHSA
1 Choke Cherry Road
Room 7-1015
Rockville, MD 20857
Phone: (240) 276-1262
Email: [email protected]
Dale Hitchcock
ASPE
200 Independence Avenue, SW
Washington, DC 20201
Phone: (202) 690-7100
Email: [email protected]
Marsha Lopez
NIDA
6001 Executive Boulevard
Room 5156, MSC-9589
Bethesda, MD 20892
Phone: (301) 443-6504
Email: [email protected]
Jim Scanlon
ASPE
200 Independence Avenue, SW
Washington, DC 20201
Phone: (202) 690-7100
Email: [email protected]
On January 16, 2003, an expert panel was convened to obtain input on the 2004 methodological study (OMB No.: 0920-0611, expiration 12/04). Issues discussed included study design, sampling, logistics, and technological issues. The panel included representatives of ASPE, NIDA, SAMHSA, the U.S. Department of Education, and the private and university-based research community. A list of the expert panel participants for the 2004 methodological study included:
Brett Brown, Ph.D.
Child Trends
4301 Connecticut Avenue, NW
Suite 100
Washington, DC 20008
Phone: (202) 362-5580
Email: [email protected]
Kathryn Chandler
U.S. Department of Education
National Center for Education Statistics
1990 K Street, NW, Room 9017
Washington, DC 20006
Phone: (202) 502-7486
Email: [email protected]
Sonia Chessen, M.A.
DHHS/Office of the Assistant Secretary for Planning and Evaluation
U. S. Department of Health and Human Services
Room 415F
200 Independence Avenue, SW
Washington, DC 20201
Phone: (202) 690-8471
Email: [email protected]
Jim Colliver, Ph.D.
See contact information above
Joe Gfroerer
See contact information above
Meredith Kelsey, Ph.D.
DHHS/Office of the Assistant Secretary for Planning and Evaluation
200 Independence Avenue, SW
Room 424E.16
Washington, DC
Phone: (202) 690-6692
Email: [email protected]
Bronwyn Nichols, Ph.D.
Project Director
National Opinion Research Corporation
1155 East 60th Street
Chicago, IL 60637
Phone: (773) 256-6092
Email: [email protected]
Patrick M. O’Malley, Ph.D.
University of Michigan
Institute for Social Research
426 Thompson Street
Room 2320
Ann Arbor, MI 48106-1248
Email: [email protected]
Phone: (734) 763-5043
Judy Thorne, Ph.D.
Westat
1650 Research Boulevard
Rockville, MD 20850
Phone: (301) 610-5591
Email: [email protected]
Charles F. Turner, Ph.D.
Research Triangle Institute
1635 M Street, NW
Washington, DC 20036
Phone: (202) 690-7100
Email: [email protected]
Specifically for the purposes of planning the 2008 study, additional consultations were held with:
Terry Batson, Ph.D.
Associate Researcher
University of Wisconsin—Milwaukee
Center for Urban Initiative & Research
Engelmann Hall - Room B50
Milwaukee, WI 53201
Phone: (414) 229-3104
Email: [email protected]
Doug Caver
President
Smart Track
133 Executive Drive
Suite A
Madison, MS 39110
Phone: (866) 714-8080
Email: [email protected]
Kathryn Chandler
U.S. Department of Education
National Center for Education Statistics
1990 K Street, NW
Room 9017
Washington, DC 20006
Phone: (202) 502-7486
Email: [email protected]
Tami Benham Deal
Associate Professor and Chair, Health Education Assessment Project
University of Wyoming
1000 East University
Corbett Hall – Department 2196
Laramie, WY 82071
Phone: (307) 766-4284
Email: [email protected]
Byron Dougherty
Project Director, Michigan Profile for Healthy Youth
Michigan Department of Education
John A. Hannah Bldg
608 West Allegan Street
P.O. Box 30008
Lansing, MI 30008
Phone: (517) 241-2293
Email: [email protected]
Joe Gfroerer
See contact information above
Bernard Greene
U.S. Department of Education
National Center for Education Statistics
1990 K Street, NW
Room 9017
Washington, DC 20006
Phone: (202) 502-7348
Email: [email protected]
Joseph Hawkins
Project Director, YRBS Technical Assistance Project
Westat
1650 Research Boulevard
TA 2130
Rockville, MD 20850
Phone: (301) 610-5591
Email: [email protected]
Nancy Hudson
Project Director, Health Education Assessment Project
Counsel of Chief State School Officers
1 Massachusetts Avenue, NW
Suite 700
Washington, DC 20001
Phone: (202) 336-7008
Email: [email protected]
Sean McCabe, PhD
Associate Professor
University of Michigan
Substance Abuse Research Center
2025 Traverwood Drive
Suite C
Ann Arbor, MI 48105-2194
Phone: (734) 998-6510
Email: [email protected]
Moira O’Brien, PhD
Former and Acting Project Officer, Monitoring the Future
National Institute on Drug Abuse
NSC – Neuro Science Center, 5153
6001 Executive Boulevard
MS 9589
Rockville, MD 20852
Phone: (301) 402 1881
Email: [email protected]
Patrick O’Malley, PhD
See contact information above
Peter Reed
Associate Director for Professional Development and Assessment
National Association of Secondary School Principals
1904 Association Drive
Reston, VA 20191-1537
Phone: (703) 860-7295
Email: [email protected]
A.9 EXPLANATION OF ANY PAYMENT OR GIFT TO RESPONDENTS
We propose that principals selected to participate in the principal data collection be offered a $50 book store gift certificate. Principals are overwhelmed with requests for information from diverse sources. In offering such an incentive, we believe we will be doing just enough to gain the attention of principals for long enough to focus on the questionnaire and provide valid responses. Without the incentive, we believe response rates would fall far short of acceptable levels.
Schools selected to have their students participate in the student data collection will be given educational materials and $500 in appreciation for their participation and cooperation. OMB first suggested that CDC offer such incentives in 1999. CDC first adopted a financial incentive for school-based data collections in the 2001 National YRBS (OMB No.: 0920-0493; expiration 11/03) to allow the survey to continue to compete effectively with other large-scale, school-based data collections. Increasingly in recent years, school-based data collections, most of which do not fall under OMB scrutiny, have used financial leverage to secure school cooperation with student surveys. On the 2001, 2003, 2005, and 2007 National YRBS (OMB No.: 0920-0493; expiration 11/07), these incentives have helped maintain or slightly increase school participation rates despite increasing demands on schools that make it difficult to obtain approval for student surveys. CDC also gave schools participating in the 2002 and 2004 methodological studies (OMB No.: 0920-0534, expiration 12/02 and OMB No.: 0920-0611, expiration 12/04, respectively) $500 in appreciation for their participation.
A.10 ASSURANCE
OF CONFIDENTIALITY PROVIDED TO RESPONDENTS
This data collection has received IRB approval from the CDC Human Research Protection Office (protocol #5177, expiration: 6/14/08). The 2008 Methodological Study IRB Approval Letter is in Appendix L.
In review of this application, it has been determined that the Privacy Act is not applicable. Although identifiable information (name and address) will be collected to allow follow-up of non-respondents for the principal data collection, the Privacy Act is not applicable because the participants will be reporting on publicly available information (e.g., number of computers in the school) and responding only on the basis of their role in the school. The data collected in the student data collection are not identifiable. Even though teachers will be required to enter student names on a Data Collection Checklist (Appendix J) to monitor parent permission form returns and make sure that questionnaires are completed only by students for whom permission has been obtained, the Data Collection Checklist is destroyed after the questionnaire has been administered. The Data Collection Checklist is not forwarded to the data collection contractor, Macro Internation, Inc., or to CDC. At no point in time is there any way to connect student’s names to their data.
For both the principal and student data collections, participation is voluntary and respondents will be assured that there is no penalty if they decide not to respond, either to the information collection as a whole or to any particular question. All principals will be informed that the data will be closely safeguarded, that no institutional or individual identifiers will be used in study reports, and that CDC plans to report data only in aggregate form. This information is contained in the principal data collection questionnaire consent form (Appendix F2). All students and their parents will be informed that anonymity will be maintained throughout the student data collection, that all data will be safeguarded closely, and that no institutional or individual identifiers will be used in study reports. This notification is included in the “Student Health Survey” parental permission form distribution script (Appendix G1), the “Student Health Survey” parental permission form (Appendices G2 & G3), and in the instructions on the front page of the “Student Health Survey” (Appendices E1 or E2). All data collectors will be professionally trained to administer the “Student Health Survey.” When introducing the questionnaire, data collectors will remind students that their responses are anonymous (Questionnaire Administration Guides, Appendix G6).
Several actions will be taken to help in administering the “Student Health Survey.” For the classroom-based and computer lab-based conditions, both paper-and-pencil and web-based questionnaires will be administered in group settings, with adequate space between respondents. No personal identifiers will appear on any questionnaire. Students selected to complete the paper-and-pencil questionnaire will submit the completed questionnaire in a sealed envelope, which will be deposited directly into a “ballot box.” After administration of the questionnaire to a class section, all questionnaires for that class will be removed from the box, deposited in an envelope, and labeled with a school and classroom identification number. Questionnaires are removed from the labeled envelope once they are received from the field by the contractor office.
Students completing the web-based questionnaire will be afforded the same level of anonymity as the students completing the paper-and-pencil questionnaire. Students in the three web-based conditions will be assigned unique identifying numbers for accessing the questionnaire. The number will embed the name of the school and the condition to which the student was assigned; however, no record will be retained of the name of the student assigned to a given unique identifying number. If a student has to break-off during the process of responding to the questionnaire, they can re-enter the questionnaire using the same unique identifying number. Students who lose their unique identifying number may request a newly assigned number.
In condition 4, students are assigned to complete a web-based questionnaire on their own, using a computer located at school, at home, or in the community, at the place and time of their choosing during a two-week period. Students in condition 4 will have the opportunity to complete the questionnaire at a time and place that offers them the most privacy. One of the purposes of the study is to examine how variations in the perception and reality of privacy affect self-reporting of risk behaviors. In this particular condition, the level of privacy for questionnaire completion is heavily under a student’s control because a student selects the time and place for completing the questionnaire. In the two conditions involving web-based administration in computer labs, we anticipate variation in the physical layout of the room, and thus a wide range of accommodations to protect student privacy. This variation is not under our control and will contribute to variability in both students’ real and perceived privacy.
The data collection contractor has several security procedures in place to safeguard data. The data will be stored in locked files, accessible only to staff directly involved in the project, retained for three years after completion of the data collection, and then destroyed. No identifying information will be retained in the data record for the principal data collection. Each principal will be assigned a unique identifying number. The connection between the principal’s unique identifying number and their name and address will be retained only long enough to permit follow-up with principals who have neither refused nor completed a questionnaire. Once a response is received either electronically or on paper, the connection between the principal’s identity and the unique identifying number will be destroyed. For the student data collection, the connection between the unique identifying number and the school name will be retained only long enough to complete data collection. Once data collection is complete, this connection will be destroyed. In addition, all electronic data for the principal and student data collections will be stored on secured servers and will be accessible only to staff directly involved in the project. All contractor staff involved with the project will be required to sign Data Collector Confidentiality Agreement (Appendix G7), which is a statement of personal commitment to guard the confidentiality of data.
A.11 JUSTIFICATION FOR SENSITIVE QUESTIONS
None of the questions on the principal data collection questionnaire – “Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys” (Appendix D) – are regarded as sensitive. Informed consent of participants will be obtained (Consent Form, Appendix F2).
On the “Student Health Survey” (Appendices E1 or E2), sexual intercourse, alcohol and other drug use, weapon carrying, suicidal ideation and attempts, and weight loss practices all may be considered sensitive topics. In fact, depending on the student and the setting in which questions are asked, nearly any health-risk behavior, including tobacco use and physical activity, could be considered sensitive. However, the sensitive questions are necessary to the purpose of risk factor surveillance. The behaviors covered in the questionnaire are the major behaviors known to cause mortality and morbidity. During the past 20 years, one of the primary responsibilities of CDC has been to monitor for the nation priority health risk behaviors among youth. To monitor such behaviors, CDC must ask youth about their participation in them. Students are told in the instructions to the “Student Health Survey” (Appendices E1 or E2) that “This survey is about health behavior. It has been developed so you can tell us what you do that may affect your health. It will also ask you questions about your experience taking this survey. The information you give will be used to develop better health education for young people like yourself.”
The questions asked in the 2008 methodological study are consistent with questions asked on the National YRBS (OMB No.: 0920-0493; expiration 11/07) and were developed in close cooperation with representatives from school systems across the Nation and are presented in a straightforward and sensitive manner.
Parental permission to participate in the student data collection will be obtained. Appendix G contains for the parental permission form (Appendices G2 & G3) and parental permission form reminder notice (Appendices G4 & G5) for the “Student Health Survey.”
A.12 ESTIMATES OF ANNUALIZED BURDEN HOURS AND COSTS
A.12.a Estimated Burden Hours
The estimated burden for this information collection is based on more than 15 years of experience with conducting similar studies that follow similar protocol. The planned study involves the use of two data collections, a questionnaire administered to principals – “Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys” (Appendix D) and a questionnaire – “Student Health Survey” (Appendices E1 or E2) administered to students. Principal and student questionnaires will be administered using paper-and-pencil and web-based modes.
School principals are respondents for the principal data collection (Appendix D). Respondents for the student data collection include students who receive instructions for and complete the “Student Health Survey” (Appendices E1 or E2), school administrators who provide information in the School Recruitment Script for the “Student Health Survey” (Appendix H), and teachers who complete the Data Collection Checklist for the “Student Health Survey” (Appendix J). More information about the Data Collection Checklist is detailed in section B.2.f. Although the time required for students to respond to the “Student Health Survey” likely will be shorter among the students who complete the “Student Health Survey” with skip patterns (Appendix E2), the response burden hours have been estimated to be equal across conditions since the overall burden for the “Student Health Survey” includes time spent obtaining parental permission, introducing the questionnaire, and reading instructions and this does not vary by study condition. The results of this study will provide important information about the amount of time required for students to complete the questionnaire when skip patterns are used.
The estimated burden hours for the 2008 methodological study are shown in Table A.12.a. The total burden hours for this study are 4,813. Two hundred burden hours are attributable to the principal data collection. The remaining 4,613.3 burden hours are attributable to the “Student Health Survey” and associated support activities for the student data collection.
Table A.12.a. Estimated Annualized Burden
Hours
Type of Respondent |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden Per Response (In Hours) |
Total Burden (In Hours) |
Principals |
“Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys” |
600 |
1 |
20/60 |
200 |
School Administrators |
School Recruitment Script for the “Student Health Survey” |
80 |
1 |
25/60 |
33.3 |
Teachers |
Data Collection Checklist for the “Student Health Survey” |
320 |
1 |
15/60 |
80 |
Students |
“Student Health Survey” |
6,000 |
1 |
45/60 |
4,500 |
Total Burden |
|
|
|
|
4,813.3 |
A.12.b Estimated Burden Costs
For the planned study, there are no direct costs to the respondents themselves or to participating schools. However, the cost for principals, school administrators, teachers, and students can be calculated in terms of their time in responding as seen in Table A.12.a. Table A.12.b illustrates the calculation of burden costs. In each category, the estimated respondent burden hours have been multiplied by an estimated average hourly salary for persons in that category. Principal and teacher hourly wages were estimated using Education Research Service data Salaries and Wages Paid Professional and Support Personnel in Public Schools 2005-06 published in Education Week. Based on previous experience working with schools in conducting student surveys, it is likely that the school administrator who serves as the initial contact with the school for the student data collection will be the school principal who will then refer the activity to an assistant principal or other administrator. Therefore, for the purposes of estimating cost, the principal hourly wage has been used. The estimated burden costs in terms of the value of time students spend in responding are based on a minimum wage for students aged less than 20 years of $5.85/hour. The total respondent burden costs for the study are $38,213.
Table A.12.b. Estimated Cost
Type of Respondent |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden Per Response (In Hours) |
Hourly Wage Rate |
Total Respondent Costs |
Principals |
“Principal Survey of the Feasibility and Acceptability of Web-based Student Assessments and Surveys” |
600 |
1 |
20/60 |
$40.63 |
$8,126 |
School Administrators |
School Recruitment Script for the “Student Health Survey” |
80 |
1 |
25/60 |
$40.63 |
$1,354 |
Teachers |
Data Collection Checklist for the “Student Health Survey” |
320 |
1 |
15/60 |
$30.10 |
$2,408 |
Students |
“Student Health Survey” |
6,000 |
1 |
45/60 |
$5.85 |
$26,325 |
Total |
|
|
|
|
|
$38,213 |
A.13 ESTIMATES
OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS
There will be no respondent capital and maintenance costs.
A.14 ANNUALIZED COST TO THE GOVERNMENT
The study is funded under Contract No. 200-2006-15929. The total contract award to Macro International Inc. is $1,911,044 over a 26-month period. Thus the annualized contract cost is $882,002. These costs cover the activities in Table A-14 below.
Additional costs will be incurred indirectly by the government in personnel costs of staff involved in oversight of the study and in conducting data analysis. It is estimated that two CDC employees will be involved for approximately 20% and 5% of their time at salaries of $36.46 and $47.00 per hour, respectively. The total direct cost in CDC staff time for the 26-month contract is $32,862 + $10,591 = $43,453. The direct annual costs in CDC staff time will approximate $15,167 + $4,888 = $20,055 annually.
The total cost for the study over a 26-month period, including the contact cost and federal government personnel cost is $1,954,457. The annualized cost to the government for the study will be $882,002 + $20,055 = $902,057.
Table
A-14. Annualized Study Cost
Activity |
Cost |
Contract Costs |
|
Design and plan |
$87,066 |
Programming and developing |
$88,422 |
Recruitment and preparation |
$101,625 |
Printing and distribution |
$21,348 |
Recruiting and training |
$68,352 |
Collection of data |
$394,390 |
Processing, cleaning, weighing and developing data files |
$83,577 |
Dissemination and reporting of results |
$37,222 |
Subtotal |
$882,002 |
Federal Employee Time Cost |
|
20% time for one FTE |
$15,167 |
5% time for one FTE |
$4,888 |
Subtotal |
$20,055 |
Total Contract Cost |
$902,057 |
A.15 EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS
This is a new data collection.
A.16 PLANS FOR TABULATION AND PUBLICATION AND PROJECT TIME SCHEDULE
A.16.a Tabulation Plans
Data
will be tabulated in ways that will address the primary research
purposes outlined in A.2. The initial types of analysis to be
performed will include descriptive statistics, such as frequency
distributions and means. Comparisons will be made using chi-square,
t-tests, and logistic regression. Analyses will be conducted using
software appropriate for preparing estimates based on complex
sampling designs. We plan to use the SAS and SUDAAN analytic
packages for these analyses.
Examples of the table shells that will be completed through analysis of the data are in Appendix M.
A.16.b Publication Plans
Two major publications are planned as a result of this data collection:
Summary of results from the principal questionnaire assessing the feasibility of web-based data collection.
Summary of results from the student questionnaire assessing the effect of administration condition on risk behavior prevalence estimates, participation rates, and data completeness.
The publications will be distributed to federal agencies and state and local health and education agencies that are interested in the feasibility of web-based data collection in the school setting.
The Journal of Adolescent Health and Public Opinion Quarterly journals have published articles on our previous methodological study results and are expected to serve as vehicles for distribution of the results from this study as well.
A.16.c Time Schedule for the Project
The following represents our proposed schedule of activities for the 2008 methodological study, in terms of months after receipt of OMB clearance. In conducting data collections with schools, it is necessary to plan activities to coincide with school schedules. For the 2008 methodological study, data collection must be initiated early in the second semester of the 2007-08 school year; i.e., in January, 2008. The end date for data collection will be determined when schools close for the summer. Therefore, data collection must be completed by the time schools close for the summer. Key project activities will occur during the time periods outlined in Table A.16.c:
Table A.16.c
Activity |
Time Period |
Recruit and schedule schools |
1 to 2 months after OMB clearance |
Print questionnaires/program computers |
1 month after OMB clearance |
Train field data collectors |
2 months after OMB clearance |
Collect data |
3 to 5 months after OMB clearance |
Process data |
5 to 6 months after OMB clearance |
Produce data file |
7 months after OMB clearance |
Analyze data |
8 to 10 months after OMB clearance |
Publish results |
12 to 14 months after OMB clearance |
A.17 REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE
The expiration date of OMB approval of data collection will be displayed.
A.18 EXCEPTIONS TO CERTIFICATION FOR PAPERWORK REDUCTION ACT SUBMISSIONS
No exemptions from the certification statement are being sought.
REFERENCES
Brener, ND, Eaton, DK, Kann, L, Grunbaum, JA; Gross, LA, Kyle, TM, Ross, JG. The association of survey setting and mode with self-reported health risk behaviors among high school students. Public Opinion Quarterly 2006; 70: 354-374.
Brener ND, Kann L, McManus T. A comparison of two survey questions on race and ethnicity among high school students. Public Opinion Quarterly 2003;67:227-236.
Cowan CD, Coverage, Sample Design, and Weighting for Three Surveys. Journal of Drug Issues 2001:31(3).
Education Research Services. (2006). Salaries and Wages Paid Professional and Support Personnel in Public Schools, 2005-06. Education Research Services, Alexandria, VA.
Fendrich M and Johnson TP, Examining Prevalence
Differences in Three National Surveys of Youth: Impact of Consent
Procedures, Mode, and Editing Rules. Journal
of Drug Issues 2001:31(3).
Fowler FJ, Learning From Experience: Estimating Teen Use of Alcohol, Cigarettes and Marijuana from Three Survey Protocols. Journal of Drug Issues 2001:31(3).
Harrison LD, Understanding the Differences in Youth Drug Prevalence Rates Produced by the MTF, NHSDA, and YRBS Studies. Journal of Drug Issues 2001:31(3).
Sudman S. Examining Substance Abuse Data Collection Methodologies. Journal of Drug Issues 2001:31(3).
| File Type | application/msword |
| File Title | OMB SUPPORTING STATEMENT FOR THE 2004 METHODOLOGICAL STUDY OF THE YOUTH RISK BEHAVIOR SURVEY CONTRACT NUMBER 2001-N |
| Author | jross |
| Last Modified By | arp5 |
| File Modified | 2007-09-17 |
| File Created | 2007-09-17 |