Ssb 8-12-20

OMB SUPPORTING STATEMENT: Part B

School Health Profiles Test-Retest Reliability Study

August 12, 2020

Submitted by:

Division of Adolescent and School Health

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention

Centers for Disease Control and Prevention

Department of Health and Human Services

Project Officer:

Sherry Everett Jones, PhD, MPH, JD

School-Based Surveillance Branch

Division of Adolescent and School Health

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention

Centers for Disease Control and Prevention

1600 Clifton Road MS US8-1

Atlanta, GA 30329-4027

Phone: 404-718-8288

Fax: 404-718-8010

E-mail: [email protected]

Contents

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1

B.1. RESPONDENT UNIVERSE AND SAMPLING METHODS 1

B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION 1

B.3 METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH
NO RESPONSE 2

B.4 TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN 3

B.5 INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS COLLECTING AND/OR ANALYZING DATA 3

List of Attachments

Attachment A – Authorizing Legislation

Attachment B-1 – 60-Day Federal Register Notice

Attachment B-2 – 60-Day Federal Register Notice Comments

Attachment C – School Principal Questionnaire

Attachment D – Lead Health Education Teacher Questionnaire

Attachment E – Time 2 Supplemental Questions

Attachment F – Advance Superintendent Letter

Attachment G – Advance Principal and Lead Health Education Teacher Letters

Attachment H – Time1 Initial Mailing Letters – Paper

Attachment I – Time 1 Initial Mailing Letters – Web-based

Attachment J – Time 1 Reminder Postcards – Paper

Attachment K – Time 1 Reminder Postcards – Web-based

Attachment L – Time 1 Follow-up and Final Mailings – Paper

Attachment M – Time 1 Follow-up and Final Mailings – Web-based

Attachment N – Telephone Prompt Script

Attachment O – Time 2 Initial Mailing Letters – Paper

Attachment P – Time 2 Initial Mailing Letters – Web-based

Attachment Q – Time 2 Reminder Postcards – Paper

Attachment R – Time 2 Reminder Postcards and Emails – Web-based

Attachment S – Time 2 Follow-up and Final Mailings – Paper

Attachment T – Time 2 Follow-up and Final Mailings to Principals – Web-based

Attachment U – IRB Approval Letter

Attachment V – Privacy Impact Assessment Form

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. RESPONDENT UNIVERSE AND SAMPLING METHODS

The respondents for the School Health Profiles Test-Retest Reliability Study will include principals and lead health education teachers in public secondary schools containing at least one of grades 6 through 12. Systematic equal probability sampling with a random start will be used to select schools for the survey. A sample of 300 regular public secondary schools in the U.S. containing at least one of grades 6 through 12 will be selected in no more than 40 districts.

The school sample will be selected from the most current Common Core of Data (CCD) Public School Universe File maintained by the National Center for Education Statistics (NCES). Based on school type description and code, charter school status, and grade span, schools will be screened, and only regular public schools containing at least one of grades 6-12 will be retained for sampling.

The number of entities in this universe is approximately 60,000 schools. The number of schools in the sample will be 300. This sample is expected to yield at least 200 participating principals and 200 participating lead health education teachers who completed both a Time 1 and Time 2 questionnaire. Although the test-retest reliability study will use a convenience sample, the sample will include schools from different regions of the United States, from urban, suburban, and rural areas, of varied enrollment sizes, and of varied socio-economic status.]

Systematic equal probability sampling with a random start will be used to select schools for the survey. The School Health Profiles Test-Retest Reliability Study is not designed to generalize the survey results to the target school population. Further, oversampling of subgroups is not required. Likewise, a complex sample design, such as stratified sampling with differential selection probabilities is not needed. An equal probability sample of schools suffices for this study.

To select a diverse sample, the sampling frame will be sorted by the following variables prior to sampling: census region (created from state), type of locale (urban, suburban/town, or rural), grade level (created from grade span with three levels: middle school, junior/senior high school, high school), enrollment in grades 6-12, and two measures of socio-economic status, percent of students eligible for free and reduced-price lunches and median household income.

B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION

For the study, a sample of 300 regular public secondary schools in the U.S. containing at least one of grades 6 through 12 will be selected in no more than 40 districts. This sample is expected to yield at least 200 participating principals and 200 participating lead health education teachers who completed both a Time 1 and Time 2 questionnaire. Any special requirements that school districts have for approval of research in schools within their district will be met before schools are contacted. In addition, the superintendents of districts that do not require special research applications will be sent a letter informing them about the survey (see Attachment F).

As stated previously, schools will be screened based on school type description and code, charter school status, and grade span, and only regular public schools containing at least one of grades 6-12 will be retained for sampling. To reduce the burden on schools, schools that were selected for the 2020 Profiles regular state and large urban school district samples will be excluded. There will also be a “reserve” sample of 200 schools which will be used if the original sample is not yielding 200 participants. Should the reserve sample be needed, the refusing school and its replacement will be matched by region, type of locale, enrollment size, and socio-economic status.

Special contact districts are those known to require completion of a research application before they will allow schools under their jurisdiction to participate in a study. Any special requirements that districts have for approval of surveys will be met before schools in those districts are contacted. We estimate needing to work with approximately 4 special contact districts. The materials sent to special contact districts will be tailored to meet the specific requirements of each district. The respondent burden for special contact districts is estimated to be approximately 2 hours for IRB review by one staff member per district, and 60 minutes per member for district IRB panel review, assuming each panel would on average be composed of six panel members.

Data collection will occur between January and June of 2021 after completion of the school clearance process. At each selected school, the principal and lead health education teacher will be asked to complete a Time 1 survey and, approximately two weeks later, an identical Time 2 survey (with the 5 additional questions included in Attachment E). The data collection goal will be to achieve participation from 200 schools completing both the principal and lead health education teacher survey components. The Time 1 data collection process will begin in January 2021 coinciding with the beginning of the 2020-2021 spring semester for most schools. Schools will be randomly assigned to either a web-based or paper data collection mode.

All schools will first receive an advance letter sent to the principal and lead health education teacher (see Attachment G) to inform them about the study and who has been selected. The advance letter mailing will be followed about three days later by the initial mailing (see Attachment H) including a copy of the Profiles paper surveys for schools assigned to the paper mode. Schools assigned to the web-based mode will receive a letter including login information and instructions for completing the survey online (see Attachment I). A reminder postcard will be mailed one week after the initial mailing, reminding them to complete the survey, or thanking them if they have already completed and returned their survey form (see Attachments J and K).

Two weeks after the initial mailing, all non-respondents will receive a follow-up mailing (see Attachment L) including a replacement paper-and-pencil version of the survey. To maximize response, the nonresponse follow-up contacts for the web group will include a paper survey in addition to the same instructions for completing the survey online (see Attachment M). All non-respondents after this mailing will receive a final nonresponse mailing using priority mail that includes a replacement paper survey two weeks after the previous mailing. During data collection, the level of participation will be evaluated to see if additional follow-up methods are necessary. If participation is projected to fall below the targeted 200 completed surveys (Time 1 and Time 2), telephone follow-up will be conducted to determine if the targeted respondent (principal or teacher) received the survey request and to encourage them to complete the questionnaire if received (see Attachment N). Experienced telephone interviewers will be trained to conduct the nonresponse follow up and will be monitored by supervisory personnel during all interviewing hours.

The Time 2 data collection will follow the same general contact procedures. The time between the two data collections will target a two-week interval. Because not all respondents will complete the Time 1 survey within the same period, Time 2 surveys will require preparing and mailing Time 2 surveys as Time 1 surveys are completed (or received). Time 1 and Time 2 data will be linked using unique IDs assigned to each school and each principal and lead health education teacher within each school. (See Attachments O, P, Q, R, S and T for comparable Time 2 materials).

The primary objective of this study is to examine the reliability of the School Health Profiles questionnaires. Using Time 1 and Time 2 data, reliability will be estimated through Cohen’s kappa. Cohen’s kappa provides a measure of agreement that corrects for chance agreement, and is regularly used in reliability studies.

No problems requiring specialized sampling procedures are foreseen. A reserve sample will be selected in case there is a challenge in meeting the targeted number of completed questionnaires.

B.3 METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NO RESPONSE

A number of public school districts have structured procedures that researchers must follow to obtain permission to contact their schools or teachers for a study. Once the school sample is selected, project staff will identify the relevant special contact districts and begin preparing research application materials to send to them. Staff will communicate with each district on a regular basis to check that the application is being processed and to respond to any additional requirements. The survey manager and project director will monitor the progress of this activity weekly and add more staff to this task if needed.

In addition, the superintendent of each district with sampled schools will be sent a letter informing them about the survey (see Attachment F). This letter will be sent to all districts with sampled schools except for the special contact districts, because those are already being notified through the research application process per their specific requirements.

Data collection for both Time 1 and Time 2 will follow a series of contacts to maximize response. All schools will first receive an advance letter to the principal and lead health education teacher to inform them about the study and who has been selected (see Attachment G).

Time 1 Paper Surveys: For schools assigned to the paper mode, the advance letter mailing will be followed about three days later by the initial mailing (see Attachment H) including a paper copy of the Profiles School Principal Questionnaire (see Attachment C) and a paper copy of the Lead Health Education Teacher Questionnaire (see Attachment D). A reminder postcard will be mailed one week after the initial mailing, reminding respondents to complete the survey, or thanking them if they have already completed and returned their questionnaire (see Attachment J). Two weeks later, all nonrespondents will receive a follow-up mailing (see Attachment L). This mailing will include a replacement paper-and-pencil version of the survey. All nonrespondents after this mailing will receive a final nonresponse mailing using priority mail that includes a replacement paper survey two weeks after the previous mailing.

During the mail data collection, the level of participation will be evaluated to see if additional follow-up methods are necessary. If participation is projected to fall below the targeted 200 completed surveys (Time 1 and Time 2), telephone follow-up will be conducted to determine if the targeted respondent (principal or teacher) received the survey request and to encourage them to complete if received (see Attachment L). Experienced telephone interviewers will be trained to conduct nonresponse follow up and will be monitored by supervisory personnel during all interviewing hours. If email addresses are available, email contacts will also be made to alert the sampled educators to the survey mailing.

Time 1 Online Surveys: Schools assigned to web mode will follow the same data collection series of contacts. However, the initial mailing will only include instructions for accessing the web-based survey, and a unique access code assigned to them (see Attachment I). The thank you/reminder postcard will include the same information (see Attachment K). To maximize response, the nonresponse follow-up contacts for the web group will include a paper survey in addition to the same instructions for completing the survey online (see Attachment M). The practice of following survey invitation pushes to a web mode with paper modes is common in sequential mixed-mode survey designs for maximizing response. The mode of survey completion will be captured for analysis. In the case where both modes are completed, the one received first will be used as the mode of record – only data from that mode will be retained.

For schools assigned to the web mode for data collection that have an email address available for the selected school representatives, an email contact will be sent to coincide with the initial mailing and nonresponse follow-up contacts (excluding the final telephone follow-up).

Time 2: The Time 2 data collection will follow the same general contact procedures. Respondents in Time 2 will be matched to the mode they completed in Time 1. Instead of starting with the survey mode assigned (web or paper), the mode completed by the principal or lead health education teacher in Time 1 will be used. This means that some schools assigned to the web mode will instead receive only paper-and-pencil surveys in the Time 2 data collection. (See Attachments O P, Q, R, S and T for comparable Time 2 materials.)

The time between the two data collections will target a two-week interval. Because not all respondents will complete the Time 1 survey within the same period, Time 2 surveys will require preparing and mailing Time 2 surveys as Time 1 surveys are completed (or received). The follow-up contacts will be similar to Time 1, with modification to the final three nonresponse contacts. After the first nonresponse follow-up mailing, remaining nonrespondents will be contacted by telephone to encourage them to complete the Time 2 web-based survey (if received) or to mail the paper survey back (which all nonrespondents will have received). The final nonresponse follow-up will do the same, but also offer to collect data over the telephone. The Time 2 series of contacts will not include an advance letter given that respondents will have been informed about the data collection.

B.4 TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN

No tests of procedures and methods will be conducted specifically for the School Health Profiles Test-Retest Reliability Study.

B.5 INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS COLLECTING AND/OR ANALYZING DATA

The School Health Profiles Test-Retest Reliability Study will be conducted by the Division of Adolescent and School Health, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention. The Centers for Disease Control and Prevention contracted Westat to administer the data collection. Barbara Queen, Senior Study Director; Doug Williams, Senior Statistician; Ting Yan, Associate Director; Molly Hershey-Arista; Senior Study Director and Karen Gray-Adams, Senior Study Director, all at Westat, designed the data collection. Annie Lo, Senior Statistician at Westat, was consulted about the statistical aspects of the sample. Sherry Everett Jones, a health scientist at the Centers for Disease Control and Prevention, will analyze the data for the agency.