Ssb-0557

Supporting Statement B for

Health Behaviors in School-Age Children – NICHD

August 15, 2008

Project Officer:

Dr. Ronald J. Iannotti

Prevention Research Branch

Division of Epidemiology, Statistics, and Prevention Research

National Institute of Child Health and Human Development

Building 6100, 7B05

9000 Rockville Pike

Bethesda, Maryland, 20892-7510

Telephone: (301) 435-6951

Fax: (301) 402-2084

E-mail: [email protected]

Table of Contents

2. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods…………………………………………4

B.2. Procedures for the Collection of Information …………………………………………8

B.3. Methods to Maximize Response Rates and Deal with Non-Response ………………..13

B.4. Test of Procedures or Methods to be Undertaken ……………………………………..16

B.5. Individuals Consulted on Statistical Aspect and Individuals Collecting and/or

Analyzing Data …………………………………………………………………………17

List of Attachments

Attachment 1: Data Collection Instruments – Student Surveys

Attachment 2: Data Collection Instruments – Administrator Survey

Attachment 5: Instructions to Respondents

Attachment 6: Recruitment Materials

B. Collections of Information Employing Statistical Methods

B1. Respondent Universe and Sampling Methods

The target population for the HBSC U.S. Survey is all students who are enrolled in grades 5 to 10 in public and private schools in the 50 states and the District of Columbia during the school year 2009/2010. The objective in the U.S. national survey is to select a nationally representative probability sample of students in each of grades 6 through 10 from public and private schools. An oversample of minority children (African American and Hispanic) is also required from each of the grades 6 through 10. The sample selected for the U.S. national survey is also expected to yield a subsample of a minimum of 1,536 students in the age groups of 11, 13 and 15 years for the international HBSC data set. This sample of students in different age groups should come for all grades. Since there are 11-year old students in grade 5, the target population also includes students from grade 5.

The required number of students in the sample for each grade was determined by first computing the sample size to meet the precision requirements under simple random sampling. To offset the loss in precision because of a complex sampling design, the sample was increased by using an average design effect (averaged over several estimates) based on the data from the 2005/2006 survey. The design for the 2005/2006 survey was similar to the one currently proposed. Therefore, the design effect from this survey reflects the loss in precision because of the design which includes clustering at various stages (number of clusters and average cluster size) and unequal sampling weights. The formation of primary sampling units by aggregating school districts and sampling a certain number of primary sampling units is a standard procedure in national school surveys because it is cost efficient. The number of primary sampling units in the sample is similar to the number used in the previous survey and based on cost considerations. The total number of classrooms required in the sample was based on the average class size as the number of students in the sample is fixed. The measure of size for the selection of the primary sampling units and schools is the total number of students in grades 5 to 10. In each selected PSU, schools teaching students in grades 5 and 6, grades 7 and 8, and grades 9 and 10 will be listed. A sample of 3 or 4 schools will be selected from each PSU with a minimum of one school from each of the three groups of grades which are grades 5 and 6, grades 7 and 8 and grades 9 and 10. In schools teaching grade 5 and 6 students, a sample of students from grades 5 and 6 will be selected. The sample of grade 5 students is primarily to screen students who are 11 years old and the data from these students will meet the requirements of the HBSC international sample. The sample of 5^th-grade students is not part of the U.S. national sample. Only samples of students in grades 6 to 10 are required for the national sample.

We do not plan on a non-probability replacement of non-responding schools. A larger than required sample of schools will be selected from each selected PSU. This will be divided into replicates at random. Replacements from the replicates will be done by random selection. Non-response adjustment at various levels will based on the ratio of the sum of the base sampling weighs of the selected sample and the base sampling weights of the responding units. For example, if an individual school district or a group of school districts considered as a PSU do not respond, then the weights of the other responding PSUs in the Census division will be adjusted to account for nonresponse. The adjustment factor will be obtained by taking the ratio of the sum of the sampling weights of all selected PSUs to the sum of the sampling weights of responding school districts. The weight adjustments will be done by forming appropriate weighting classes to justify the missing at random assumption and to reduce bias in the estimates because of non-response. The sampling weights for each responding school and student will be constructed by combining the base sampling weights (inverse of the probability of selection) and non-response adjustments at various stages of selection. We will be looking at item non-response and consider appropriate imputation methods as was done in the 2005/2006 survey.

For the selection of the sample of students from public schools, a three-stage cluster sampling design will be used. The first-stage sampling frame will contain primary sampling units (PSUs) formed by grouping individual school districts within a county or in adjacent counties. For sampling PSUs, the population of PSUs will be stratified by Census divisions. A sample of PSUs will be selected within each Census division with probability proportional to total enrollment in grades 5 through 10 in each PSU in the Census division. A small number of very large PSUs will be included in the sample with certainty. At the second stage, a sample of schools will be selected from each selected PSU with probability proportional to the total enrollment in grades 5 through 10 from the list of schools belonging to the school districts in the PSU. At the third stage of sampling, a simple random sample of classes will be selected from grades 5 though 10 from selected schools. We plan to retain around 22 PSUs and 90 schools selected in the 2005/2006 survey to have an overlap of the sample and also to increase response rates and reduce data collections costs. All students in the selected classes will be included in the sample. For the selection of the minority oversample, an independent sample of PSUs will be selected with probability proportional to the total minority enrollment in grades 6 through 10. At the second stage, schools will be selected from each selected PSU with probability proportional to the total minority enrollment. At the third stage, classes will be selected from a list of classes from selected schools.

The sampling design for the selection of students both in the main sample and the oversample will closely follow the design adopted for the 2005/2006 U.S. national survey. The sampling frame for the selection of PSUs, schools and students will be constructed using the most current data files provided by the Quality Education Data (QED), Inc. The information available from the Common Core of Data (CCD) maintained by the National Center for Education Statistics will also be examined to supplement the information from the QED. The total sample that will be selected for the survey excluding the oversample is 14,672 students. This sample includes students selected both from private and public schools.  13,472 students will be selected from public schools and 1,200 students will be selected for private schools giving a total of 14,672 students.  In addition to this, an oversample of minority students will be selected. For identifying minority students a sample of 1,160 students will be selected out of which 1,000 will come from public schools and 160 from private schools.  Therefore, the overall total sample selected for the survey is 14,672 + 1,160 = 15,832 students (Table 1).

Table 1: Sample Size by Grade (Number to be Selected)

Grade	Main Sample			Oversample			Overall Total
	Public	Private	Total	Public	Private	Total
Grade 5	2,200	200	2,400	-	-	-	2,400
Grade 6	2,200	200	2,400	200	32	232	2,632
Grade 7	2,218	200	2,418	200	32	232	2,650
Grade 8	2,218	200	2,418	200	32	232	2,650
Grade 9	2,318	200	2,518	200	32	232	2,750
Grade 10	2,318	200	2,518	200	32	232	2,750
Total	13,472	1,200	14,672	1,000	160	1,160	15,832

The sampling frame for sampling private schools will also come from the QED. The private schools in the sampling frame will be assigned to the PSUs created for sampling public schools depending on the location of the school. Only those private schools which are in the selected PSUs will be considered for sampling. Therefore, selected private schools will be within the same PSUs selected for sampling public schools. This strategy will minimize data collection costs.

Table 2: Distribution of the Population and Expected Sample by Strata

(Not Including Oversample)

Census Division	Number in the Population			Number in the Sample
	School Districts	Schools	Students (grades 5 thru 10)	Students	Schools	Classes
New England	1,301	4,880	1,042,138	665	18	27
Middle Atlantic	2,063	10,396	2,848,183	1,816	48	73
East North Central	3,461	16,783	3,582,289	2,284	60	91
West North Central	2,533	10,535	1,504,699	960	25	38
South Atlantic	863	14,557	4,256,781	2,715	71	109
East South Central	608	5,783	1,338,804	854	22	34
West South Central	2,133	13,295	2,852,502	1,819	48	73
Mountain	1,561	8,138	1,660,017	1,059	28	42
Pacific	1,620	14,197	3,920,414	2,500	66	100
Total	16,143	98,564	23,005,827	14,672	386	587

Source for Population Numbers: NCES – Common Core of Data – 2005-2006.

To end up with a nationally representative sample of completed surveys, the number of students in the targeted classroom given in Tables 1 and 2 are based on the assumption of an overall 80% student response rate to the survey. In the 2005/2006 survey, the school response rate was 53% and the student response rate differed by grade level with the highest response rate of 89.5% for grade 7 and 8 grade students and the lowest response rate of 79.3% for grade 6 students. The overall student response rate was 85%. The combined school and student response rate was 51%.

The low school response rate and, consequently, low combined school and student response rate was attributable to several factors. A major problem with the 2005/2006 survey was the delayed start date. The contract for the 2005/2006 HBSC survey was not awarded until the summer before the 2005/2006 school year. The OMB application was not submitted to NICHD until June and forwarded to OMB in September; OMB approval was obtained in January. As a result, school recruitment and surveying was delayed until well into the school year. Schools were very reluctant to permit surveying of students during the latter half of the academic year when extensive preparation and assessments were taking place for the No Child Left Behind initiative. We plan to address this problem in the 2009/2010 survey by applying for OMB approval in sufficient time to be able to recruit schools before the 2009/2010 school year and to survey students in the fall and early winter of the 2009/2010 academic year. As noted above, we will purposefully sample schools that cooperated in the 2005/2006 survey which should also contribute to a higher school response rate.

The reason for planning to form PSUs consisting of school districts within a county or contiguous counties depending on the size of the school districts was to reduce travel costs for data collection. The school districts in the population will be clustered to form approximately 1,500 PSUs. A sample of 90 PSUs will be selected. As indicated earlier, for the selection of PSUs, the population of PSUs will be stratified by Census divisions. This stratification is primarily to ensure geographic representation to the sample of students. The total sample of PSUs will be allocated to each stratum (Census division) in proportion to the total enrollment in grades 6 to 10 in each PSU. Probability proportional to size systematic sampling will be used to select PSUs within Census division after sorting the PSUs by zip codes.

The sample from each grade will be large enough to provide estimates of population percentages with a margin of error of plus or minus 3 percentage points at 95% confidence level assuming a design effect of 1.7, based on the results of the 2005/2006 survey. It is planned to select a sample of 2,400 students from each grade to meet the requirements of both the national and international HBSC surveys. With an expected overall response rate 80% at the student level, this sample is expected to give 1,920 responses. Assuming a design effect of 1.7, this sample size will provide estimates with a margin of error of plus or minus 3 percentage points. As indicated earlier, the sample of 14, 672 students will also yield a sample of 5,000 across the three age groups.

The estimates for each age group based on a sample of 1,600 students will also meet the requirement that the margin of error be within plus or minus 3 percentage points at 95% confidence level. The oversample of minority students will improve the precision of the subpopulation (minority population) estimates.

Estimation Procedure. For producing population-based estimates, each responding student will be assigned a sampling weight. This weight combines a base sampling weight which is the inverse of the probability of selection of the student and an adjustment for non-response at the school level and the student level. The probability of selecting a student is the product of the probability of selecting the school district, the probability of selecting the school within the district and the probability of selecting the class in which the student is present. The inverse of the overall probability gives the base weight. Various selection probabilities will be recorded and used to construct the sampling weight. The base weights will be adjusted for non-response. All student level estimates will be weighted estimates using the student weight. All student level analyses of data will also use student weights.

There are no specialized sampling procedures for the selection of PSUs, schools, or classes.

B.2. Procedures for the Collection of Information

Consistent with international HBSC guidelines, the primary data collection will be completed within a 16-week period, beginning October 1, 2009 and ending by February 1, 2010. Make-up dates will extend the final data collection an additional two weeks beyond the February 1 date. To cover approximately 386 schools within a 16-week data collection period, a team of 45 local data collectors will conduct multiple school visits. This number includes backups in case of illness or poor performance as well as staff to cover make-up visits to survey student absentees, if necessary. This also offers the flexibility to double-staff the schools that are anticipated to be difficult.

Administrator Surveys. School administrators will be asked to complete a brief survey describing the school’s health-related environment, policies, and programs (see Attachment 2). Areas covered include policies regarding physical education, food services, health education and health promotion, violence prevention, safety and other education programs. The survey can be completed in less than 20 minutes using paper and pencil.

Student Surveys. The U.S. student survey can be completed in less than 45 minutes. The student survey includes a core set of questions that are mandatory for all countries participating in the international HBSC data collection. In addition, based on current policy needs and research questions targeted by investigators at NICHD and HRSA, the U.S. survey includes additional questions designated by the international HBSC; these questions permit international comparisons for those countries electing to use these optional questions. There are also U.S.-specific questions that have appeared in previous U.S. surveys and address unique issues related to the health of students in the U.S. (see Attachment 1). Topic areas covered in the survey include:

• Eating habits, weight control, and body image;

• Dental health;

• Physical activity;

• Sedentary Behavior;

• Substance use;

• Injuries and bullying;

• Family structure, environment, and communication;

• Peers;

• School environment;

• Positive health;

• Health status;

• Puberty; and

• Demographics.

The international standard questionnaire for each HBSC survey was devised through cooperative research between members of the HBSC research network representing all member countries. Participating countries are required to include all the mandatory questionnaire items and the questions should conform exactly to the format indicated in the Survey Protocol, as far as the translation process will allow. The international standard questionnaire for the 2009/2010 survey was produced through a process of review and revision of the ‘core’ instrument used in previous surveys. In order to assure that we can examine trends over time, all HBSC mandatory questions will remain the same in the 2009/2010 survey as those in 2001/2002 and 2005/2006. ‘Optional’ items have also appeared in previous HBSC surveys or have been extensively pilot tested in at least three countries and appear in published research prior to being accepted as an ‘optional’ item. Thus, the major difference between the ‘optional’ and ‘mandatory’ items is that not all countries elect to include the ‘optional’ items. International comparisons are still possible, but for only those countries that elect to include particular ‘optional’ items. The U.S.-specific items meet the same criteria as the HBSC items, except their prior use may be limited to a U.S. sample, i.e., they have appeared in previous U.S. HBSC surveys and have been validated in U.S. publications.

Student Survey Data Collection. The following assumptions and accommodations will be made to facilitate planning for the administration of the student surveys:

• PSUs will have an average of 3 schools each;

• In about 75% of cases, two classes will be selected from each school;

• Typically, a 3-day site visit by one data collector will be needed at each school district to complete surveys in six selected classes, (one day per school, to complete surveys in two classes);

• Schools will be staffed by multiple data collectors when such a need is determined.

Student Survey Procedures. The following steps will be taken to facilitate the administration of the student surveys in the schools:

• Survey Coordinators (project staff who will oversee recruiting, site development, and data collection activities) and School Survey Liaisons (teachers or school administrators who will be “hired” at each school to serve as liaisons between the HBSC research team and the school’s staff, students, and parents; School Survey Liaisons (SSLs) will provide access to schools, assist with coordinating the visit, obtain consent, and arrange make-up surveys, but not assist with the administration of student surveys) will determine the best dates for the visit to each school within their region.

• The data collectors will receive two days of training, including consultation on managing classroom behavior.

• The data collectors assigned to a school district will receive the materials necessary to complete all data collection at each school, including the survey instruments, envelopes, student incentives, administrative notes, shipping materials, and local maps.

• Environmental planning will assure an appropriate and private survey environment.

• School personnel will be asked to honor students’ privacy during the test.

• A standardized introduction to the survey will be used, stressing the importance and confidential nature of the survey, and pointing out that names are not attached, only ID numbers. Students will be reminded that their participation is voluntary.

• Project staff will be trained to actively monitor and move about the room, letting students know they are present and available to answer questions, but honoring their privacy. Teachers will be encouraged to remain in the classroom to monitor student behavior but, if present, will be strongly discouraged from looking at student surveys during this process.

• Data collectors will be available to help students and reduce missing data.

• Survey booklets will include no personal identifiers and upon completion will be sealed in envelopes.

• Students will be thanked and given the opportunity to select a small, attractive gift for completing the survey.

• Students who complete the survey early will be given word-finding games and similar puzzles; students who do not have permission to take the survey but are required to remain in the same room will be given math and language-arts packets provided by the school.

Absentee Procedures. It is anticipated that some students will be absent from class on the day of the visit. Frequently, absentees include a greater proportion of high-risk children. Consistent with the procedures used for YRBS, an attempt will be made to capture the data from these absentees in order to increase student response rate and increase the generalizability of the results. However, unlike YRBS, in order to increase the level of confidentiality, school personnel will not administer these surveys. These data will be obtained either by an in-school assessment conducted by the field staff or these students will be asked to complete the survey online, using a computer in a private space in the school. SSLs will have clear directions on how to maintain the confidentiality of the survey for the students, and the online instructions will also emphasize to the students the importance of completing this in a private space. Access to the survey will be password-protected, and the passwords will be given to students in a sealed envelope that will indicate that it should not be opened until starting the survey in privacy. Students will be asked to destroy the password when they have completed the survey. Using these procedures in the 2005/2006 survey, we were able to capture 23% of students who were absent on the day of the classroom survey.

B3. Methods to Maximize Response Rate and Deal with Non-Response

The following procedures, which have been employed with great success in previous national surveys, will be used to recruit schools.

Notification of State Education Administrators. An initial study notification mailing will be sent to State education superintendents prior to contacting each school. The purpose of this initial mailing is to inform the top officials in each participating State about the survey and to inform them of the districts that have been selected for participation. The mailing will include a cover letter, a color brochure introducing the study, and letters of support from key organizations. The cover letter, over the signature of Dwayne Alexander, Director, NICHD and Peter van Dyck, Associate Administrator, HRSA, will briefly introduce the study, explain its purpose and importance, describe the expected use of study findings, and refer readers to the detailed information in the HBSC Survey Fact Sheet. The Fact Sheet will describe the following: objectives and importance of the study; desired study participants and eligibility requirements; data collection activities and schedule; incentives for participation; responsibilities of the data collection staff, schools, principals, and participants; and measures that will be taken to protect respondents’ privacy and confidentiality.

Notification of School District Superintendents. An initial study notification mailing will be sent to district superintendents to provide districts with information about the survey and a list of the schools selected in their district. The mailing will include a personalized cover letter similar to the one developed for state administrators and the HBSC Fact Sheet, describing the survey objectives and requirements. Mailings will also include letters of support from influential professional societies and organizations and a copy of a color brochure designed for parents. To track the sampled districts, schools, principals, and classes, each will be assigned an ID code and entered into a tracking database from which status reports can be generated. Each mailing will have the ID code pre-printed to indicate the recipient. Districts will be provided with a toll-free number and email address to direct any questions or concerns they may have.

Recruitment of Schools. Once school districts have been notified of the study and given a week to ask questions, a mailing will be sent to the principals of the sampled schools. The mailing to principals will contain the same information that was sent to the district superintendents, including a cover letter and the HBSC Survey Fact Sheet describing the study and the role of the principal in its implementation, along with letters of support from influential professional societies and organizations similar to those noted for district mailings and a copy of a color brochure designed for parents. The cover letter to principals will differ in that it will be targeted to the schools (rather than districts) and will note the school district has been notified of their selection. The letter will be personalized and will again include contact information for the Survey Coordinators so that principals can call or e-mail them directly if they have questions. The mailing will inform the principals that Survey Coordinators will contact them to discuss arrangements for HBSC survey administration.

Approximately one week after the estimated delivery of the mailing to school principals, Survey Coordinators will initiate telephone calls to confirm that principals have received the mailing, answer any questions they might have, and obtain permission for HBSC administration. At any time during the recruitment process, if a school official is non-responsive or refuses participation, or expresses strong concerns about confidentiality to the Survey Coordinator, this information will be provided to senior HBSC research staff. This senior staff person will contact such schools directly in an effort to address any concerns and obtain the school’s participation. Weekly recruitment progress reports will be generated.

Recruitment of Students. Procedures for obtaining consent are consistent with those of similar U.S. national studies. Based on the 2005/2006 HBSC Study, it is anticipated that approximately 80% of the schools will permit passive consent and 20% of schools will require active parental permission. In the passive consent situations, parents will be provided with information about the study and will be able to review the survey instrument and can refuse to allow their child to participate. In the active consent cases, parents will be provided a letter from the school principal introducing the survey, a color brochure describing the study, and the consent form requesting permission for their children to participate. Survey Coordinators will work with the school principals to construct, distribute, and collect the letters. The consent forms and letters will be sent home with students and classroom teachers will be instructed to encourage students to return the consent forms as quickly as possible. Returns will be monitored by the SSL. For parents who have not returned signed consent forms within three days, a second consent form and letter will be sent home with the student, reminding the parent of the study and requesting that they return the form. This process will be repeated for parents who have still not returned the form after one week. Students will be informed verbally and in writing that they may skip any or all questions or refuse to participate in the survey, in which case an alternative activity approved by the school administration will be provided.

Incentives. Each participating school will receive $500 and the teachers in each participating classroom will receive a $25 on-line gift certificate to a vendor such as Amazon, Borders Books, or Barnes and Noble, where they can buy books, software, or games for their classrooms. Students will receive a token incentive (novelty pencils or erasers that are appealing to students) for returning signed (agreeing or declining to participate) consent forms (in active consent schools) and for completing the survey.

Non-response. Issues of non-response and methods of dealing with non-response are discussed in section B1.

B4. Tests of Procedures or Methods to be Undertaken

The 2009/2010 survey is being conducted by the same firms that conducted the 2005/2006 survey. No test of procedures is necessary because these same methods were used in the earlier survey. The 2009/2010 survey is essentially identical to the 2005/2006 survey and items appearing in the 2009/2010 survey have been used in previous surveys. In addition, the 2009/2010 survey is shorter than the 2005/2006 survey. Therefore, we do not believe that pilot testing for length (as we did in 2005/2006) will be necessary.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The role of outside consultants collecting the data, performing preliminary analyses, and staffing of this project is discussed in earlier sections of this application (B2, B3).

Other consultants for this study include the research members of 40 HBSC countries who reviewed and recommended all mandatory and optional questions for the 2009/2010study according to their specialty interests, as members of HBSC focus groups. Those HBSC member names have not been included here. The HBSC Scientific Development Group required that all of the mandatory and optional questions be piloted and reviewed externally before the questions could be included in the HBSC protocol. Besides the review of focus group questions, global external review was required under the HBSC protocol for significance of research topics, concepts, clarity of language used, and validity of measures to address those topics. Many of these reviews were completed by e-mail. In addition, the U.S. PI’s obtained external statistical review of four proposed sample designs to meet HBSC and U.S. criteria for valid nationally representative estimates based on requirements for age, adequate sampling response, efficiency, and other issues. Consultations from other agencies and national organizations were also obtained concerning the usefulness and validity of the linked QED and CCD data files and Administrator Survey questions to address program issues and the social community context. The protocol is undergoing a final external review by experts identified by the NICHD Director of Intramural Research. The protocol and surveys are currently under review by the NICHD Institutional Review Board (IRB). Final determinations will be provided to OMB as soon as they are completed and before the final OMB determination.

20