NHES Justification

Volume I:

Request for Clearance for Spanish Cognitive Interviews for the 2010 National Household Education Survey (NHES) Study Draft Questionnaires

1850-0803 v.34

July 29, 2010

Justification

The Random Digit Dial (RDD) National Household Education Survey (NHES) like most RDD surveys, has experienced a rapid decline in response rates. As a result of these declining response rates, the National Center for Education Statistics (NCES) has implemented a multi-stage redesign of the study. The primary goal for a revised NHES is to increase response rates without increasing measurement or coverage error in the study. To achieve this objective, a number of modes including face-to-face interviews and Internet data collection were evaluated for the redesign. Based on this evaluation, it was determined that a mail out – mail back survey with telephone non response follow up design had the greatest potential to achieve the response rate goal within the budget and precision constraints of the study. The NHES is a two stage study. In the first stage, households are screened to determine if they contain eligible members. If eligible members are in the household, within household sampling is performed. Finally, topical module(s) are administered to the selected household members. Initially, there were concerns about the feasibility of conducting this screening operation by mail. A small pilot test of the methodology was undertaken during Fall 2009. The results of this pilot indicate that it is possible to conduct the NHES by mail and obtain a reasonable response rate while maintaining coverage of the target population. A large scale field test is planned to further refine the data collection protocols in 2011. The purpose of this submission is to conduct cognitive interviews to test the Spanish language translations and procedures for the 2011 field test. A 60-day public comment notice for the request for clearance of the 2011 field test was published in the Federal Register on July 8th, 2010 (75FR130).

Cognitive testing has been a feature of the NHES program in past surveys. The objective of the interviewing in 2010 is to identify and correct problems of ambiguity or misunderstanding in the Spanish language materials. The result should be a set of questionnaires that are easier to understand and therefore less burdensome for respondents while also yielding more accurate information. The interviews will focus on three key areas:

Screener to topical transition. Because of the NHES two-stage mailing administration, the interviews will evaluate the transition from the screener to the topical survey. It is important that we evaluate the respondent’s understanding of the transition from the screener to the topical to ensure the screener survey encourages response at the second stage. The screener is also critical to correctly sampling a child for participation in the second stage.

Content. The English questionnaire wording went through extensive cognitive interviewing in 2009 and 2010. The purpose of this round of interviewing is to ensure that the Spanish translation of the materials convey the same meaning as the English forms. The materials and design described in this package have been approved by OMB on March 22, 2010, for the English version (OMB# 1850-0803 v.22).

Design

Cognitive interviews are intensive, one-on-one interviews in which the respondent is asked to “think aloud” as he or she answers survey questions, or to answer a series of questions about the items they just answered. Techniques include asking probing questions, as necessary, to clarify points that are not evident from the think-aloud comments and responding to scenarios. Probes that will be used include,

• probes to verify respondents’ interpretation of the question (e.g. asking for specific examples of activities in which the respondent reports participating),

• probes about the meaning of specific terms or phrases used in the questions, or

• probes for experiences or ideas that the respondent did not think were covered by the question but we would have considered relevant.

Interviews will be conducted by trained Spanish speaking cognitive interviewers. Volume II of this submission includes draft English questionnaires and mailing materials that will be tested during these interviews, as well as preliminary protocols for conducting the interviews. These materials will be translated to Spanish prior to conducting the interviews. It is expected that the translations will evolve during testing.

Screener Variations for testing:

• Bilingual screener

• Spanish only screener

Topical Module Variations for testing:

• Parent-Family Involvement (PFI)

• Early Childhood Program Participation (ECPP)

Contact Strategies:

• Letters

• Cover page

• FAQ’s

• Envelopes

20 Spanish speaking participants will be recruited. The participants will all have at least one NHES eligible child (infant – age 18). Participants will be asked to complete either the bilingual or Spanish screener and one topical questionnaire. Approximately half of the participants will be recruited to complete the Early Childhood Program Participation (ECPP) topical survey and half will complete the Parent Family Involvement in Education (PFI) topical survey. The interviews are expected to last 90 minutes.

Recruiting and Paying Respondents

Participants completing the 1.5 hour interview will receive $60. These are the same amounts that were offered in the 2009 and 2010 NHES cognitive interviews. These amounts are necessary to attract a wide spectrum of income and demographic groups.

Participants will be recruited using multiple sources, including a database of prescreened research volunteers, advertisements in print and online publications, community organizations, and personal contacts. Participants must be native Spanish speakers with limited English ability to be eligible for the study. People who have participated in cognitive studies or focus groups in the past 12 months or are direct employees of the firms conducting the research will be excluded from participating. We anticipate that approximately 240 people will be contacted to produce 20 respondents, and that a screening interview will take about 3 minutes to complete.

Assurance of Confidentiality

Participation is voluntary and written consent will be obtained from respondents or from legal guardians of minor respondents before interviews are conducted. No personally identifiable information will be maintained after the cognitive laboratory analyses are completed. A copy of the respondent consent form is attached at the end of this document.

Estimate of Hour Burden

Table 2. Burden time by response type.

Respondents	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Burden Hours
Screener Recruitment Interview	240	240	0.03	12
Screener and Topical Review	20	20	1.5	30
Total	240	260	-	35

The cognitive interviews are expected to last approximately one and one-half hours. This will result in 30 burden hours. Screening potential participants will require 3 minutes per screening and we anticipate approximately 12 screening interviews per eligible participant. This will result in 12 hours of burden. Thus, the estimated total respondent burden for this research will be: 35 hours.

Estimate of Cost Burden

There is no direct cost to respondents.

Cost to the Federal Government

The total cost for conducting the cognitive interviews is expected to be $35,000.

Project Schedule

The project schedule calls for the research to be started the week of August 15. Interviewing is expected to last until September 14th. After interviewing, instruments will be finalized and submitted as part of the 2011 field test request for clearance.

Assurance of Confidentiality

The American Institutes for Research and XXX are conducting this study for the National Center for Education Statistics (NCES) of the U.S. Department of Education. This study is authorized by law under the Education Sciences Reform Act (ESRA). Your participation is voluntary. Your responses are protected from disclosure by federal statute (P.L. 107-279, Title 1, Part E, Sec. 183). All responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose, unless otherwise compelled by law.

Please sign below to indicate that you have read the assurance of confidentiality.

Signature Date

Printed Name

Researcher’s Signature

Researcher’s Printed Name