National Household Education Survey 2019 (NHES:2019)

Given continued declines in response rates to both the National Household Education Surveys (NHES) and to household surveys more broadly – and the growing challenges associated with conducting cost-efficient, high-quality, representative data collections, NCES will conduct the In-Person Study of NHES:2019 Nonresponding Households during the NHES:2019 administration. This study will primarily address the following research questions:

What are the barriers to response among screener nonrespondents to NHES? For example, do screener nonrespondents remember receiving the survey mailings, and, if so, do they open them? What stops some households from receiving or opening the mailings?
What are the characteristics of nonresponding households? Is there information about nonresponding households that could be used to tailor contact protocols in future collections?
- Is the information available on the address-based sampling frame for nonresponding households accurate to ensure variables used for tailoring materials or predicting data collection protocols are reliable?
- Are there variables related to household survey nonresponse that have not been previously observed or measured?

To address these research questions we will conduct approximately: (a) 100 of 90-minute interviews using an unstructured, qualitative interview protocol and (b) 750 address and neighborhood observations. This document discusses the importance of conducting this study and provides an overview of its methods.

1. Background

The nonresponse problem

Survey nonresponse has been increasing for several decades (Brick and Williams 2013; Czajka and Beyler 2016). NHES has not escaped this trend; between the first mail-based NHES administration in 2012 and the most recent one in 2016, the screener response rate has dropped from 74 percent to 66 percent (McPhee et al. 2015; McPhee et al. 2018). In addition, NHES nonrespondents tend to be younger, less educated, and have lower incomes than respondents; they are also less likely to be White or to be married than are respondents.

Given the global nature of this phenomenon, researchers have devoted considerable attention to understanding the factors that contribute to survey nonresponse. Proposed individual-level explanations for nonresponse include privacy concerns, anti-government sentiment, busyness and fatigue, concerns about response burden, lack of interest in the topic, and low levels of civic engagement or community integration (Abraham et al. 2006; Amaya and Harring 2017; ASA Task Force on Improving the Climate for Surveys 2017; Groves et al. 1992; Groves et al. 2004; Singer 2016). Investigators also have suggested that societal-level changes, such as the increasing number of survey and solicitation requests, declining confidence in public institutions, and growing concerns about security and identity theft are contributing to declining response rates (ASA Task Force on Improving the Climate for Surveys 2017; Czajka and Beyler 2016; Presser and McCulloch 2011).

However, much remains unknown about sample members’ reasons for nonresponse and the best way to counteract those concerns. Recent task force reports focused on survey nonresponse argue there is a need for more research about the current survey climate and the various factors that could improve survey participation (AAPOR Task Force on Survey Refusals 2014; ASA Task Force on Improving the Climate for Surveys 2017). Due to the emphasis on telephone surveys in the latter decades of the 1900s, little research has been carried out since the 1970s about the response decision process associated with mail surveys like NHES (Singer 2016). Questions remain about whether research on nonresponse within mail collections from previous decades holds up in today’s survey environment.

The utility of conducting follow-up studies

Studies that have followed up with survey sample members after the survey request has been made have proven to be a useful tool for understanding the reasons for response (or lack thereof). In-depth follow-up interviews with respondents to an earlier survey found that materials that clearly communicated the topic and benefits of the study were particularly important for gaining response (Harcomb et al. 2011). In another study, semi-structured follow-up interviews with sample members identified altruism, perceived personal benefit, understanding the study, and being committed to its success as primary reasons for responding (Nakash et al. 2008). A focus group of immigrant and ethnic minority nonrespondents to a Danish national health survey discovered that linguistic and educational limitations, alienation resulting from sensitive questions and cultural assumptions, and concerns about anonymity were the primary cause of nonresponse among that group (Ahlmark et al. 2015).

The Bureau of Labor Statistics’ Response Analysis Survey used qualitative telephone interviews to examine nonresponse (refusals) to the American Time Use Survey (ATUS) (O’Neill and Sincavage 2004). The study found that the advance materials were ineffective at communicating the differences between the Current Population Survey, which served as the sampling frame for the ATUS, and the ATUS. It concluded that how respondents perceive the survey sponsor is an important factor in whether to participate and that the use of incentives should be explored to encourage participation.

The American Community Survey (ACS) conducts an in-person computer-assisted personal interview (CAPI) follow-up for a subsample of nonrespondents. Special emphasis is placed on collecting data from many languages (e.g., providing language assistance guides to the interviewers in Chinese and Korean, special training). Language availability and the mandatory nature of the survey keep the response rates high (United States Census Bureau 2014). CAPI interviewers achieve about a 95.5% interview rate. Cases that showed initial resistance or had at least one coded refusal had interview rates of 94.5% and 26.8% respectively (Zelnack et al. 2013). The ACS also uses the Contact History Instrument (CHI) to monitor field performance and collect paradata during in-person interviews. The purpose of the interview is to collect data from nonrespondents.

Despite the research cited here, there are relatively few studies that attempt to explore the reasons for nonresponse in American household surveys at the scale or depth planned for the In-Person Study of NHES:2019 Nonresponding Households, making this an important and unique contribution to the survey literature and a potential resource for researchers facing the same challenges in other national, federal government household surveys.

The challenges associated with using address-based sampling (ABS) frames

The NHES sample is comprised of addresses drawn from an ABS frame. The quality of the information about these addresses on the frame is another factor that could contribute to nonresponse. For example, vacancy or seasonal status indicators available on the frame could be inaccurate and lead to addresses being sampled where there is no one available to respond (AAPOR Task Force on Address-Based Sampling 2016). In addition, if the information available about the individuals living at sampled addresses (such as whether they prefer to speak Spanish) is inaccurate, this creates challenges for designing survey protocols that are targeted at sampled addresses’ response preferences (such as whether to send Spanish-language materials). The NHES frame is not immune to this problem; analysis of NHES:2016 data shows that only about 55% of the screener respondent households flagged on the commercial vendor frame as having children ended up reporting a child on the screener. Assessing the quality of additional sampling frame variables through interview questions and observations will help to assess the extent to which inaccurate frame data may be hindering efforts to gain response to NHES and making recruitment efforts inefficient.

The benefits of collecting interviewer observations

Interviewer observations are a low-burden tool for learning about the context in which sample members receive the survey request and for verifying ABS frame data. Little and Vartivarian (2005) suggest that interviewer observations of variables related to the likelihood to participate in a survey and to the survey’s key estimates can be used to adjust for nonresponse bias. Several large-scale, in-person surveys collect observations using tools such as the Contact History Instrument or Neighborhood Observation Instrument (NOI). Since 2013, National Health Interview Survey (NHIS) interviewers have used the NOI to observe neighborhood characteristics, such as the condition of units or the presence of barriers to access, and to make inferences about sample member characteristics of interest, such as income, employment, and the presence of children (United States Census Bureau 2017). NHIS researchers have found that some, but not all, NOI measures are positively associated with refusal to participate (Walsh et al. 2013). The National Survey of Family Growth (NSFG) also collects interviewer observations about the neighborhood, housing unit, and sampled persons (User’s Guide for Paradata File n.d.). NSFG researchers again find that some, but not all, of the recorded observations are associated with lower propensity to respond to the in-person interview request (Lepkowski et al. 2013). These findings highlight the potential benefit of collecting observations, as well as the need for further research into which neighborhood and address characteristics are related to responsiveness to a mailed survey instead of an in-person survey.

2. The proposed study

The proposed In-Person Study of NHES:2019 Nonresponding Households will use qualitative in-person interviews, as well as address and neighborhood observations to assess whether the existing theories surrounding household survey nonresponse are applicable to NHES. The study is designed to provide additional actionable information about how to combat the growing nonresponse problem in NHES. It is expected that the results of this study will be used to improve the design of NHES, with the goal of increasing the response rate and the representativeness of the respondents. Potential changes could include modifications to the nonresponse follow-up protocol, the type of incentives offered, and the presentation of contact materials.

Screener nonrespondents, rather than topical nonrespondents, are the focus of the study because of the impact that screener nonresponse has on both the screener response rate and each NHES survey’s overall response rate and because NHES screener response rates are lower than topical response rates.^¹ Larger numbers of screener nonrespondents are anticipated to be clustered in metropolitan areas than in more rural locales. Exhibit 1 plots the location of NHES:2016 screener nonrespondents. As expected, screener nonrespondents are widely dispersed, but there is some clustering around large cities.

Exhibit 1. Location of NHES:2016 Screener Nonrespondents

Moreover, in NHES:2016, household in tracts with 40% of more Hispanic persons, high rise apartments, and renters had lower weighted screener response rates, 55.9%, 58.9%, and 56.6% respectively (see Attachment A in this document for screener response rates by additional subgroups) (McPhee et al. 2018).

Analyses of the resulting data are expected to illuminate some of the factors associated with nonresponse and inform revisions to the content and design of recruitment materials to be used in NHES:2022.

Qualitative interview methodology

Qualitative research is particularly useful for trying to understand how people make sense of their lives and experiences. To gain a deeper understanding of why some sample members do not respond to the NHES screener, approximately 100 unstructured, qualitative interviews will be conducted with screener nonrespondents. These interviews are designed to provide rich and nuanced information about survey nonresponse.

The NHES:2019 screener nonrespondents will be stratified to focus on a handful of key household characteristics available on the sampling frame that are likely to drive differences in reasons for nonresponse and could be used for targeting NHES materials and contact protocols in future administrations, such as (1) whether the household is Hispanic or is expected to prefer to respond in Spanish, and (2) in what part of the United States the address is located. Cases will be sampled from three or more sites and the cases sampled for each site will be clustered within an area with approximately a 30-mile radius.² This design will allow for sampling of some rural addresses, while still maintaining a manageable driving distance for the interviewers. Approximately 750 addresses will be sampled from among NHES:2019 early screener nonrespondents for address and neighborhood observations. Of these, 500 will be selected for and invited to take part in the qualitative interview study (to yield approximately 100 completed 90-minute interviews). If nonresponse clustering patterns result in lower than estimated cost of observing and interviewing the target numbers of addresses, more addresses will be selected for the address/neighborhood observations.

The cases included in the nonresponse follow-up study will be selected using frame variables that will be identified based on the observed patterns of nonresponse to early NHES:2019 screener mailings. Because NHES:2019 will be the first NHES administration that uses a web-push mixed-mode design (whereas recent NHES administrations used a paper-only design), we are not certain what patterns of screener nonresponse will be observed and thus must wait to determine the actual selection criteria for the nonresponse follow-up study until NHES:2019 screener data collection has begun and patterns of the early NHES:2019 screener nonresponse become known. However, patterns of screener nonresponse to NHES:2016 give some indication of address characteristics that are likely to drive screener nonresponse in 2019, such as: being in an area with a higher percentage of Black or Hispanic residents, being in an area with a higher poverty rate, being in the south, being in a rented (vs. owned) dwelling, being in a multiunit (vs. a single unit) dwelling, not having a phone number available on the frame, or lacking demographic information about the head of the household (such as gender, age, or marital status).

Once we determine the subgroups of address characteristics that show the highest likelihood of nonresponse to early NHES:2019 screener mailings, we will identify three or more geographic clusters, of approximately 30-mile-radius each, that will include large enough numbers of nonrespondents from the address characteristics subgroups to meet the recruiting targets for the nonresponse follow-up study. Due to differences in mail delivery challenges in urban versus rural areas (which are key for understanding potential reasons for nonresponse to a mailed survey such as NHES), urbanicity will be one of the factors used in the selection of sites. In addition, we will ensure that each of the at least three clusters will include a diverse group of addresses by characteristics such as region, address characteristics, demographics, etc. Initially, in each selected cluster, address/neighborhood observations will be conducted with the nonresponding households selected from that cluster. From the observed addresses, across all clusters, a subset of 500 addresses will be selected for recruitment for the target 100 in-person qualitative interviews. At this time, the cost of observations and interviews will be determined and, budget permitting, additional NHES:2019 screener nonrespondent addresses may be selected for additional address/neighborhood observations.

2.1.1 Recruitment

The following recruitment protocol will be used after the NHES:2019 screener data collection closes in late May 2019:

Sampled cases will be sent an invitation letter that includes $5 cash, invites them to participate in the in-person study, informs them of the contingent incentive, and provides them with NHES staff contact information for letting the researchers know that they would like to participate. All sampled cases will be offered an additional $120 cash incentive for completing the interview and this incentive will be mentioned in all recruitment materials. This incentive is designed to demonstrate to participants that their time and participation is valued and takes into account that the sample is comprised of households that have already shown reluctance to participate in the survey (see below for additional discussion of the incentive amount).
Up to two reminder post cards will be sent to each sampled household that has not yet responded to the nonresponse follow-up study invitation. The first will be sent in the early part of the recruitment period, approximately one week after the invitation letter. A second will be sent to the remaining follow-up nonrespondents about halfway through the recruitment period. These postcards will invite sampled households to participate in the study, remind them about the contingent incentive, provide them with NHES staff contact information for letting the researchers know that they would like to participate, and inform them that an interviewer will be visiting their home to try to schedule an interview.
Recruitment phone calls will occur in two windows for sampled households that have phone numbers available on the sampling frame and have not yet responded to the nonresponse follow-up study contacts. The first will be early in the recruitment period (shortly after the first reminder postcard) and the other will be about halfway through the recruitment period. During each window, each household will be called up to two times. One voicemail will be left per calling window if the household does not answer the phone.
An in-person recruitment period will also take place, where all households that have neither agreed nor declined to participate will be visited by an interviewer. Interviewers will have an identification badge, which they will show to each household along with study forms to indicate that they are part of the study.

The phone and in-person recruitment will encourage the household to participate in the in-person interview, screen for eligibility (e.g., the participant needs to have been living at the address when the NHES:2019 materials were sent), and set up a time to do the interview. The interviewer will attempt to speak with an adult living at this address who usually opens the mail. Should that person not be available, the interviewer will attempt to speak to another available adult living in the household. If, when the interviewer arrives, no one answers the door, the interviewer will leave a “Sorry We Missed You” card at the address. The card will indicate that someone stopped by and will return at a later day and time. It will also contain a phone number and e-mail address the sample member can contact to learn more about the study and/or schedule and interview.

All materials will be branded as coming from the National Center for Education Statistics (NCES). For sampled addresses that are expected to have a high likelihood of being Spanish speaking, the mailed recruitment materials will include both English and Spanish, and bilingual phone and in-person staff will be assigned to these cases.

To maintain engagement and increase the likelihood that recruited households keep their scheduled appointments, the interviewers also will ask recruited households to provide contact information that can be used to keep them engaged and aware of their upcoming interview appointment. Depending on the type of contact information provided, recruited households will receive: (1) a confirmation email or text shortly after scheduling the appointment (or an appointment reminder card if they are recruited in-person), and (2) a reminder email and/or phone call the day before the interview.

2.1.1.1 Use of Incentives

Given the length of the interview and the fact that the target population is comprised of survey nonrespondents, an incentive will be offered to households who complete the interview. Some examples of other incentives in in-person data collections follow:

In a comparison study of incentive groups in qualitative research, 3 monetary groups ($25, $50, and $75) all resulted in greater willingness to participate in the survey than no incentive or a nonmonetary incentive (Kelly et al 2017).
Similar incentive amounts were used in experiments as part of the National Survey of Family Growth (NSFG). In one experiment, a preincentive ($10 or $40 was mailed to sample members) and an additional $40 was offered to a small sub-set of nonresponders in later weeks in the data collection. The $80 total incentive increased response rates in both the screener and main survey by 10 and 12 percentage points respectively, and was more successful at recruiting different people (busy, college-educated, childless women, high-income men, and Hispanic men) into the sample than the $50 total incentive (Lepkowski et al. 2013).
The 2012/2014 Program for the International Assessment of Adult Competencies (PIAAC), an international household survey of adult skills conducted in-person, offered $5 to screener respondents. PIACC offered an additional $50 to households who then completed the assessment in-person. PIAAC achieved an overall weighted response rate of 70.3% for the main study (Hogan et al. 2016).
The in-person household component of the Medical Expenditure Panel Survey (MEPS) conducted an experiment beginning in 2008 for 5 rounds of the panel data collection. Households received $30, $50, or $70 for completing the survey. Response rates were higher for both the $50 and $70 incentive group relative to the $30 group, and the difference between the $70 (71.13%) and $50 (66.74%) groups was also statistically significant. In 2011, OMB approved the new $50 incentive for the MEPS collection.
The National Health and Nutrition Examination Survey (NHANES) involves multiple health exams and tests for respondents. NHANES has various incentive amounts for different subgroups of people. The maximum incentive for adults ages 16 and older who agree to participate in the exam component at a preselected time is $125. The amount decreases as cooperation decreases (e.g., adults ages 16 and older who refuse the exam component at a preselected time is $90). Additional follow-ups include additional $30-$50 incentives. NHANES respondents from low-income households often perceive the medical exams as a tangible, direct benefit to their well-being, which will not be true of the NHES interviews, wherein the benefits of participation are civic engagement and volunteerism, which are comparatively indirect benefits.
In the National Food Acquisition and Purchase Survey (FoodAPS), all contacted households were offered $5. The week-long data collection which included training, a food acquisition diary with scanners, two CAPI interviews, one computer-assisted telephone interview (CATI), and 3 respondent initiated call-in periods, offered eligible households a multi-part incentive designed to encourage initial agreement to participate. A base of $100 was offered with additional $10-$20 to other household members who agreed to participate. The average household incentive was $180 with a range of $130-$230 depending on household size. A previous experiment was conducted using $50 as the base incentive with the average household incentive receiving $130 and a range of $80-$180 depending on household size. Results showed that providing the larger base incentive yielded higher response rates.
The Survey of Consumer Finance (SCF) analyzed field effort outcomes between 2007 and 2010 after the base incentive increased from $20 in 2007 to $50 in 2010. On average, the $50 group needed four fewer attempted contacts before agreeing to participate relative to families receiving no incentive (To 2015).

Though many large studies use in-person data collection methods and/or burden respondents in excess of 1 hour, no study is a good match in the same type of sample members, burden, methods, and respondent experience to the proposed study of nonresponding households. To fully understand this unique target population, it is crucial to ensure a large enough, representative, and cooperative sample. Additionally, the need for high quality data from these sample members is of paramount importance for understanding reasons of nonresponse in studies like NHES. Based on the varying literature on higher incentive amounts and the unique target population of nonrespondents, all households will be offered a $120 cash incentive for completing the interview and this incentive will be mentioned in all recruitment materials.

2.1.2 Qualitative interview protocol

Qualitative interviews will be voluntary and will last 90 minutes to allow time to obtain consent, give participants sufficient time to reflect on the more open-ended questions included in this protocol, and conduct observations. Because this is exploratory research (trying to find out why something is happening as opposed to testing a hypothesis) maximizing contact time with the participants and giving them enough space to discuss their experience is crucial. Interviews will be conducted in both English and Spanish. All interviews will be audio-recorded with participants’ permission. They will consist of the following kinds of questions:

Qualitative interview questions that aim to gain a deep understanding of the household’s reasons for nonresponse and what might convince them to respond (covering domains such as understanding of surveys, privacy concerns, attitudes toward the government, attitudes toward education, time use, and civic engagement);
More structured questions, such as asking about the participant’s demographic characteristics, or asking if the participant remembers receiving the NHES mailings;
A mail interaction discussion, using a packet of example mail where the interviewer asks the participant to model how they sort mail and make decisions about what to keep, read, or throw away, while the interviewer probes about that decision process; and
A request for feedback on NHES:2019 screener mailed materials.

2.1.3 Observations

The interview will conclude with the interviewer making observations about the home and/or household members, beyond any observations recorded while interacting with the participant during the interview. Observations are important to include in qualitative research because interview questions can often only address attitudes or actions of which participants are cognitively aware. There are likely other observable aspects of participants’ lives that they may not explicitly connect to their lack of participation (e.g., how they organize their mail), as well as observable qualities that nonresponding households may have in common (e.g., whether they display children’s educational achievements). These observations will be used to capture such more nuanced indicators.

The finalized interview protocol and contact materials for the In-Person Study of NHES:2019 Nonresponding Households qualitative interviews can be found in the Attachment B of this document.

2.2. Address/neighborhood observation

Approximately 750 addresses will be sampled for address and neighborhood observations. Of these, 500 will be also invited to take part in the qualitative interview study (see above), and for the remaining 250 households, only observations will be collected. The objective of these observations is to determine the types of addresses that are prone to nonresponse or having their NHES mailings be undeliverable, and to assess the accuracy of the information available on the frame for such addresses.

The objective of these observations, which will be carried out by staff contracted for NHES, is to determine the types of addresses that are prone to nonresponse or having their NHES mailings be undeliverable and to assess the accuracy of the information available on the frame for such addresses. The instrument has been programmed to be administered on a tablet or computer to ease data collection efforts and minimize error. Each observer will receive a tablet to complete their observations. The 7-minute observation instrument is designed to evaluate variables from the address-based sample frame. Several of the items are based on existing instruments (e.g., the Neighborhood Observation Instrument (NOI), Contact History Instrument (CHI), the Los Angeles Family and Neighborhood Survey). The instrument also includes a series of characteristics that interviewers will look for and mark off when they observe them for that address or neighborhood. There is also space for entering notes that might provide helpful context about the address or neighborhood. Exhibit 2 below displays a summary of the proposed measures, the frame variables that will be evaluated, and the potential impacts those findings could have on the design of future NHES administrations. To increase efficiency, observers may audio record their observations and transcribe them onto the instrument later.

Exhibit 2. Summary of proposed measures, frame variables evaluated, and potential impact on NHES design

Measure	Frame Variable(s) evaluated	Potential Impact
Structure type	Mailing address type Dwelling type Address vacancy status Seasonal address Delivery point usage code	Mailing method (e.g., use courier service such as FedEx)
		Contact materials (e.g., use larger font for address to minimize sorting issues for multi-unit addresses, tailor language of contact materials)
		Incentive use (e.g., use varied incentives by structure type)
		Frame accuracy (e.g., evaluate frame information to improve adaptive design models)
Occupancy	Address vacancy status Seasonal address	Frame accuracy (e.g., evaluate frame information to improve adaptive design models)
Occupancy	Address vacancy status Seasonal address	Eligibility (e.g., confirm whether the postmaster ‘vacant household’ status is accurate)
Eligibility	Presence of children	Topical Eligibility (e.g., assess whether the nonresponse process differs based on topical eligibility)
		Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
		Contact materials (e.g., tailor language used in survey invitation to alleviate privacy concerns of certain households such as households with children)
Mail Access	Mailing address type Drop point Delivery point type code	Mailing method (e.g., using courier services that provide door-to-door service)
		Contact materials (e.g., use bolder envelope designs such as larger font for addresses where mail mix-up is likely)
		Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
Household income	Household income	Incentive use (e.g., use varying incentive structures by income level)
Household income	Household income	Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
Additional address / neighborhood observations	None (these observations will be used to assess whether additional census planning database variables should be used for making data collection protocol changes)	Contact materials (e.g., tailor emphasis of individual (incentive) versus communal (policy impact) advantages of NHES participation)

As part of the 7-minute observation, observers will also take a picture of the sampled address to validate that the observer was indeed present at the address, to document a time and location of the visit, and to clarify any issues related to the observation the observer may have recorded. The photos will only be taken from public spaces such as the street, and will not be taken while on someone’s private property. The finalized observation protocol for the In-Person Study of NHES:2019 Nonresponding Households is provided in Attachment B.3 of this document.

Looking ahead: making use of the study findings

The priority for analysis will be on identifying actionable responses to the themes identified in the interviewer observations and the participant responses and comments. The protocols for this study are written with that aim in mind. Some global changes to NHES are likely to result from the In-Person Study of NHES:2019 Nonresponding Households, but it is anticipated that most findings will be targeted at specific priority or hard-to-reach subgroups that can be identified prior to the start of data collection. This may include, for example, changing the timing or mode of nonresponse follow-up for certain subgroups, altering the wording or presentation of contact materials, or changing the value or timing of incentives.

To help ensure that the results of this study reach the maximum possible audience, contribute to the survey methodology literature, and serve as a resource for others conducting federal government household surveys (particularly those using ABS frames or contacting household by mail), it is anticipated that the findings may be shared through the following outlets:

An official NCES publication posted on the NCES website that summarizes the methodology and key findings of the study. This report will identify key themes from the interviews and propose actionable strategies that could be implemented in future administrations of NHES and other household surveys.
A manuscript, based on the above report, that is submitted to a leading survey methodology journal, such as Public Opinion Quarterly, the Journal of Survey Statistics and Methodology, or Field Methods.
At least two presentations submitted to conferences, such as the American Association for Public Opinion Research national conference or the Federal Committee on Statistical Methodology Research and Policy conference.

References

AAPOR Task Force on Survey Refusals. (2014). Current knowledge and considerations regarding survey refusals. American Association for Public Opinion Research.

AAPOR Task Force on Address-Based Sampling. (2016). Address-based sampling. American Association for Public Opinion Research.

Abraham, K., Maitland, A., and Bianchi, S. (2006). Nonresponse in the American Time Use Survey: Who is missing from the data and how much does it matter? Public Opinion Quarterly, 70, 676-703.

Ahlmark, N., Algren, M., Holmberg, T., Norredam, M., Nielsen, S., Blom, A., Bo, A., and Juel, K. (2015). Survey nonresponse among ethnic minorities in a national health survey -- a mixed-method study of participation, barriers, and potentials. Ethnicity & Health, 20, 611-632.

Amaya, A., and Harring, J. (2017). Assessing the effect of social integration on unit nonresponse in household surveys. Journal of Survey Statistics and Methodology, 2017, 480-508.

ASA Task Force on Improving the Climate for Surveys. (2017). Interim report. American Association for Public Opinion Research.

Bates, N. (2007). The Simplified Questionnaire Report. Results from the Debriefing Interviews. Accesses 4/10/18 from https://www.census.gov/srd/papers/pdf/rsm2007-03.pdf

Brick, J., and Williams, D. (2013). Explaining rising nonresponse rates in cross-sectional surveys. Annals of the American Academy of Political and Social Science, 645, 36-59.

Czajka, J., and Beyler, A. (2016). Declining response rates in federal surveys. Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. Washington, DC.

Groves, R., Cialdini, R., and Couper, M. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475-495.

Groves, R., Presser, S., Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, 2-31.

Harcombe, H., Derrett, S., Herbison, P., and McBride, D. (2011). "Do I really want to do this?" Longitudinal cohort survey participants' perspectives on postal survey design: a qualitative study. BMC Medical Research Methodology, 11, 1-9.

Hogan, J., Thornton, N., Diaz-Hoffmann, L., Mohadjer, L., Krenzke, T., Li, J., VanDeKerckhove, W., Yamamoto, K., and Khorramdel, L. (2016). U.S. Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014: Main Study and National Supplement Technical Report (NCES 2016-036REV). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Available from http://nces.ed.gov/pubsearch

Kelly, B., Margolis, M., McCormack, L., LeBaron, P.A., and Chowdhury, D. (2017). What affects people’s willingness to participate in qualitative research? An experimental comparison of five incentives. In Field Methods (29).

Kulka, R. A., Holt, N. A., Carter, W., and Dowd, K. L. (1991). Self-reports of time pressures,

concerns for privacy, and participation in the 1990 mail Census. Proceedings of the Bureau of the Census Annual Research Conference, Arlington, VA.

Lepkowski, J., Mosher, W., Groves, R., West, B., Wagner, J., and Gu, H. (2013). Responsive design weighting, and variance estimation in the 2006-2010 National Survey of Family Growth. 2(158). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Retrieved from https://stacks.cdc.gov/view/cdc/22069

Little, R. and Vartivarian, S. (2005). Does weighting for nonresponse increase the variance of survey means? Survey Methodology, 31, 161-168.

McPhee, C., Bielick, S., Masterton, M., Flores, L., Parmer, R., Masica, S., Shin, H., Stern, S., and McGowan, H. (2015). National Household Education Surveys Program of 2012: Data file user’s manual (NCES 2015-030). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

McPhee, C., Jackson, M., Bielick, S., Masterton, M., Battle, D., McQuiggan, M., Payri, M., Cox, C., and Medway, R. (2018). National Household Education Surveys Program of 2016: Data file user’s manual (NCES 2017-100). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

Nakash, R., Hutton, J., Lamb, S., Gates, S., and Fisher, J. (2008). Response and non-response to postal questionnaire follow-up in a clinical trial – a qualitative study of the patient's perspective. Journal of Evaluation in Clinical Practice, 14, 226-235.

O’Neill, G. and Sincavage, J. (2004). Response Analysis Survey: A Qualitative look at Response and Nonresponse in the American Time Use Survey. Retrieved from https://www.bls.gov/ore/pdf/st040140.pdf Washington, DC: Bureau of Labor Statistics.

Presser, S., and McCulloch, S. (2011). The growth of survey research in the United States: Government-sponsored surveys, 1984–2004. Social Science Research, 40, 1019-1024.

Singer, E., and Presser, S. (2008). Privacy, confidentiality, and respondent burden as factors in telephone survey nonresponse. In J. M. Lepkowski, C. Tucker, J. M. Brick, E. D. de Leeuw, L. Japec, P. J. Lavrakas, M. W. Link, and R. L. Sangster (Eds.), Advances in telephone survey methodology (449–470). New York: John Wiley.

Singer, E. (2016). Reflections on surveys’ past and future. Journal of Statistics and Survey Methodology, 4, 463-475.

To, N. (2015). Review of Federal Survey Program Experiences with Incentives. Washington, DC: Bureau of Labor Statistics. Retrieved from: https://www.bls.gov/cex/research_papers/pdf/Review-of-Incentive-Experiences-Report.pdf

U.S. Census Bureau. (2014). Design and Methodology: American Community Survey. Washington, DC: U.S. Census Bureau. Retrieved from: https://www.census.gov/programs-surveys/acs/methodology/design-and-methodology.html.

United States Census Bureau. (2017). CAPI Manual for NHIS field representatives. Retrieved from https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Survey_Questionnaires/NHIS/2017/frmanual.pdf

User’s guide for the paradata file and codebook documentation: Special file covering 2011-2015 of the National Survey of Family Growth. (n.d.). Retrieved from https://www.cdc.gov/nchs/data/nsfg/NSFG2011_2015_Paradata_UserGuide.pdf

Walsh, R., Dahlhamer, J., and Bates, N. (2013). Assessing interviewer observations in the NHIS. Retrieved from https://www.researchgate.net/publication/296847278_Assessing_Interviewer_Observations_in_the_NHIS

Zelnack, M. and David, M. (2013). Impact of Multiple Contacts by Computer-Assisted Telephone Interview and Computer-Assisted Personal Interview on Final Interview Outcome in the American Community Survey. Washington, DC: U.S. Census Bureau. Retrieved from: https://www.census.gov/content/dam/Census/library/working-papers/2013/acs/2013_Zelenak_01.pdf

Attachment A. Count of sampled households by response status, and weighted screener response rate, by selected household characteristics: NHES:2016

Table A-1. Count of sampled households by response status, and weighted screener response rate, by selected household characteristics

Household characteristic	Count of sampled households					Weighted screener response rate
Household characteristic	Total	Responded	Refused	Ineligible	Unknown eligibility	Weighted screener response rate
Total	206,000	115,342	1,606	19,136	69,916	66.4
Frame variables
Sampling stratum
Tracts with 25% or more Black persons	41,200	18,593	330	5,472	16,805	58.2
Tracts with 40% or more Hispanic persons	30,900	13,906	206	3,404	13,384	55.9
All other tracts	133,900	82,843	1,070	10,260	39,727	69.4
Tract poverty rate
20% or higher	64,760	29,030	468	9,025	26,237	59.5
Below 20%	141,240	86,312	1,138	10,111	43,679	68.9
Census region¹
Northeast	35,398	20,341	281	3,019	11,757	67.2
South	82,478	43,580	662	8,500	29,736	64.0
Midwest	42,817	25,859	336	4,056	12,566	70.6
West	45,307	25,562	327	3,561	15,857	65.6
Route type
City style/street	155,113	93,178	1,252	11,302	49,381	68.4
P.O. box	2,181	785	19	845	532	74.5
High rise	48,507	21,276	333	6,958	19,940	58.9
Rural route	199	103	2	31	63	67.8
Delivery point is a drop point
Yes	3,784	1,558	50	573	1,603	57.5
No	202,216	113,784	1,556	18,563	68,313	66.5
Dwelling type
Single family	150,545	91,419	1,219	10,650	47,257	68.9
Multiple unit	53,274	23,138	368	7,641	22,127	58.5
Dwelling type unknown	2,181	785	19	845	532	74.5
Number of adults in household
1	78,318	39,090	589	8,008	30,631	61.1
2	50,592	32,743	395	2,388	15,066	70.6
3 or 4	38,440	26,435	288	1,225	10,492	73.1
5–8	9,200	6,321	63	260	2,556	72.7
Number of adults unknown	29,450	10,753	271	7,255	11,171	61.6
Children in household
Yes	47,236	28,146	297	2,415	16,378	65.8
No/unknown	158,764	87,196	1,309	16,721	53,538	66.5

See notes at end of table.

Table A-1. Count of sampled households by response status, and weighted screener response rate, by selected household characteristics—Continued

Household characteristic	Count of sampled households					Weighted screener response rate
Household characteristic	Total	Responded	Refused	Ineligible	Unknown eligibility	Weighted screener response rate
Phone number matched
Yes	134,574	84,010	1,099	7,687	41,778	69.4
No	71,426	31,332	507	11,449	28,138	60.6
Home tenure
Rent	52,220	23,091	330	6,037	22,762	56.6
Own	120,115	79,806	988	5,165	34,156	71.6
Home tenure unknown	33,665	12,445	288	7,934	12,998	61.0
Income
$50,000 or less	83,949	43,740	673	8,065	31,471	62.8
$50,001–$100,000	55,967	36,271	401	2,446	16,849	70.2
$100,001–$150,000	21,839	14,376	147	822	6,494	70.5
$150,001 or more	14,772	10,189	114	544	3,925	73.3
Income unknown	29,473	10,766	271	7,259	11,177	61.6
Treatment variables
Assigned mode at first screener mailing
Paper	171,000	97,315	1,256	15,819	56,610	67.2
$5 incentive	126,000	72,064	914	11,593	41,429	67.5
Other incentive ($2 or modeled)	45,000	25,251	342	4,226	15,181	66.5
Web²	35,000	18,027	350	3,317	13,306	62.1
Screener incentive
$5	161,000	90,091	1,264	14,910	54,735	66.3
$2	10,000	5,449	71	922	3,558	64.9
Modeled incentives³	35,000	19,802	271	3,304	11,623	67.0
$0	1,750	1,393	18	55	284	82.8
$2	6,996	5,176	63	210	1,547	77.2
$5	21,007	11,502	150	1,932	7,423	64.2
$10	5,247	1,731	40	1,107	2,369	54.0

¹ Northeast includes Pennsylvania, New York, New Jersey, Connecticut, Rhode Island, Massachusetts, Vermont, New Hampshire, and Maine. South includes Florida, Georgia, South Carolina, North Carolina, Virginia, District of Columbia, Maryland, Delaware, West Virginia, Alabama, Mississippi, Tennessee, Kentucky, Arkansas, Louisiana, Texas, and Oklahoma. Midwest includes North Dakota, South Dakota, Nebraska, Kansas, Missouri, Iowa, Minnesota, Wisconsin, Illinois, Indiana, Michigan, and Ohio. West includes New Mexico, Colorado, Wyoming, Montana, Idaho, Utah, Arizona, Nevada, Washington, Oregon, California, Hawaii, and Alaska.

²All households assigned to the web mode received a $5 screener incentive.

³Incentives in the modeled group were assigned according to predicted response propensity, with households with a higher predicted response propensity receiving a lower incentive.

NOTE: Weighted screener response rate is calculated using the single-eligibility formula (AAPOR RR3).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Household Education Surveys Program (NHES) of 2016.

Attachment B. In-Person Study of NHES:2019 Nonresponding Households Qualitative Interview Materials

B.1. English Materials

Attachment B.1. presents the English version of the materials that will be used to interact with households sampled for the qualitative interview. These materials include:

Invitation Letter
First Reminder Postcard
Second Reminder Postcard
Phone Follow-up Recruitment Script
“Sorry We Missed You!” Card
In-Person Recruitment Script
Commonly Asked Questions
Interview Consent Form
Interview Protocol
Income Show Card
Incentive Receipt Acknowledgment Form

B.1.1. Invitation Letter

<Mailing_add1>

<Mailing_add2>

Dear <City> Household:

The National Center for Education Statistics (NCES) and American Institutes for Research (AIR) are doing an important study in your area.

This study will help us understand how to make NCES research more useful and helpful. We would like to get your feedback on a variety of topics, such as how you like to spend your time, how you go about day-to-day tasks like sorting your mail, and how you feel about participating in research.
This voluntary, 90-minute conversation takes place in your home at a convenient time for you.

We know that you are busy, and we have enclosed $5 as a thank you for your time. Because your participation is so important, you also will get $120 cash as a thank you for sharing your opinions and experiences with us.

To see if you are eligible to participate, please do one of the following:

Call us toll-free at 1-XXX-XXX-XXXX
E-mail us at [email protected]

Thank you in advance for participating. We look forward to hearing from you.

Sincerely,

[SIGNATURE]

James Lynn Woodworth

Commissioner, National Center for Education Statistics

Authorization and Confidentiality of Information

American Institutes for Research is administering this voluntary study on behalf of the National Center for Education Statistics (NCES). There are no penalties should you choose not to participate in this study. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). The National Center for Education Statistics (NCES), within the U.S. Department of Education, is authorized to conduct the National Household Education Survey (NHES) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543).

B.1.2. First Reminder Postcard

Shape1

Would you like $120 for sharing your opinions?

The National Center for Education Statistics (NCES) and American Institutes for Research (AIR) are doing an important study in your area. This study will help us understand how to make NCES research more useful and helpful.

Because your participation is so critical, you will get $120 cash as a thank you for sharing your opinions and experiences with us.
This voluntary, 90-minute conversation takes place in your home at a convenient time for you.
We would like to get your feedback on a variety of topics, such as how you like to spend your time, how you go about day-to-day tasks like sorting your mail, and how you feel about participating in research.
A researcher from AIR will stop by in the next few weeks to invite you to participate.
To find out sooner if you are eligible to participate: call us toll-free at 1-XXX-XXX-XXXX or e-mail us at [email protected].

Thank you in advance for participating. We look forward to hearing from you!

B.1.3. Second Reminder Postcard

Shape2

Chat with us to help

make research more useful!

The National Center for Education Statistics (NCES) and American Institutes for Research (AIR) are doing an important study in your area. This study will help us understand how to make NCES research more useful and helpful.

An AIR researcher will stop by in the next few weeks to invite you to participate.
To find out sooner if you are eligible to participate: call us toll-free at 1-XXX-XXX-XXX

or e-mail us at [email protected].

We would like to get your feedback on a variety of topics, such as how you like to spend your time, how you go about day-to-day tasks like sorting your mail, and how you feel about participating in research.
Because your participation is so important, you will get $120 cash as a thank you for sharing your opinions and experiences with us.
This voluntary, 90-minute conversation takes place in your home at a convenient time for you.

Thank you in advance for your participation. We look forward to hearing from you!

B.1.4. Phone Follow-up Recruitment Script

IF NO ONE ANSWERS THE PHONE, LEAVE THE FOLLOWING MESSAGE:

Hello, this is [YOUR NAME] from American Institutes for Research, calling on behalf of the National Center for Education Statistics. We are doing an important study in your area. We would like to meet with you to get your feedback on a variety of topics, such as how you like to spend your time and how you go about daily tasks like sorting mail. Because your participation is so critical, you will get $120 cash as a thank you for speaking with us. If you would like to participate, please call us toll free at 1-xxx-xxx-xxxx.

IF SOMEONE ANSWERS THE PHONE:

Hello, this is [NAME] from American Institutes for Research, calling on behalf of the National Center for Education Statistics. Am I speaking with a resident of [ADDRESS]?

If yes, continue with the next paragraph.
If no, say: I am sorry. I have the wrong number. Thank you for your time. Have a nice day. End phone call.

I am calling to follow up on a letter that was recently mailed to your house. The National Center for Education Statistics and American Institutes for Research are conducting an important study in your area that will help us understand how to make our research more useful and helpful for people like you. We would like to meet with you to get your feedback on a variety of topics, such as how you like to spend your time and go about daily tasks like sorting mail. If you complete a 90-minute conversation with us in your home, we will give you $120 cash to show our thanks.

Are you interested in this opportunity?

If yes, continue with the next paragraph.
If no, try to address their concerns. If still not interested, say: No problem. Thank you for your time. Have a good day. End phone call.

Great. First, I need to ask a few questions to make sure you are eligible to participate.

Did you live or stay here most of the time between January and April 2019?

If yes, continue with next screener item.
If no, say: Unfortunately, we can only meet with people who lived or stayed at this address most of the time between January and April 2019. Is there anyone else living here who lived or stayed here most of that time?

If yes, ask if you can speak with them.
- When that person is on the phone, go back to the study introduction paragraph and then restart the screener.
- If the person is not available right now, ask when would be a good time for you to call back, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.
If no one else lived here during that time period, thank them for their time, end phone call and do not call again.

Ok, great. Can you please confirm your age?

If 18 or older, continue with next screener item.
If 17 or younger, say: Unfortunately, only adults can participate. Is there an adult at home with whom I can speak?

When the adult is on the phone, go back to the study introduction paragraph and then restart the screener.
If an adult is not available right now, ask when would be a good time for you to call back to speak with one, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.

Great, thanks. Are you involved in handling your household’s postal mail - for example, collecting, reviewing, or responding to the mail?

If yes, continue with next screener item.

If no, say: Unfortunately, we can only meet with one of the people who handles your household’s mail. Can I please speak to one of those people?

When that person is on the phone, go back to the study introduction paragraph and then restart the screener.
If the person is not available right now, ask when would be a good time for you to call back, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.

Great, thank you! You are eligible to participate in the study. What days [this/next] week would be convenient for you to meet with us?

Looking at when interview team is available in that area, work with participant to schedule the interview. Our preference is to do interviews sooner than later.

Once interview date/time are decided, record them and say: We have you scheduled on [DATE] at [TIME]. You will be meeting with [INTERVIEWER NAME].

We would like to send you a confirmation message with the date and time of your interview. Do you prefer that we text you or that we e-mail you?

If willing to receive confirmation message: And what is the best [phone number/e-mail address] for reaching you?
If not willing to receive confirmation message, reassure them that this information will only be used for communicating with them about the interview. If still not willing, say: Ok, no problem.

We would also like to reach out to you the day before the meeting to confirm that this time still works for you. Do you prefer that we call you or that we e-mail you?

If willing to receive reminder message and request a different mode than for the confirmation message: And what is the best [phone number/e-mail address] for reaching you?
If not willing to receive reminder message, reassure them that this information will only be used for communicating with them about the interview. If still not willing, say: Ok, no problem.

Thank you for participating! If any conflicts come up between now and [DATE OF INTERVIEW] please call our toll-free number at 1-XXX-XXX-XXXX or email us at [email protected].

We look forward to seeing you soon. Bye. End call.

B.1.5. “Sorry We Missed You!” Card

Sorry We Missed You!

The National Center for Education Statistics (NCES) and American Institutes for Research (AIR) are doing an important study in your area. We stopped by to invite you to participate.

This study will help us understand how to make NCES research more useful and helpful. We would like to get your feedback on a variety of topics, such as how you like to spend your time, how you go about day-to-day tasks like sorting your mail, and how you feel about participating in research.
If you meet with us, you will receive $120.
This voluntary conversation lasts about 90 minutes and takes place in your home at a convenient time for you.

Contact us at 1-xxx-xxx-xxxx or [email protected] to see if you are eligible!

B.1.6. In-Person Recruitment Script

INTRODUCTION

If the person who answers the door is clearly a child, ask about speaking to an adult in the house. Otherwise, say:

Hi, my name is [YOUR NAME] from American Institutes for Research. We recently sent a letter to your house about an important study we are doing with the National Center for Education Statistics. We would like to have a conversation with you to get your opinions on a variety of topics, such as how you like to spend your time, how you go about day-to-day activities like sorting your mail, and how you feel about participating in research. The conversation lasts 90 minutes, and we will give you $120 cash as a thank you for your time. Are you interested in this opportunity?

If yes, continue with the next section.
If no, try to address their concerns. If still not interested, say: No problem, I understand. Is anyone else from this household available to speak with us?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No problem. We can come back at a more convenient time for them. What would be a good time for us to come back? [record date/time, and if possible the other household member’s first name to facilitate the return visit] Here is a card with more information about the study that you can give to them. It also lists our contact information in case they would like to contact us sooner about participating. [hand Sorry We Missed You card to household member] Thank you for your time! Have a nice day.
- If no one else lives here or not interested in you speaking with anyone else, say: I understand. Would you be willing to answer a few quick questions instead? It will take less than five minutes.
  - If no: No problem. Thank you for your time. Have a nice day. (end interview)
  - If yes, go to Short Interview with Full-Interview Refusals: Great, thank you.

SHORT INTERVIEW WITH FULL-INTERVIEW REFUSALS

First, can you please confirm that you are at least 18 years old?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
Can you also confirm that we are at [STREET ADDRESS]?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
Did you live or stay here most of the time between January and April 2019?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
Do you handle your home’s mail – for example, do you collect, review, or respond to the mail?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
Have you gotten any mail from the Department of Commerce or Census Bureau since January 1st, 2019?
- Yes (continue)
- No (continue)
- Don’t know (continue)
- Refused (continue)
How about any mail about the National Household Education Survey? Have you gotten any mail like this since January 1st, 2019?
- Yes (ask QG)
- No (go to QH)
- Don’t know (go to QH)
- Refused (go to QH)
Would you be willing to share some of the reasons you did not complete this survey? (Allow respondent a moment to think.)
Are any of the people who live here age 20 or less?
- Yes (go to end interview)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)

End interview: Those are all my questions. Thank you for your time. Have a nice day!

SCREENER

Once the interested household member is at the door:

Great! First, I need to ask a few quick questions to make sure you are eligible to participate.

Can you please confirm that we are at [STREET ADDRESS]?

If yes, continue with next screener item.
If no, say: Unfortunately, I can only meet with someone who lives at [STREET ADDRESS]. I am sorry; I must have the wrong house. Thank you for your time. Have a nice day. End conversation.

Did you live or stay here most of the time between January and April 2019?

If yes, continue with next screener item.
If no, say: Unfortunately, we can only meet with people who lived or stayed at this address most of the time between January and April 2019. Is there anyone else living here who lived or stayed here most of that time?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No problem. We can come back at a more convenient time for them. What would be a good time for us to come back? [record date/time, and if possible the other household member’s first name to facilitate return visit] Here is a card with more information about the study that you can give to them. It also lists our contact information in case they would like to contact us sooner about participating. [hand Sorry We Missed You card to household member] Thank you for your time! Have a nice day.
- If no one else lives here or not interested in you speaking with anyone else, say: No problem. Thank you for your time. Have a good day.

Ok, great. Can you please confirm your age? [do not ask this if respondent is clearly much older than 18, but if there is any uncertainty make sure you confirm their age]

If 18 or older, continue with next screener item.
If 17 or younger, say: Unfortunately, only adults can participate. Is there an adult at home with whom I can speak?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No problem. We can come back at a more convenient time for them. What would be a good time for us to come back? [record date/time, and if possible the other household member’s first name] Here is a card with more information about the study that you can give to them. It also lists our contact information in case they would like to contact us sooner about participating. [hand Sorry We Missed You card to household member] Thank you for your time! Have a nice day.
- If no one else lives here or not interested in you speaking with anyone else, say: No problem. Thank you for your time. Have a good day.

One last question: Are you involved in handling your household’s postal mail - for example, collecting, reviewing, or responding to the mail?

If yes, continue with next screener item.
If no, say: Unfortunately, we need to have the conversation with one of the people who handles your household’s mail. Can I please speak to one of those people?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No problem. We can come back at a more convenient time for them. What would be a good time for us to come back? [record date/time, and if possible the other household member’s first name to facilitate return visit] Here is a card with more information about the study that you can give to them. It also lists our contact information in case they would like to contact us sooner about participating. [hand Sorry We Missed You card to household member] Thank you for your time! Have a nice day.
- If no one else lives here or not interested in you speaking with anyone else, say: No problem. Thank you for your time. Have a good day.

SCHEDULING

Great, thank you! You are eligible to participate in the study. We can meet right now if it is convenient for you. The conversation will take about 90 minutes.

If they are available now, start the interview.
If they are not available now, say: No problem. We can come back at a time that is more convenient for you. What days this week would be convenient for you to meet?
- Based on your availability, work with the respondent to find a time for the interview.
- Record the date/time for yourself, and on an Appointment card, and say: We have you scheduled on [DATE] at [TIME]. Here’s an Appointment confirmation card for you to keep as a reminder.
- Hand them the card and then say: And could I have your first name for my appointment calendar?
- Record name, then say: We would like to reach out to you the day before the conversation to confirm that this time still works for you. Do you prefer that we call you or that we e-mail you?
- Once they respond, say: And what is the best [phone number/email address] for reaching you? If they express concern about providing this information, let them know this will only be used to remind them about the appointment and confirm they are still available.
- Record answer for your records and say: Thank you for participating! If any questions or conflicts come up between now and [DATE OF INTERVIEW], please call or email us using the information listed on your Appointment card. I look forward to seeing you soon. Have a nice day.

B.1.7. Commonly Asked Questions

Commonly Asked Questions

What is this study about? What will you do with my responses?

The focus of the study is to understand how to improve research done by the National Center for Education Statistics, which is part of the Department of Education. We would like to hear about your opinions and experiences with a variety of topics. For example, we would like to know what kinds of things in your everyday life are important to you. We would also like to know about how you handle certain tasks, like collecting and reviewing your home’s postal mail. Finally, we would like to get your feedback on some of our research materials. Ultimately, this information will help us improve the National Household Education Surveys and assure that they are relevant to all household in the United States.

Who is conducting this study?

The Department of Education, specifically the National Center for Education Statistics (NCES), is conducting this study. You can learn more about NCES by going to www.nces.ed.gov.

American Institutes for Research (AIR), a not-for-profit behavioral and social science research firm, is carrying out this study on behalf of NCES. You can learn more about AIR by going to www.air.org.

How much time will this take?

The conversation should take about 90 minutes to complete.

Where will this take place?

The conversation will take place at your home at a time that is convenient for you. We would like to speak with you in your home because we want you to share your experiences and opinions in a setting that is comfortable and familiar to you.

Who will know if I participate in the study?

Only the study team at AIR will know who participated in the study. The information collected during our conversation will be stored on AIR’s secure computer servers. We will not link what you say to anything identifiable about you, such as your name or address.

What do I get if I meet with you?

You will receive $120 cash as a thank you for your participation.

Who can I contact if I have questions about participating in the study?

You can call us toll-free at 1-XXX-XXX-XXXX or e-mail us at [email protected].

B.1.8. Informed Consent Form

Study Participation Consent Form

The U.S. Department of Education’s National Center for Education Statistics (NCES) is conducting this study to understand how to make its research more useful and helpful. NCES has hired American Institutes for Research (AIR) to carry out this study.

We are asking you to participate in a 90-minute conversation where we will ask your opinion about a variety of topics, such as how you like to spend your time, how you go about day-to-day activities like sorting your mail, and how you feel about participating in research.

Your participation will contribute valuable insights that will help us improve NCES surveys; however, your participation is completely voluntary, and there are no penalties should you choose not to participate in this study. You may stop the conversation at any time without penalty.
The information you share with us will be combined with the responses of other participants in a summary report that does not identify you or anyone else individually. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).
If you have any questions or concerns related to your participation in this interview, please contact Sarah Grady at the National Center for Education Statistics. You can reach her at 202-245-7063 or [email protected].
We would like to record this interview to make sure that we accurately capture the details that you provide. However, if you do not agree to be recorded, we will not record the interview. If you agree to be recorded, only AIR and NCES employees working on this study will be able to listen to the recording. All recordings will be destroyed at the end of the study. Do we have your permission to record the interview?

YES NO

By signing your name below, you agree to participate in this study.

Signed: ____________________________________________ Date: ___________________

Printed Name: ________________________________________

B.1.9. Interview Protocol

The goal of each interview is to conduct an efficient interview and observation with an ethnographic approach in order to glean an in-depth understanding of participants’ perspectives and experiences on a variety of topics related to surveys. The interaction should be a conversation where the participant does most of the talking and you, through active listening and engagement, probe them on particular objectives. Therefore, there is no set script. You must rely on your experience and training to emerge from the situation with the rich detail needed to address why this particular household did not respond to the National Household Education Survey.

Previous research has elucidated the role of certain factors, such as civic engagement and privacy concerns, in why people do not respond to surveys. However, an important component of this qualitative research project is to explore the context (physical, social, and emotional) in which these nonrespondents are embedded, so that themes can arise from the data that help explain why people did not respond to the screener. We are interested in why they did not participate, which is likely to be different for different people. For some, they may not be interested in taking surveys in general. For others, they may have wanted to participate but other factors in their lives intervened. Additionally, some households may not have received the initial screener. To accomplish that goal, the interviewer and the interview guide need to be flexible and able to explore and probe on issues the participant deems relevant.

Before you begin the interview, give the informed consent document to the participant, answer any questions they may have, and begin recording with their permission. Once they sign the informed consent, put it away in a folder.
Start with rapport building through basic questions about household make-up (Domain 1).
Next, proceed with the Experience of Mail Delivery and Experience of NHES Materials and Screener domains (Domains 2 and 3).
After this, you should rely on your training to decide what is the best course forward that will yield the most informative interview; however, you should at least touch on all of the domains listed in this document.
When there are about 5 minutes left in the interview, ask the questions in the demographics section.
Finally, hand the participant their incentive and make sure they sign the incentive receipt form.

Though we generally want to start with the Experience of Mail Delivery and Experience of NHES Letter and Screener domains, we expect that other domains, such as time use and privacy concerns, will arise during these discussions. The domains may flow into one another or may not be discrete categories to the participant. For example, you may begin talking about the participant checking her mail, which leads into a conversation about what activities she enjoys in her free time. Should that happen, you would want to interweave questions about time use and the related objectives. Although the conversation will be free-flowing, we want to get a chronological sense of how they handle their mail. For example, do they check their mailbox every evening around the same time? Do they review the mail immediately? Do they sort the mail while standing at the mailbox? Do they come inside and put it away to sort for later?

The remainder of this document lays out the key domain/objectives to be covered in the interview and proposes sample questions and probes you may want to ask in order to fully explore that objective. However, the key to this interview is your ability to listen to participants’ responses and devise follow-up questions in the moment that serve to deepen our understanding of their point of view. The sample questions are examples and may not be relevant or meaningful to every participant. Also, it is very important to let the participant use their own words to answer questions. Probe without putting words in their mouth with very open questions. For instance, ask “Why do you do that?” as opposed to “Do you do that so you can X?”

In addition to understanding the participants’ experiences around the given objectives, each interview will include two activities as part of the Experience of Mail Delivery and Experience of NHES Materials and Screener domains.

Mail Delivery: In the first, you will offer the participant a stack of mail and the participant will talk aloud about the process of sorting, organizing, and making determinations about how they would handle each piece if they received it, and will show you where and how that process occurs in the home.
NHES Materials and Screener: In the second, you will hand the participant NHES materials that were mailed to them earlier this year and the participant will talk aloud about his or her reactions to and opinions about the mailings. Finally, you will show the participant the NHES screener questionnaire and talk through their opinions and reactions.

Throughout the interview, please incorporate the participant’s surroundings into your questioning as appropriate. Ultimately, you want to pull in objects or observations to discuss if you believe they may be relevant to why this household did not respond to the screener. For example, if you see an American flag on the desk you may want to ask something like: “Can you tell me a little bit about the American flag on your desk?” or “I see you have an American flag on your desk. What, in particular, is important to you about displaying the flag?” The household observation is part of the interview process. You will want to take note of their surroundings: what do they hang on their walls? Do you see visible mail piles? Are there children’s toys present? What types of books or magazines are displayed? At the conclusion of the interview, in your field notes, you will need to write down your observations and describe (using thick description as discussed in the training) the exterior of the home and the rooms to which you had access during interview.

INTERVIEW DOMAINS

Domain 1: Rapport Building and Household Make up

Objective: To understand key aspects of the participant’s household composition and build rapport.

Sample Questions

Tell me a little about who lives here with you.
I see that you have three kids. What types of activities do they enjoy?
What drew you to the neighborhood?
I noticed X (such as a security sign, holiday décor, etc.) outside your home. Can you tell me a little about it?

Domain 2: Experience with Mail Delivery

Objective: To understand how the household interacts with their mail. For example, how mail enters the participant’s household, what happens to it after it arrives, how participants decide what mail to keep/open/throw away, and how easy or difficult it is for the household to check or send their mail. Have the participant walk you through their mail receipt routine, asking questions about timing and activities along the way. During this exchange you will incorporate the activity below.

Sample Questions

Where do you get your mail (home delivery mailbox, home delivery mail slot, P.O. Box)? Can you show me? (if nearby)
What kinds of mail do you typically get?
How do you know when mail is delivered?
How often do you check your mail?
Where do you send your mail (walk to P.O. box, down a building mail chute, mailbox, etc.)? Can you show me? (if different from delivery)
How do you decide which pieces of mail to open?
How would you describe the kinds of mail you keep when you sort through unopened mail?
How about the kinds of mail that you throw away?
What makes you throw away a piece of mail before opening it?
What makes you throw away a piece of mail before reading it?
What do you do with your mail once you’ve read it?
How do you keep track of mail you need to send/respond to?
How often does mail for other addresses get mixed up with your mail?

Mail Interaction Activity

Hand the participant the stack of mail prepared for this interview. Tell the participant to imagine that this was what they received in today’s mail and that their name is Jaime Smith and that they live at 123 Main St Anytown USA. Ask them to talk out loud as they walk through what they would do with the stack. Participants should verbalize what they are doing at each step. Make sure the participant covers sorting, organizing, distributing among household members, opening, storing, saving, discarding, and responding to mail. It is important to try to get the participant to verbalize which aspects of each mailings are leading them to make various decisions about what to do with them so that we can apply these findings to the potential redesign of future NHES mailings. The focus is on the envelope/exterior of the mailing and not its contents (many of the envelopes are empty). The prepared stack should include one NHES mailing. Observe what the participant does with this envelope during the activity, but wait to probe about it until the sorting activity is complete. In other words, let the participant treat it like the other pieces of mail. You will only call attention to it at the start of the next activity.

Domain 3: Experience with NHES Mailings and Screener

Objective: To gather participants’ opinions and reactions to the survey packages that were mailed to them earlier this year. To the participant, this should be a seamless transition from that activity above. In practice, this is one activity that covers multiple domains.

NHES Materials Activity

Locate the NHES screener material from the stack of mail and ask them to open it and “process” it as they normally would. Observe how they open it. Once it is opened, ask participants their opinions and reactions to it. Please cover all aspects of the materials – the design, layout, color, wording, the return address, the mailing address, the stamp, the envelope size, etc. Probe on what would make participants more likely to open, read, and respond to the mailing.

Sample Questions

Do you remember receiving this? (Hand the respondent the mailings (see below for the list) one at a time in the order they would have received them. Allow them a moment to look at each mailing so that they can answer if they remember receiving it. After they say if they remember, follow the general line of questioning below).

Advance letter + envelope
Initial screener invitation letter + $5 cash + envelope
Screener pressure-sealed mailer/postcard
Second screener reminder letter + envelope
Third screener reminder letter + questionnaire + envelope
Fourth screener reminder letter + questionnaire + envelope

If the participant remembers: (Note to interviewers: Repeat this line of questioning for each of the NHES mailings that the participant remembers.)
- What do you remember about receiving this mailing?
- What did you do with this envelope when you received it earlier this year?
- (If opened the mailing): What do you remember about opening the letter? What stood out to you?
- (If didn’t open the mailing): What changes could we make that would make it more likely that you would open this mailing?
- Why did you throw it away (or keep it)?
- (If remembers a letter that mentions the web screener): Why did you (or did you not) access the web page listed in the letter?
  - (If accessed the web page): What do you remember about accessing the web page listed in the letter?
  - (If accessed the web page): What stopped you from filling out the survey? (Confidentiality concerns, technical issues, not interested in the topic?)
- (If remembers a mailing that includes a paper questionnaire): What do you remember about the paper screener that you received?
  - What stopped you from filling it out?
- What changes could we make that would make it more likely that you would respond to this survey?
If the participant doesn’t remember any of the materials:
- Based on what you see here, do you think you would complete this survey if you got it in the mail? Why or why not?
- What stands out to you as you look at it right now?
- If you received this in the mail today, how likely would you be to open it?
- What would make you open (or not open) it?
- What changes could we make that would make it more likely that you would open these mailings?
- What changes could we make that would make it more likely that you would respond to this survey?

Domain 4: Understanding of and Attitudes Toward Surveys

Objective: To gauge the participant’s level of knowledge of and comfort with surveys

Sample Questions

What do you think of when I use the word ‘survey’? What do you understand about what a survey is?
Have you ever done a survey before?
What are the benefits, if any, to participating in a survey? What, in your opinion, are the costs/risks/burdens to participating?
Tell me about the last time you completed a survey.
Do you enjoy completing surveys? What do you like/not like about doing them?
How do you like to take surveys? (probe on paper, computer, tablet, phone, in-person)
What are your experiences with getting money or gift cards for completing surveys?

Domain 5: Privacy Concerns

Objective: To understand how the participant views privacy and how this may affect their willingness to do surveys.

Sample Questions

What does the word “privacy” mean to you? Tell me the first few words that pop into your head.
How do you feel about information that is collected about you and how it is being used? How do you think it is used? Does it matter to you who is collecting the information? (Note: If the participant is unclear about the meaning of the question, you can ask about different settings. “For example, how do you feel about filling out forms at the doctor’s office? How do you feel about web sites collecting cookies?”)
How do you feel about the amount of control you have over your personal information?
What are some things you do to protect your personal information?
How do you feel about providing information on a survey about yourself or your family? Does it matter who is doing the survey? Does it matter what the question is asking? (If so): What kinds of questions make you uncomfortable?
Thinking about your daily life, when you have private information you would like to share with another trusted person or organization, how secure do you feel sharing it through the mail? Online? Via email? Over the phone? What makes you feel secure / insecure about information sharing?
For respondents with children:
- What rules, if any, do you have about your child/children using the Internet?
- What discussions have you had with your child/children about giving out information when s/he is/are online?

Domain 6: Attitudes toward government

Objective: To understand the participant’s attitudes toward the federal government and how this may affect their willingness to participate in government-sponsored surveys. (Try to avoid extended discussion of presidential and electoral politics).

Sample Questions

What role do you believe federal government agencies should play in collecting information about people through surveys? What information do they need?
Why does the U.S. continue to do a Census every ten years? What is that information for? How do you feel about being asked to complete a Census form?
What are your thoughts on the Department of Education? What role does it play in your life?
How important is voting to you?
How much impact do you think the federal government has on your daily life?

Domain 7: Education

Objective: To understand the participant’s opinions about education.

Sample Questions

What role do you think parents should take when it comes to their child’s schooling?
In general, what are your thoughts on today’s education system?
Are you happy with the school your child attends? Why/why not?
How involved are you with your child’s school?
How important/valuable was your own education experience to your daily life?

Domain 8: Time Use

Objective: To understand how people use their time day-to-day to get a sense of their time commitments and priorities.

Sample Questions

Can you briefly describe a typical weekday/weekend day?
What activities do you make sure you do every day?
How much time do different people in your family typically spend at home?
What do you do to relax?

Domain 9: Civic and Community Engagement

Objective: To explore the participant’s connection to the community, which has been shown to influence survey participation.

Sample Questions

Tell me about any news sources you follow.
To what extent to do you feel like an important part of your neighborhood or community?
- What makes you feel like an important part? What makes you feel like not an important part?
What types of groups are you involved in?
- What about other members of your family?

Domain 10: Household Information and Demographics

Objective: To understand the types of households and people with whom we are speaking and to determine if different subsets of nonrespondents have different types of concerns or reasons for nonresponse.

Lastly, I’ll ask you a few questions about your home and the people who live here. These are basic demographic questions like the kinds you are asked on the Census. [Note to interviewers: Please read all the questions in this section and ask them exactly as worded.]

If applicable: We may have already spoken about some of these topics, but I would just like to confirm this information now. If you know the participant has already shared this information with you, add “Just to confirm” before the question; otherwise, questions should be read exactly as shown.

Including yourself, how many people live here? _______________

[] Don’t know

[] Refused

(if only 1 person lives here, skip to question 2; otherwise, ask question 1a.)

How many people who live here are younger than 21 years old? _______________

[] Don’t know

[] Refused

What is your current age? _______________

[] Refused

What is your gender? (Do not read response options.)

[] Male

[] Female

[] Other, specify: ______________________

[] Refused

Are you of Hispanic, Latino, or Spanish origin?

[] Yes

[] No

[] Other, specify: ______________________

[] Refused

What is your race? (Do not read response options. Select all that apply.)

[] American Indian or Alaska Native

[] Asian

[] Black or African American

[] Native Hawaiian or other Pacific Islander

[] White

[] Refused

Which language is spoken most often by the adults who live here? (Do not read response options.)

[] English

[] Spanish

[] Other, specify: ______________________

[] Refused

Do you have Internet access at home…?

On a computer?	[] Yes	[] No	[] Don’t know	[] Refused
How about on a tablet?	[] Yes	[] No	[] Don’t know	[] Refused
How about on a cell phone?	[] Yes	[] No	[] Don’t know	[] Refused

What is the highest grade or level of school that you have completed? (Do not read response options.)

[] 8^th grade or less

[] High school, but no diploma

[] High school diploma or equivalent (GED)

[] Vocational diploma after high school

[] Some college, but no degree

[] Associate’s degree (AA, AS)

[] Bachelor’s degree (BA, BS)

[] Some graduate or professional education, but no degree

[] Master’s degree (MA, MS)

[] Doctorate degree (PhD, EdD)

[] Professional degree (MD, DDS, JD, LLB)

[] Refused

Are you currently employed and working for pay?

[] Yes

[] No

[] Refused

Are you a full-time or part-time student?

[] Yes

[] No

[] Refused

Based on this sheet [Hand respondent the income sheet], which category best fits the total income of all persons in your home over the past 12 months? Include your own income. Include money from jobs or other earnings, pensions, interest, rent, Social Security payments, and so on. (Do not read response options. If respondent is struggling, let them know they can make their best guess.)

[] $0 to $10,000

[] $10,001 to $20,000

[] $20,001 to $30,000

[] $30,001 to $40,000

[] $40,001 to $50,000

[] $50,001 to $60,000

[] $60,001 to $75,000

[] $75,001 to $100,000

[] $100,001 to $150,000

[] $150,001 to $200,000

[] $200,001 to $250,000

[] $250,001 or more

[] Don’t know

[] Refused

B.1.10. Income Show Card

A	$0 to $10,000
B	$10,001 to $20,000
C	$20,001 to $30,000
D	$30,001 to $40,000
E	$40,001 to $50,000
F	$50,001 to $60,000
G		$60,001 to $75,000
H		$75,001 to $100,000
I		$100,001 to $150,000
J		$150,001 to $200,000
K		$200,001 to $250,000
L		$250,001 or more

B.1.11. Incentive Receipt Acknowledgment Form

I, ________________________ (Participant Name), certify that I have received the following incentive from American Institutes for Research as a thank you for participating in an interview.

Type	Amount
Cash	$120

Participant Signature __________________________ Date ________________________

Interviewer Signature __________________________ Date ________________________

B.2. Spanish Materials

Attachment B.2. provides the Spanish versions of these materials that will be used to interact with households sampled for the qualitative interview.

Invitation Letter
First Reminder Postcard
Second Reminder Postcard
Phone Follow-up Recruitment Script
“Sorry We Missed You!” Card
In-Person Recruitment Script
Commonly Asked Questions
Interview Consent Form
Interview Protocol
Income Show Card
Incentive Receipt Acknowledgment Form

B.2.1. Spanish Invitation Letter

<Mailing_add1>

<Mailing_add2>

Estimado hogar de <City>:

El Centro Nacional para Estadísticas de la Educación (NCES) y American Institutes for Research (AIR) realizan un estudio importante en su localidad.

Este estudio nos ayudará a comprender en qué forma la investigación que se lleva a cabo en el Centro NCES sería más útil y beneficiosa. Nos gustaría obtener sus respuestas sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, organizar el correo postal que recibe, y cómo se siente con la idea de participar en estudios de investigación.
Esta conversación voluntaria dura alrededor de 90 minutos y se llevará a cabo en su hogar, en un horario que sea conveniente para usted.

Sabemos que es una persona ocupada y hemos incluido $5 como muestra de nuestro agradecimiento por su tiempo. Dado que su participación es esencial, obtendrá $120 en efectivo como agradecimiento por compartir sus opiniones y experiencias con nosotros.

Para determinar si reúne los requisitos para participar, realice alguna de las siguientes acciones:

Llame a nuestro número gratuito 1-XXX-XXX-XXXX
Comuníquese con nosotros mediante el correo electrónico [email protected]

Agradecemos de antemano su participación. Esperamos tener más noticias suyas.

Saludos cordiales,

[SIGNATURE]

James Lynn Woodworth

Comisionado del Centro Nacional para Estadísticas de la Educación

Authorization and Confidentiality of Information

American Institutes for Research administra este studio voluntario en nombre del Centro Nacional para Estadísticas de la Educación (NCES por sus siglas en Inglés). No hay sanciones si decide no participar en este estudio. Toda la información que brinde puede ser utilizada solo con fines estadísticos y no puede divulgarse ni usarse en forma identificable para ningún otro fin, excepto cuando lo exija la ley (20 U.S.C. §9573 y 6 U.S.C. §151). NCES, dentro del Departamento de Educación, está autorizado a realizar este estudio por medio de la Reforma a la Ley de las Ciencias de la Educación del 2002 (ESRA 2002; titulo 20; sección 9543).

B.2.2. Spanish First Reminder Postcard

¿Le gustaría recibir $120

por compartir sus opiniones?

El Centro Nacional para Estadísticas de la Educación (NCES) y American Institutes for Research (AIR) realizan un estudio importante en su localidad. Este estudio nos ayudará a comprender en qué forma la investigación que se lleva a cabo en el NCES sería más útil y beneficiosa.

Dado que su participación es esencial, obtendrá $120 en efectivo como agradecimiento por compartir sus opiniones y experiencias con nosotros.
Esta conversación voluntaria dura alrededor de 90 minutos y se llevará a cabo en su hogar, en un horario que sea conveniente para usted.
Nos gustaría obtener sus respuestas sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, organizar el correo postal que recibe, y cómo se siente con la idea de participar en estudios de investigación.
Un investigador de AIR pasará por su casa para invitarlo a participar en las próximas semanas.
Para averiguar rápidamente si cumple con los requisitos para participar, llame a nuestro número gratuito 1-XXX-XXX-XXXX o envíenos un correo electrónico [email protected].

Agradecemos de antemano su participación. ¡Esperamos tener más noticias suyas!

B.2.3. Spanish Second Reminder Postcard

¡Platique con nosotros para ayudar a que

los estudios de investigación sean más útil!

El Centro Nacional para Estadísticas de la Educación (NCES) y American Institutes for Research (AIR) realizan un estudio importante en su localidad. Este estudio nos ayudará a comprender en qué forma la investigación que se lleva a cabo en el NCES sería más útil y beneficiosa.

Un investigador de AIR pasará por su casa para invitarlo a participar en las próximas semanas.
Para averiguar rápidamente si cumple con los requisitos para participar, llame a nuestro número gratuito 1-XXX-XXX-XXX o envíenos un correo electrónico al [email protected].

Nos gustaría recibir sus respuestas sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, organizar el correo postal que recibe y como se siente con la idea de participar en estudios de investigación.
Dado que su participación es esencial, obtendrá $120 en efectivo como agradecimiento por compartir sus opiniones y experiencias con nosotros.
Esta conversación voluntaria dura alrededor de 90 minutos y se llevará a cabo en su hogar, en un horario que sea conveniente para usted.

Agradecemos de antemano su participación. ¡Esperamos tener más noticias suyas!

B.2.4. Spanish Phone Follow-up Recruitment Script

IF NO ONE ANSWERS THE PHONE, LEAVE THE FOLLOWING MESSAGE:

Hola, soy [YOUR NAME] y le llamo de American Institutes for Research en nombre del Centro Nacional de Estadísticas de la Educación. Estamos realizando un estudio importante en su localidad. Nos gustaría tener una conversación con usted para obtener sus opiniones sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, ordenar el correo postal que recibe. Dado que su participación es esencial, obtendrá $120 en efectivo como agradecimiento por platicar con nosotros. Si desea participar, llame a nuestro número gratuito 1-xxx-xxx-xxxx.

IF SOMEONE ANSWERS THE PHONE:

Hola, soy [YOUR NAME] y le llamo de American Institutes for Research en nombre del Centro Nacional de Estadísticas de la Educación.. ¿Estoy hablando con un residente de [ADDRESS]?

If yes, continue with the next paragraph.
If no, say: Disculpe. Me equivoqué de número. Agradezco su tiempo. Que tenga un excelente día. End phone call.

Llamo como seguimiento a una carta que se envió recientemente a su hogar. El Centro Nacional para Estadísticas de la Educación y American Institutes for Research realizan un estudio importante en su localidad que nos ayudará a comprender en qué forma nuestra investigación sería más útil y beneficiosa. Nos gustaría tener una conversación con usted para obtener sus opiniones sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, ordenar el correo postal que recibe. Si completa la conversación de 90 minutos, le daremos $120 en efectivo como muestra de nuestro agradecimiento.

¿Está interesado en esta oportunidad?

If yes, continue with the next paragraph.
If no, try to address their concerns. If still not interested, say: No hay ningún problema. Gracias por su tiempo. Que tenga un excelente día.
End phone call.

Excelente. Primero, debo hacerle algunas preguntas para garantizar que cumple con los requisitos para participar.

¿Vivió o permaneció en esta dirección la mayor parte del tiempo entre enero y abril del 2019?

If yes, continue with next screener item.
If no, say: Lamentablemente, solo podemos reunirnos con personas que vivieron o permanecieron en esta dirección la mayor parte del tiempo entre enero y abril del 2019. ¿Hay alguien más que haya vivido o permanecido en ese lugar la mayor parte de ese periodo?
If yes, ask if you can speak with them about participating.

When that person is on the phone, go back to the study introduction paragraph and then restart the screener.
If the person is not available right now, ask when would be a good time for you to call back, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.

If no one else lived here during that time period, thank them for their time, end phone call and do not call again.

Excelente. ¿Puede confirmar su edad?

If 18 or older, continue with next screener item.
If 17 or younger, say: Lamentablemente solo pueden participar personas adultas. ¿Hay algún adulto en su hogar con quién pueda hablar?

When the adult is on the phone, go back to the study introduction paragraph and then restart the screener.
If an adult is not available right now, ask when would be a good time for you to call back to speak with one, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.

Excelente, gracias. ¿ Participa usted en la organización de la correspondencia que recibe en su hogar, por ejemplo, reúne, revisa o responde el correo postal que recibe?

If yes, continue with next screener item.

If no, say: Lamentablemente, solo podemos reunirnos con alguna de las personas que organizan el correo postal que recibe. ¿Podría comunicarme con alguna de las personas que realizan esa tarea?

When that person is on the phone, go back to the study introduction paragraph and then restart the screener.
If the person is not available right now, ask when would be a good time for you to call back, thank them for their time, and then end the call. Try to get the other person’s first name to facilitate the return call.

¡Excelente, gracias! Usted reúne las condiciones necesarias para participar en este estudio. ¿Qué días de [esta/la próxima] semana serían convenientes para que se reúna con nosotros?

Looking at when interview team is available in that area, work with participant to schedule the interview. Our preference is to do interviews sooner than later.

Once interview date/time are decided, record them and say: Tenemos una fecha programada para usted el [DATE] a las [TIME]. Usted se reunirá con [INTERVIEWER NAME].

Nos gustaría enviarle un mensaje de confirmación con la fecha y hora de su reunión. ¿Prefiere que le enviemos un mensaje de texto o un correo electrónico?

If willing to receive confirmation message: ¿Cuál es el mejor medio de comunicación [número de teléfono/dirección de correo electrónico] para ponernos en contacto con usted?
If not willing to receive confirmation message, reassure them that this information will only be used for communicating with them about the interview. If still not willing, say: Está bien, no hay ningún problema.

También quisiéramos comunicarnos con usted un día antes de la reunión para confirmar que este horario sigue siendo conveniente para usted. ¿Prefiere que le enviemos un mensaje de texto o un correo electrónico?

If willing to receive reminder message and request a different mode than for the confirmation message: ¿Cuál es el mejor medio de comunicación [número de teléfono/dirección de correo electrónico] para ponernos en contacto con usted?
If not willing to receive reminder message, reassure them that this information will only be used for communicating with them about the interview. If still not willing, say: Está bien, no hay ningún problema.

¡Agradecemos su participación! Si surge algún inconveniente a partir de este momento, llame a nuestro número gratuito 1-XXX-XXX-XXXX o envíenos un correo electrónico [email protected].

Esperamos verlo pronto. Hasta luego. End call.

B.2.5. Spanish “Sorry We Missed You!” Card

Disculpe, ¡no le encontramos en casa!

El Centro Nacional para Estadísticas de la Educación (NCES por sus siglas en Inglés) y American Institutes for Research (AIR) realizan un estudio importante en su localidad. Hemos pasado por su casa para invitarle a participar.

Este estudio nos ayudará a comprender en qué forma la investigación que se lleva a cabo en el NCES sería más útil y beneficiosa. Nos gustaría obtener sus respuestas sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, organizar el correo postal que recibe, y cómo se siente con la idea de participar en estudios de investigación.
Si se reúne con nosotros, recibirá $120.
Esta conversación voluntaria dura alrededor de 90 minutos y se llevará a cabo en su hogar, en un horario que sea conveniente para usted.

¡Póngase en contacto con nosotros al 1-xxx-xxx-xxxx o [email protected] para determinar si cumple con los requisitos!

B.2.6. Spanish In-Person Recruitment Script

INTRODUCTION

If the person who answers the door is clearly a child, ask about speaking to an adult in the house. Otherwise, say:

Hola, mi nombre es [YOUR NAME], y provengo del American Institutes for Research. Recientemente enviamos una carta a su domicilio que trata sobre la importancia del estudio que realizamos en colaboración con el Centro Nacional de Estadísticas de la Educación. Nos gustaría tener una conversación con usted para obtener sus opiniones sobre distintos temas, entre ellos, cómo le gusta invertir su tiempo y cómo realiza labores cotidianas, por ejemplo, organizar el correo postal que recibe y como se siente con la idea de participar en estudios de investigación. La conversación dura 90 minutos y le daremos $120 en efectivo como agradecimiento por su tiempo. ¿Está interesado en esta oportunidad?

If yes, continue with the next section.
If no, try to address their concerns. If still not interested, say: No hay ningún problema, lo entiendo. ¿Hay alguien más en este hogar que esté disponible para hablar con nosotros?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No hay ningún problema. Podemos regresar en algún momento que sea más conveniente. ¿Cuál sería un buen momento para que regresemos? [record date/time, and if possible the other household member’s first name to facilitate the return visit] Le dejo una tarjeta que contiene más información sobre el estudio y que puede darles. Además, incluye nuestra información de contacto para que la utilicen en caso de que deseen comunicarse con nosotros lo más pronto posible para participar. [hand Sorry We Missed You card to household member] ¡Agradezco su tiempo! Que tenga un excelente día.
- If no one else lives here or not interested in you speaking with anyone else, say: Entiendo. En vez de ello, ¿estaría dispuesto a responder algunas preguntas rápidas? Esto tarda menos de cinco minutos.
  - If no: No hay ningún problema. Agradezco su tiempo. Que tenga un excelente día. (end interview)
  - If yes, go to Short Interview with Full-Interview Refusals: Excelente, gracias.

SHORT INTERVIEW WITH FULL-INTERVIEW REFUSALS

Primero, ¿podría confirmar que tiene al menos 18 años?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
¿Podría confirmar que hemos llegado a [STREET ADDRESS]?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
¿Vivió o permaneció en esta dirección la mayor parte del tiempo entre enero y abril del 2019?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
¿Organiza usted el correo de su hogar, por ejemplo, reúne, revisa o responde el correo postal que recibe?
- Yes (continue)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)
¿Ha recibido algún correo proveniente del Departamento de Comercio y la Oficina del Censo desde el 1 de enero del 2019?
- Yes (continue)
- No (continue)
- Don’t know (continue)
- Refused (continue)
¿Y de la Encuesta Nacional en Hogares sobre la Educación? ¿Ha recibido algún correo como este desde el 1 de enero del 2019?
- Yes (ask QG)
- No (go to QH)
- Don’t know (go to QH)
- Refused (go to QH)
¿Estaría dispuesto a compartir algunas razones acerca de por qué no contestó la encuesta? (Allow respondent a moment to think.)
¿Alguna de las personas que vive aquí tiene 20 años o menos?
- Yes (go to end interview)
- No (go to end interview)
- Don’t know (go to end interview)
- Refused (go to end interview)

End interview: Estas son todas las preguntas que quería hacerle. Agradezco su tiempo. ¡Que tenga un excelente día!

SCREENER

Once the interested household member is at the door:

¡Excelente! Pero primero, debo hacerle algunas preguntas para garantizar que cumple con los requisitos para participar.

¿Podría confirmar que hemos llegado a [STREET ADDRESS]?

If yes, continue with next screener item.
If no, say: Lamentablemente, solo podemos reunirnos con alguien que viva en [STREET ADDRESS]. Disculpe, me equivoqué de casa. Agradezco su tiempo. Que tenga un excelente día. End conversation.

¿Vivió o permaneció en esta dirección la mayor parte del tiempo entre enero y abril del 2019?

If yes, continue with next screener item.
If no, say: Lamentablemente, solo podemos reunirnos con personas que vivieron o permanecieron en esta dirección la mayor parte del tiempo entre enero y abril del 2019. ¿Hay alguien más que haya vivido o permanecido en ese lugar la mayor parte de ese periodo?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No hay ningún problema. Podemos regresar en algún momento que sea más conveniente. ¿Cuál sería un buen momento para que regresemos? [record date/time, and if possible the other household member’s first name to facilitate return visit] Le dejo una tarjeta que contiene más información sobre el estudio y que puede darles. Además, incluye nuestra información de contacto para que la utilicen en caso de que deseen comunicarse con nosotros lo más pronto posible para participar. [hand Sorry We Missed You card to household member] ¡Agradezco su tiempo! Que tenga un excelente día.
- If no one else lives here or not interested in you speaking with anyone else, say: No hay ningún problema. Agradezco su tiempo. Que tenga un excelente día.

Excelente. ¿Puede confirmar su edad? [do not ask this if respondent is clearly much older than 18, but if there is any uncertainty make sure you confirm their age]

If 18 or older, continue with next screener item.
If 17 or younger, say: Lamentablemente solo pueden participar personas adultas. ¿Hay un adulto en su hogar con quien pueda hablar?
- If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No hay ningún problema. Podemos regresar en algún momento que sea más conveniente. ¿Cuál sería un buen momento para que regresemos? [record date/time, and if possible the other household member’s first name] Le dejo una tarjeta que contiene más información sobre el estudio y que puede darles. Además, incluye nuestra información de contacto para que la utilicen en caso de que deseen comunicarse con nosotros rápidamente para participar. [hand Sorry We Missed You card to household member] ¡Agradezco su tiempo! Que tenga un excelente día.
- If no one else lives here or not interested in you speaking with anyone else, say: No hay ningún problema. Agradezco su tiempo. Que tenga un excelente día.

Una última pregunta, ¿ participa usted en la organización de la correspondencia que recibe en su hogar, por ejemplo, reúne, revisa o responde el correo postal que recibe?

If yes, continue with next screener item.
If no, say: Lamentablemente, necesitamos hablar con alguna de las personas que organizan el correo postal que recibe en su hogar. ¿Podría hablar con alguna de las personas que realizan esa tarea?
If yes and available, repeat the introduction once that person comes to the door.
- If yes but not available right now, say: No hay ningún problema. Podemos regresar en algún momento que sea más conveniente. ¿Cuál sería un buen momento para que regresemos? [record date/time, and if possible the other household member’s first name to facilitate return visit] Le dejo una tarjeta que contiene más información sobre el estudio y que puede darles. Además, incluye nuestra información de contacto para que la utilicen en caso de que deseen comunicarse con nosotros lo más pronto posible para participar. [hand Sorry We Missed You card to household member] ¡Agradezco su tiempo! Que tenga un excelente día.
- If no one else lives here or not interested in you speaking with anyone else, say: No hay ningún problema. Agradezco su tiempo. Que tenga un excelente día.

SCHEDULING

¡Excelente, gracias! Usted reúne las condiciones necesarias para participar en este estudio. Podemos reunirnos en este momento, si es conveniente para usted. La conversación dura cerca de 90 minutos.

If they are available now, start the interview.
If they are not available now, say: No hay ningún problema. Podemos regresar en algún momento que sea más conveniente para usted. ¿Qué días de esta semana serían convenientes para que se reúna con nosotros?
- Based on your availability, work with the respondent to find a time for the interview.
- Record the date/time for yourself, and on an Appointment card, and say: Tenemos una fecha programada para usted el [DATE] a las [TIME]. Le dejo una tarjeta de confirmación de citas para que la conserve como recordatorio.
- Hand them the card and then say: ¿Podría darme su nombre para agregarlo en mi calendario de citas?
- Record name, then say: Nos gustaría comunicarnos con usted un día antes de la conversación para confirmar que este horario aún es conveniente para usted. ¿Prefiere que le enviemos un mensaje de texto o un correo electrónico?
- Once they respond, say: ¿Cuál es el mejor medio de comunicación [número de teléfono/dirección de correo electrónico] para ponernos en contacto con usted? If they express concern about providing this information, let them know this will only be used to remind them about the appointment and confirm they are still available.
- Record answer for your records and say: ¡Agradecemos su participación! Si surge algún inconveniente a partir de este momento, llame a nuestro número gratuito o envíenos un correo electrónico con la información que se encuentra en su tarjeta de citas. Espero verlo pronto. Que tenga un excelente día.

B.2.7. Spanish Commonly Asked Questions

Preguntas frecuentes

¿De qué se trata el estudio? ¿De qué manera utilizarán mis respuestas?

El objetivo de este estudio es conocer cómo podríamos mejorar la investigación que realizó el Centro Nacional de Estadísticas de la Educación, el cual forma parte del Departamento de Educación. Nos gustaría escuchar sus opiniones y experiencias sobre diversos temas. Por ejemplo, nos gustaría saber qué tipo de cosas son importantes para usted en su vida cotidiana. También queremos saber cómo realiza ciertas actividades como recibir y revisar el correo postal que recibe en su casa. Por último, deseamos recibir sus comentarios sobre algunos de los materiales que proporcionamos en nuestra investigación, ya que esta información nos ayudará a mejorar la Encuesta Nacional en Hogares sobre la Educación (NHES, por sus siglas en inglés) y asegúrese de que sean relevantes para todos los hogares en los Estados Unidos.

¿Quién realizando este estudio?

El Departamento de Educación, específicamente el Centro Nacional para Estadísticas de la Educación (NCES por sus siglas en inglés), está realizando este estudio. Puede obtener más información sobre el NCES en: www.nces.ed.gov

American Institutes for Research (AIR, por sus siglas en inglés), una agencia sin fines de lucro que se dedica a la investigación en ciencias sociales y sobre el comportamiento, es la agencia que realiza este estudio en nombre del NCES. Puede obtener más información sobre AIR en: www.air.org.

¿Cuánto tiempo durará la conversación?

La conversación durará alrededor de 90 minutos.

¿Dónde se realizará la conversación?

La conversación se llevará a cabo en su hogar, en un horario que sea conveniente para usted. Nos gustaría hablar con usted en su hogar, porque queremos que comparta sus experiencias y opiniones en un ambiente que sea cómodo y familiar para usted.

¿Quién sabrá si participo en el estudio?

Únicamente el equipo de AIR que se encarga del estudio sabrá quiénes participaron en el estudio. La información que recopilemos durante nuestra conversación se almacenará en los servidores seguros de las computadoras de AIR. No relacionaremos la información que nos proporcione con nada que pueda identificarle, como su nombre o dirección.

¿Qué recibiré a cambio si me reúno con ustedes?

Recibirá $120 en efectivo como agradecimiento por su participación.

¿Con quién puedo comunicarme si tengo alguna duda sobre la participación en este estudio?

Puede llamar a nuestro número gratuito 1-XXX-XXX-XXXX o enviarnos un correo electrónico a la siguiente dirección [email protected].

B.2.8. Spanish Informed Consent Form

F ormulario de consentimiento para la participación en este estudio

El Centro Nacional para Estadísticas de la Educación (NCES, por sus siglas en inglés) del Departamento de Educación de los Estados Unidos realizando este estudio para comprender en qué forma su investigación sería más útil y beneficiosa. El NCES contrató a American Institutes for Research (AIR) para realizar este estudio.

Le pedimos que participe en una conversación de 90 minutos, en la cual le pediremos su opinión sobre diversos temas, por ejemplo, cómo le gusta utilizar su tiempo, cómo realiza actividades cotidianas como ordenar el correo postal que recibe, y cómo se siente al participar en estudios de investigación.
Su participación aportará información valiosa que nos ayudará a mejorar las encuestas que realiza el NCES, sin embargo, su participación es completamente voluntaria y no habrá ninguna penalización si decide no participar en este estudio. Puede detener la entrevista en cualquier momento sin recibir ninguna penalización.
La información que comparte con nosotros se combinará con las respuestas proporcionadas por otros participantes a través de un breve informe en el que usted o cualquier otra persona no será identificada individualmente. Toda la información que proporcione se utilizará solamente con fines estadísticos y no podrá divulgarse ni usarse en forma identificable para ningún otro propósito, excepto en la forma en que lo exija la ley (20 U.S.C. §9573 and 6 U.S.C. §151).
Si tiene alguna duda o inquietud relacionada con su participación, comuníquese con Sarah Grady en el Centro Nacional para Estadísticas de la Educación (NCES). Puede comunicarse con ella al 202-245-7063 o [email protected].
Nos gustaría grabar esta conversación para garantizar que recopilamos con precisión la información que nos proporcione. Sin embargo, si no está de acuerdo, no grabaremos la conversación. Si acepta que lo grabemos, únicamente los empleados de AIR y NCES que trabajen en este estudio tendrán autorización para escuchar la grabación. Todas las grabaciones se destruirán cuando finalice el estudio. ¿Nos autoriza a grabar la conversación?

 SÍ  NO

Al firmar y escribir su nombre abajo, acepta participar en este estudio.

Firma: ____________________________________________ Fecha: ___________________

Nombre escrito en letra de molde: _____________________________________

B.2.9. Spanish Interview Protocol

Before you begin the interview, give the informed consent document to the participant, answer any questions they may have, and begin recording with their permission. Once they sign the informed consent, put it away in a folder.
Start with rapport building through basic questions about household make-up (Domain 1).
Next, proceed with the Experience of Mail Delivery and Experience of NHES Materials and Screener domains (Domains 2 and 3).
After this, you should rely on your training to decide what is the best course forward that will yield the most informative interview; however, you should at least touch on all of the domains listed in this document.
When there are about 5 minutes left in the interview, ask the questions in the demographics section.
Finally, hand the participant their incentive and make sure they sign the incentive receipt form.

Mail Delivery: In the first, you will offer the participant a stack of mail and the participant will talk aloud about the process of sorting, organizing, and making determinations about how they would handle each piece if they received it, and will show you where and how that process occurs in the home.
NHES Materials and Screener: In the second, you will hand the participant NHES materials that were mailed to them earlier this year and the participant will talk aloud about his or her reactions to and opinions about the mailings. Finally, you will show the participant the NHES screener questionnaire and talk through their opinions and reactions.

INTERVIEW DOMAINS

Domain 1: Rapport Building and Household Make up

Objective: To understand key aspects of the participant’s household composition and build rapport.

Sample Questions

Cuénteme un poco sobre las personas que viven con usted.
Veo que tiene tres hijos. ¿Qué tipo de actividades les gusta hacer?
¿Qué le atrajo a este barrio?
Noté que hay X (such as a security sign, holiday décor, etc.) fuera de su casa. ¿Puede contarme un poco más sobre esto?

Domain 2: Experience with Mail Delivery

Sample Questions

¿Cómo recibe su correo (entrega a domicilio en buzón exterior, ranura para cartas, apartado postal)? ¿Puede mostrarme? (if nearby)
¿Qué tipo de correo recibe normalmente?
¿Cómo sabe cuándo recibe su correo?
¿Con qué frecuencia revisa sus correo?
¿Cómo envía su correo (camina hasta la oficina de correos, lo lleva al edificio de correos, por medio de un buzón de correos, etc.)? ¿Puede mostrarme? (if different from delivery)
¿Cómo decide qué correo abrirá?
¿Cómo describe el tipo de correo que conservará cuando clasifica el correo sin abrir?
¿Cómo describe el tipo de correo que desechará?
¿Qué le hace desechar un correo antes de abrirlo?
¿Qué le hace desechar un correo antes de leerlo?
¿Qué hace con el correo una vez que lo ha leído?
¿Cómo hace un seguimiento del correo al que necesita enviar/responder?
¿Con qué frecuencia su correo se mezcla con el correo que va dirigido a otras personas?

Mail Interaction Activity

Domain 3: Experience with NHES Mailings and Screener

NHES Materials Activity

Sample Questions

¿Recuerda haber recibido lo siguiente? (Hand the respondent the mailings (see below for the list) one at a time in the order they would have received them. Allow them a moment to look at each mailing so that they can answer if they remember receiving it. After they say if they remember, follow the general line of questioning below).

Advance letter + envelope
Initial screener invitation letter + $5 cash + envelope
Screener pressure-sealed mailer/postcard
Second screener reminder letter + envelope
Third screener reminder letter + questionnaire + envelope
Fourth screener reminder letter + questionnaire + envelope

If the participant remembers: (Note to interviewers: Repeat this line of questioning for each of the NHES mailings that the participant remembers.)
- ¿Qué recuerda sobre la recepción de este correo?
- ¿Qué hizo con el sobre cuando lo recibió a principios de este año?
- (If opened the mailing): ¿Qué recuerda acerca de abrir la carta? ¿Qué fue lo que más le llamó la atención?
- (If didn’t open the mailing): ¿Qué cambios podríamos hacer para que sea más probable que usted abra este correo?
- ¿Por qué lo tiró (o lo guardó)?
- (If remembers a letter that mentions the web screener): ¿Por qué visitó (o no) la página web que se menciona en la carta?
  - (If accessed the web page): ¿Qué recuerda sobre su visita a la página web que se menciona en la carta?
  - (If accessed the web page): ¿Qué le impidió llenar la encuesta? (Confidentiality concerns, technical issues, not interested in the topic?)
- (If remembers a mailing that includes a paper questionnaire): ¿Qué recuerda sobre los documentos para la evaluación inicial que recibió?
  - ¿Qué impidió que los llenara?
- ¿Qué cambios podríamos hacer para que sea más probable que usted responda a esa encuesta?
If the participant doesn’t remember any of the materials:
- Con base en lo que ve aquí, ¿cree que completaría esta encuesta si la recibiera por correo? ¿Por qué sí o por qué no?
- ¿Qué es lo que más le llama la atención cuando la ve en este momento?
- Si recibiera esto hoy por correo, ¿qué probabilidades habría de que usted lo abriera?
- ¿Qué le haría abrirlo (o no abrirlo)?
- ¿Qué cambios podríamos hacer para que sea más probable que usted abra estos correos?
- ¿Qué cambios podríamos hacer para que sea más probable que usted responda a esa encuesta?

Domain 4: Understanding of and Attitudes Toward Surveys

Objective: To gauge the participant’s level of knowledge of and comfort with surveys

Sample Questions

¿Qué piensa cuando utilizo la palabra "encuesta"? ¿Qué entiende acerca de lo que es una encuesta?
¿Ha hecho antes alguna encuesta?
¿Cuáles son los beneficios, si los hay, de participar en una encuesta? ¿En su opinión, cuáles son los costos/riesgos/responsabilidades de participar?
Hábleme sobre la última vez que completó una encuesta.
¿Le gusta completar encuestas? ¿Qué le gusta/no le gusta de completarlas?
¿Cómo le gusta completar las encuestas? (probe on papel, computadora, tableta, teléfono, en persona)
¿Cuáles son sus experiencias con respecto a la obtención de dinero o tarjetas de regalo por completar encuestas?

Domain 5: Privacy Concerns

Objective: To understand how the participant views privacy and how this may affect their willingness to do surveys.

Sample Questions

¿Qué significa para usted la palabra “privacidad”? Diga las primeras palabras que le vengan a la mente.
¿Cómo se siente acerca de la información que se recopila sobre usted y sobre la forma en que se utiliza? ¿Cómo cree que se utiliza? ¿Es importante para usted quién recopila esa información? (Note: If the participant is unclear about the meaning of the question, you can ask about different settings. “For example, how do you feel about filling out forms at the doctor’s office? How do you feel about web sites collecting cookies?”)
¿Cómo se siente acerca del grado de control que tiene sobre su información personal?
¿Qué acciones lleva a cabo para proteger su información personal?
¿Cómo se siente al proporcionar información sobre usted o su familia en una encuesta? ¿Es importante para usted saber quién está haciendo la encuesta? ¿Es importante para usted lo que significa la pregunta? (If so): ¿Qué tipo de preguntas le hacen sentir incómodo?
Pensando en su vida cotidiana, cuando posee información privada que le gustaría compartir con otra persona u organización de confianza, ¿qué tan seguro se siente si la comparte por medio del correo? ¿En línea? ¿Por correo electrónico? ¿Por teléfono? ¿Qué le hace sentir seguro / inseguro sobre el intercambio de información?
For respondents with children:
- ¿Qué reglas, si las hay, aplica al uso de Internet por parte de sus hijos?
- ¿Qué conversaciones ha tenido con sus hijos acerca de dar información cuando están en línea?

Domain 6: Attitudes toward government

Sample Questions

¿Qué papel cree que deberían desempeñar las agencias del gobierno federal con respecto a la recopilación de información sobre las personas por medio de encuestas? ¿Qué información necesitan?
¿Por qué en los Estados Unidos se sigue haciendo un censo cada diez años? ¿Para qué sirve esa información? ¿Cómo se siente cuando le piden que llene un formulario del censo?
¿Qué opina sobre el Departamento de Educación? ¿Qué papel juega en su vida?
Para usted, ¿qué tan importante es votar?
¿Qué impacto considera que tiene el gobierno federal en su vida cotidiana?

Domain 7: Education

Objective: To understand the participant’s opinions about education.

Sample Questions

¿Qué papel considera que deben desempeñar los padres cuando se trata de la educación de sus hijos?
En general, ¿cuál es su opinión sobre el sistema educativo actual?
¿Está contento con la escuela a la que asiste su hijo? ¿Por qué sí/por qué no?
¿Qué tan involucrado está con la escuela en la que estudia su hijo?
¿Qué tan importante/valiosa fue su propia experiencia educativa para su vida cotidiana?

Domain 8: Time Use

Objective: To understand how people use their time day-to-day to get a sense of their time commitments and priorities.

Sample Questions

¿Puede describir brevemente un día típico de la semana/un día del fin de semana?
¿Qué actividades se asegura de hacer todos los días?
¿Cuánto tiempo pasan en casa los miembros de su familia?
¿Qué actividades hace para relajarse?

Domain 9: Civic and Community Engagement

Objective: To explore the participant’s connection to the community, which has been shown to influence survey participation.

Sample Questions

Hábleme de cualquier fuente de noticias que siga.
¿Hasta qué punto siente que es una parte importante de su vecindario o comunidad?
- ¿Qué le hace sentir que es una parte importante? ¿Qué le hace sentir que no es una parte importante?
¿Con qué tipos de grupos se involucra?
- ¿Y los demás miembros de su familia?

Domain 10: Household Information and Demographics

Para terminar, le haré algunas preguntas sobre su casa y la gente que vive ahí. Estas son preguntas demográficas básicas, similares las que se hacen durante un censo. [Note to interviewers: Please read all the questions in this section and ask them exactly as worded.]

If applicable: Es posible que ya hayamos hablado de algunos de estos temas, pero quisiera confirmar esta información. If you know the participant has already shared this information with you, add “Just to confirm” before the question; otherwise, questions should be read exactly as shown.

Incluyéndole, ¿cuántas personas viven aquí? _______________

[] Don’t know

[] Refused

(if only 1 person lives here, skip to question 2; otherwise, ask question 1a.)

¿Cuántas de las personas que viven aquí son menores de 21 años? _______________

[] Don’t know

[] Refused

¿Cuál es su edad en este momento? _______________

[] Refused

¿Cuál es su género? (Do not read response options.)

[] Masculino

[] Femenino

[] Otro, especifiqe: ______________________

[] Refused

¿Es usted de origen hispano, latino o español?

[] Sí

[] No

[]Otro, especifiqe: ______________________

[] Refused

¿Cuál es su raza? (Do not read response options. Select all that apply.)

[] India americana o nativa de Alaska

[] Asiática

[] Negra o africana americana

[] Nativa de Hawái u otra de las islas

del Pacífico

[] Blanca

[] Refused

¿Cuál es el idioma que más hablan los adultos que viven aquí? (Do not read response options.)

[] Inglés

[] Español

[] Otro, especifiqe: ______________________

[] Refused

¿Tiene acceso a Internet en casa…?

¿En una computadora?	[] Sí	[] No	[] Don’t know	[] Refused
¿En una tableta?	[] Sí	[] No	[] Don’t know	[] Refused
¿En un teléfono celular?	[] Sí	[] No	[] Don’t know	[] Refused

¿Cuál es el grado o nivel más alto de escolaridad que ha completado? (Do not read response options.)

[] Grado 8 o menos

[] Escuela preparatoria/secundaria (high school), pero no se graduó

[] Diploma de escuela preparatoria/secundaria (high school) o un equivalente (GED)

[] Se graduó de escuela vocacional después de la escuela preparatoria/secundaria (high school)

[] Algunos cursos universitarios, sin sacar título

[] Grado asociado (AA, AS)

[] Título de licenciatura (BA, BS)

[] Algunos cursos universitarios de posgrado o profesional, pero sin sacar título

[] Título de maestría (MA, MS)

[] Título de doctorado (PhD, EdD)

[] Título de escuela profesional más allá de un título de licenciatura (MD, DDS, JD, LLB)

[] Refused

¿Actualmente trabaja y recibe un sueldo?

[] Sí

[] No

[] Refused

¿Es estudiante de tiempo parcial o completo?

[] Sí

[] No

[] Refused

Con base en esta hoja [Hand respondent the income sheet], ¿qué categoría se ajusta mejor al ingreso total de los últimos 12 meses de todas las personas que viven en su hogar? Incluya sus propios ingresos. Incluya el dinero que reciba por otros trabajos o ingresos, pensiones, intereses, alquiler, pagos del Seguro Social, etc. (Do not read response options. If respondent is struggling, let them know they can make their best guess.)

[] $0 a $10,000

[] $10,001 a $20,000

[] $20,001 a $30,000

[] $30,001 a $40,000

[] $40,001 a $50,000

[] $50,001 a $60,000

[] $60,001 a $75,000

[] $75,001 a $100,000

[] $100,001 a $150,000

[] $150,001 a $200,000

[] $200,001 a $250,000

[] $250,001 o más

[] Don’t know

[] Refused

B.2.10. Spanish Income Show Card

A	$0 a $10,000
B	$10,001 a $20,000
C	$20,001 a $30,000
D	$30,001 a $40,000
E	$40,001 a $50,000
F	$50,001 a $60,000
G		$60,001 a $75,000
H		$75,001 a $100,000
I		$100,001 a $150,000
J		$150,001 a $200,000
K		$200,001 a $250,000
L		$250,001 o más

B.2.11. Spanish Incentive Receipt Acknowledgment Form

Yo, ________________________ (Nombre del Participante), confirmo que recibí el siguiente incentivo por parte de American Institutes for Research (Institutos Americanos para la Investigación), como agradecimiento por participar en un estudio.

Tipo	Cantidad
Dinero en efectivo	$120

Firma del participante __________________________ Fecha ________________________

Firma del entrevistador __________________________ Fecha ________________________

B.3. Observation Instrument

Please complete the observation protocol below. Record all observations as unobtrusively as possible. If necessary, you may explain the purpose of your observations if you are asked by any individual in the neighborhood, including anyone from the sampled household.

DEFINITION

What do we mean by the sample unit OR the building within which the sample unit resides? Here we are making a distinction between:

a sample unit that we might call a single occupancy dwelling unit (a single residential structure that stands alone) and
a multi-unit structure where the sample unit resides in a building that houses other units.

GENERAL INFORMATION

Date of observation: _______________

Time of day: ______:_______ AM / PM

Observer ID: ____________________

Nonresponse Study Case ID: ________________________

[Take two or three photos of the address you are going to observe. Photos should be taken from the front of the address, if possible. Photos should include the entire structure, if possible. They should be taken in landscape but can be taken in portrait orientation when needed (e.g., for tall buildings). Photos should only be taken from public spaces such as the street, and will not be taken while on someone’s private property.]

OBSERVABILITY

Are you able to observe the sample unit?

Yes
- No, cannot locate the address
No, can observe the multi-unit building but cannot gain entry to observe the unit
- No, cannot gain sufficient access to observe the building
- No, other reason

[IF Q1 = YES, SKIP TO QUESTION 3; OTHERWISE, CONTINUE.]

Please explain why you are not able to observe the sample unit.

____________________________________________________________________________________________________________________________________________________________________________________

[IF Q1 = NO, CANNOT LOCATE THE ADDRESS, OR NO, OTHER REASON, GO TO Q8; IF Q1 = NO, CANNOT GAIN SUFFICIENT ACCESS TO OBSERVE THE BUILDING, GO TO Q9; OTHERWISE, CONTINUE.]

STRUCTURE TYPE

Select the option that best describes the sample unit or the building within which the sample unit resides:

Single occupancy dwelling unit (one household in a single residential structure that stands alone)
Duplex (two-household structures attached together, but with separate entrances)
Townhouse/rowhouse (3 or more household structures attached together, but with separate entrances)
Low-rise apartment or condominium building (1-3 floors)
Mid-rise apartment or condominium building (4-6 floors)
High-rise apartment or condominium building (7+ floors)
Commercial business
Vacant lot
Other non-residential
Other – please describe: ________________________________________________________________
Could not determine – please describe: ____________________________________________________

[IF Q3 = COMMERCIAL BUSINESS, VACANT LOT, OR OTHER NON-RESIDENTIAL, SKIP TO Q9; OTHERWISE, CONTINUE.]

HOUSEHOLD RESIDENTIAL OCCUPANCY STATUS

Based on your observation, indicate whether the sample unit is an/a:

Currently occupied address (e.g., car in the driveway, toys/chairs/yard tools outside the house, lights on inside the home)
Seasonal address, no current occupants (e.g., resort area and no evidence of current occupants)
Temporarily vacant address (e.g., for sale/rent sign/lock box and no evidence of current occupants)
Permanently vacant address (e.g., windows/doors boarded up, condemned notice/unlivable conditions, currently under construction)
Other
Could not determine

Please describe what aspects of the address made you decide on the specific occupancy status above.

____________________________________________________________________________________________________________________________________________________________________________________

[IF Q4 = PERMANENTLY VACANT ADDRESS, SKIP TO Q9; OTHERWISE, CONTINUE.]

MAIL ACCESS

Based on your observation, how does this sample unit receive mail?

Mail slot
Mailbox attached to their home
Mailbox at the end of a driveway
Mailbox across the street/at the end of the road
Mailbox, slot, or room in the multi-unit building
More than one way - please describe: _____________________________________________________
No mailbox or slot in view
Other – please describe: _______________________________________________________________
Could not determine - please describe: ____________________________________________________

[IF Q1= NO, CAN OBSERVE THE MULTI-UNIT BUILDING BUT CANNOT GAIN ENTRY TO OBSERVE THE UNIT OR IF Q4 = TEMPORARILY VACANT ADDRESS, SKIP TO Q8; OTHERWISE, CONTINUE.]

HOUSEHOLD MEMBER ELIGIBILITY FOR NHES TOPICALS

Based on your observation of the sample unit or building within which the sample unit resides, is there any indication that children 18 or under (including babies) may live IN THE SAMPLE UNIT (e.g., visible toys, bikes, basketball hoop, sports equipment, car seats, strollers, chalk, outdoor swing/play set, bumper sticker or car window sticker, child finder sticker for firefighters, etc.)?

Yes
No
Could not determine

Please describe what aspects of the address made you decide whether or not children live in the sample unit.

____________________________________________________________________________________________________________________________________________________________________________________

HOUSEHOLD ATTRIBUTES

Are there any unique attributes or decorations (e.g., flags, wreaths, bumper stickers, signs) on or around the sample unit that indicate -- If yes, write them down.

Pride in education (e.g., honor roll bumper stickers, university/college flags or decor)

[] Yes [] No

What? _______________________________________

Community involvement (e.g., political candidate signs, indicators of being involved with charities, kids sports/clubs, neighborhood associations)

[] Yes [] No

What? _______________________________________

Patriotism (e.g., American flag, state flag)

[] Yes [] No

What? _______________________________________

Privacy (e.g., security camera, no loitering sign, household alarm sign, locked gate?)

[] Yes [] No

What? _______________________________________

A language other than English (e.g., signs or bumper stickers in another language)

[] Yes [] No

What language? _______________________________

What indicators? _______________________________

Anything else

[] Yes [] No

What? _______________________________________

HOUSEHOLD INCOME

Relative to the general population and based on your observations, would you judge this sample unit’s household income to be… Use visual cues of the conditions of the house/building; number, make, and age of cars at the residence; and knowledge of the neighborhood/environment.

In the bottom third of the population
In the middle third of the population
In the top third of the population
Could not determine

Please describe what aspects of the address made you decide on the specific household income level above.

____________________________________________________________________________________________________________________________________________________________________________________

Provide a general description of the unit and the surrounding neighborhood, commenting only on things that have not already been noted in the questions above (e.g., building entry requirements, condition of address/neighborhood, objects around the address, visible people, other building types in the neighborhood). ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

1 The final base weighted NHES:2016 screener response rate was 66.4 percent. The Early Childhood Program Participation survey yielded a 73.4 percent topical response rate. The Parent and Family Involvement in Education survey yielded a 74.3 percent topical response rate. The Adult Training and Education Survey yielded a 73.1 percent topical response rate. These rates resulted in overall response rates (the product of the screener response rate and topical response rate) of 48.7 percent, 49.3 percent, and 48.5 percent, respectively. The increased use of the web survey mode in 2019 is expected to increase topical response rates.

2 P.O. Boxes that are included in the NHES:2019 sampling frame will be excluded from the nonrespondent sampling frame due to not having a street address associated with the P.O. Box to visit.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Megra, Mahlet (Mahi)
File Modified	0000-00-00
File Created	2021-01-15

National Household Education Survey 2019 (NHES:2019)