OMB83C Change Memo v16

U NITED STATES DEPARTMENT OF EDUCATION

National Center for Education Statistics

October 23, 2018

MEMORANDUM

To: Robert Siviniski, OMB

From: Sarah Grady, NCES

Through: Kashka Kubzdela, NCES

Re: National Household Education Survey 2019 (NHES:2019) Nonresponding Households In-Person Study Update (OMB# 1850-0768 v.16)

The National Household Education Survey (NHES) is a data collection program of the National Center for Education Statistics (NCES) designed to provide descriptive data on the education activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. Such topics have covered a wide range of issues, including early childhood care and education, children’s readiness for school, parents’ perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and career education, parents’ involvement in their children’s education, school choice, homeschooling, and civic involvement. The request to conduct the NHES:2019 full scale data collection, from December 2018 through September 2019, in conjunction with an In-Person Study of Nonresponding Households, designed to provide insight about nonresponse that can help plan future survey administrations was approved in September 2018 (OMB# 1850-0768 v.14-15). NHES:2019 will use mail and web data collection modes and will field two surveys: The Early Childhood Program Participation survey (ECPP) and the Parent and Family Involvement in Education survey (PFI). This request provides the expected update on the final plan for the NHES:2019 In-Person Study of Nonresponding Households.

Due to resource constraints and concerns about generalizability, the originally proposed short structured interviews and comparable sample of responding households are no longer proposed. These are replaced by an enhanced address observation effort.

All materials on pages 14-36 and 48-51 of Appendix 4 are newly added. Please also note that Attachment B.2. of Appendix 4 will provide the Spanish versions of the materials that will be used to interact with households sampled for the qualitative interview. These materials will be provided to OMB as a change request in November 2018.

The changes described in this document do not affect the total cost to the federal government for this study. The revision to the estimated response burden is detailed below.

Revisions to the content of the approved submission documents (Parts A and B and Appendix 4) are reflected in tracked changes below.

Part A Supporting Statement

(p.1)

The contact materials, final interview protocols, and final sample selection details for this study will be submitted to OMB for review with a 30-day public comment period in fall 2018 are also included in Appendix 4.

(pp.5-6)

In-Person Study of NHES:2019 Nonresponding Households

Given continued declines in response rates to both NHES and to household surveys more broadly – and the growing challenges associated with conducting cost-efficient, high-quality, representative data collections, NCES will conduct the In-Person Study of NHES:2019 Nonresponding Households during the NHES:2019 administration. This study will focus on understanding the reasons for the growing level of nonresponse to NHES and to mail-based household surveys. Furthermore, the study will focus on screener rather than topical nonrespondents because of the impact that screener nonresponse has on both the screener response rate and each NHES topical survey’s overall response rate; and because NHES screener response rates are lower than topical response rates.¹ This study will consist of two separate interview operations: (a) structured interviews and (b) qualitative interviews. Due to the differences in interview protocols, data from these two sets of interviews will be analyzed separately. and (b) address and neighborhood observations. Appendix 4 of this submission discusses the importance of conducting this study and provides a more detailed overview of its methods. The finalized procedures and It also provides the study’s contact and interview protocol materials for the In-Person Study of NHES:2019 Nonresponding Households will be submitted for OMB’s review with a 30-day public comment period in fall 2018.

Structured interviews

The primary goal of the structured interviews is to understand who lives in nonrespondent households and the household members’ primary reasons for nonresponse. Approximately 350 screener nonrespondents will be sampled and 230 structured interviews will be conducted with NHES screener nonrespondents. Sampled cases will be clustered around 2 to 4 U.S. cities and will be located within 70 miles of the center of those cities. They will be sent an advance letter that includes $2 or $5 cash², invites them to participate in the in-person study, and indicates that someone will be visiting their home to ask them to complete an interview. The advance letter will not ask for a response or action from the household member, only inform them that an interviewer will visit their house. All sampled cases will be offered an additional $25 cash incentive for completing the interview. When field interviewers first reach a sampled address, they will complete a brief observation of the neighborhood and address as unobtrusively as possible to record observable characteristics of nonresponding addresses. Next, the field interviewer will attempt to conduct a 10-minute (on average) structured interview with the adult who usually opens the mail, or with another adult who is available. Most of the follow-up interview will be devoted to understanding the reason(s) for the household’s lack of response to the screener, with a particular focus on understanding at which stage this household’s response process was interrupted (e.g., did the package reach the household, did the household members open it, etc.), in addition to obtaining basic demographic characteristics and attitudes that may impact the response decision, such as privacy concerns or perceived level of burden. Interviews will be conducted in both English and Spanish.

Qualitative interviews

Approximately 480 additional 500 NHES:2019 screener nonrespondents will be sampled selected to conduct up to 80 participate in qualitative, semi-structured interviews, with a goal of obtaining approximately 100 completed qualitative interviews (see appendix 4 for sampling details). The NHES:2019 screener nonrespondents may be stratified to focus on a handful of key household characteristics available on the sampling frame that are likely to drive differences in reasons for nonresponse and could be used for targeting NHES materials and contact protocols in future administrations: (1) whether the household has children living in it, (2) whether the household is Hispanic or is expected to prefer to respond in Spanish, and (3) whether the household is located in an urban area or a rural area. A final sample design and strata will be specified in the fall 2018 OMB submission (with a 30-day public comment period). Cases will be clustered near 2 to 3 U.S. cities and will be located within 70 miles of the center of those cities. The 70-mile radius will allow for sampling of rural addresses. Sampled cases will be sent an advance invitation letter that includes $2 or $5 cash³, invites them to participate in the in-person study, and indicates that someone will informs them of the contingent incentive, and provides them with NHES staff contact them information for letting the researchers know that they would like to participate. Sample members that do not respond to schedule the invitation letter will receive up to two additional reminder postcards. Addresses with telephone numbers will additionally receive up to 4 phone call reminders. An in-person recruitment period will also take place, where all households that have neither agreed nor declined to participate will be visited by an interviewer. The follow-up mailings will be spread out over several weeks and will be paired with in-person recruitment. All sampled cases will be offered an additional $120 cash incentive for completing the 90-minute interview and this incentive will be mentioned in all recruitment materials. Interviews will be conducted in both English and Spanish, and will be audio-recorded, with participant’s permission (note that due to their brevity and structured nature, the structured interviews will not be audio recorded).

Address and neighborhood observations

Approximately 750 addresses will be sampled from among NHES:2019 early screener nonrespondents for address and neighborhood observations. Of these, 500 will be selected for and invited to take part in the qualitative interview study (to yield approximately 100 completed 90-minute interviews). If nonresponse clustering patterns result in lower than estimated cost of observing and interviewing the target numbers of addresses, more addresses will be selected for the address/neighborhood observations. The objective of these observations is to determine the types of addresses that are prone to nonresponse or having their NHES mailings be undeliverable, and to assess the accuracy of the information available on the frame for such addresses.

(p.10)

In-Person Study of NHES:2019 Nonresponding Households incentives. Sampled cases Cases sampled for the qualitative interview will be sent an advance letter that includes $2 or $5 cash³ and invites them to participate in the in-person study. All cases sampled for the structured interview will be offered an additional $25 in cash for completing the 10-minute in-person interview, while all cases sampled for the qualitative interview will be offered an additional $120 in cash for completing the 90-minute in-person interview. These incentives are designed to demonstrate to participants that their time and participation is valued and to take into account that both samples are comprised of households that have already shown reluctance to participate in the survey (see Appendix 4 for additional discussion of the incentive amount).

(p.13)

	Table 1. Estimated response burden for NHES:2019
Interview forms		Number Sampled	Anticipated Response Rate	Estimated Number of Respondents	Estimated Number of Responses	Average Time Per Response (minutes)	Total Time (hours)
Main study1
Screener		205,000*	53.545%*	98,790	98,790	3	4,940
ECPP questionnaire		8,495	82.220%	6,984	6,984	20	2,328
PFI questionnaire		19,893	82.910%	16,493	16,493	20	5,498
Nonresponding household study2
Structured interview recruitment screener		350	85.714%	300	300	3	15
Structured interview		300	76.667%	230	230	10	38
Qualitative interview recruitment screener		480 500	62.500 75.000%	300 375	300 375	5	25 32
Qualitative interview		300 375	26.667%	80100	80100	90	120150
Study Total				98,790 99,165	123,177 122,742		12,964 948
2The estimated number of respondents to the recruitment screeners for both the structured and qualitative interviews is different than the number of completed interviews to each of the two studies because it is expected that about 23% of recruited participants in the 10-minute structured interview will not participate and about 73% of recruited participants in the 90-minute qualitative interview will not participate due to either not being home during the scheduled interview time or refusing to start the interview after recruitment, given the burden.

NHES:2019 will screen 205,000 households. An expected screener response rate of approximately 54 percent and an address ineligibility⁴ rate of approximately 10 percent are assumed, bringing the total number of expected screeners to 98,790.⁵ From these completed screeners, it is expected that approximately 29 percent will contain an eligible child. A detailed description of the planned sampling design is provided in this submission in Supporting Statement Part B. The hourly wage rate of $24.33 is based on the average for all civilian workers from the September 2017 National Compensation Survey (http://www.bls.gov/news.release/ecec.t02.htm). For NHES:2019, a total of 12,964948 burden hours are anticipated, resulting in a total burden time cost to respondents of approximately $315,414025.

Part B Supporting Statement

(p.9)

Additionally, in conjunction with NHES:2019, NCES plans to conduct an In-Person Study of NHES:2019 Nonresponding Households, designed to provide insight about nonresponse that can help plan future survey administrations. This in-person study is described in detail in Appendix 4 of this submission; its contact materials, final interview protocols, and final sample selection details will be submitted to OMB for review with a 30-day public comment period in fall 2018.

(p.23)

The in-person study of nonresponding households will use structured and qualitative in-person interviews, as well as and address and household neighborhood observations to assess whether the existing theories surrounding household survey nonresponse are applicable to NHES. The study is designed to provide additional actionable information about how to combat the growing nonresponse problem in NHES and other federal government household surveys that use mail to contact sample members. It is expected that the results of this study will be used to improve the design of NHES:2022, with the goal of increasing the response rate and the representativeness of the respondents. Potential changes could include modifications to the nonresponse follow-up protocol, the type of incentives offered, and the presentation of contact materials.

This in-person study is described in detail in Appendix 4 of this submission. The Appendix also includes contact materials, final interview protocols, and final sample selection details for this study will be submitted to OMB for review with a 30-day public comment period in fall 2018.

(p.31)

In-Person Study of NHES:2019 Nonresponding Households: As described in Part A, section A.2, and section B.2 of this document, the in-person study of nonresponding households will use structured and qualitative in-person interviews, as well as and address and household neighborhood observations to assess whether the existing theories surrounding household survey nonresponse are applicable to NHES. 230 completed structured interviews and 80 100 semi-structured, qualitative interviews and approximately 750 (or more of budget permits) address and neighborhood observations will be completed with NHES:2019 nonrespondentsing addresses. Additional detail is provided in Appendix 4, which also includes contact materials, final interview protocols, and final sample selection details and planned analyses for this study will be submitted to OMB for review with a 30-day public comment period in fall 2018.

Appendix 1 – NHES 2019 Contact Materials

Appendix 1 has been updated to reflect the endorsements that have been received for NHES. Specifically, the sections titled “Initial screener package letters” and “Third screener package letters” was revised as follows:

(p.24)

Further, addresses that are part of the tailored materials experiment group and are predicted to have Spanish speakers will receive NHES-11LB(T). Those sampled for the tailored materials experiment group but not predicted to have Spanish speakers will receive the NHES-11L(W) noted above. The tailored letter includes endorsements from organizations and programs that are currently being pursued have been obtained by NCES but have not yet been obtained. A The letter that will be used to seek endorsements is provided preceding the tailored screener materials.

(p.75)

Addresses that are part of the tailored materials experiment group and are predicted to have Spanish speakers will receive the NHES-13LB(T) letter. Those sampled for the tailored materials experiment group but not predicted to have Spanish speakers will receive the NHES-13L(M) noted above. The tailored letter includes endorsements from organizations and programs that are currently being pursued have been obtained by NCES but have not yet been obtained.

The letters in these sections have been revised to show the endorsements.

Appendix 4 – In-Person Study of NHES:2019 Nonresponding Households

All materials on pages 14-36 and 48-51 are newly added. Additionally, the following revisions were made:

(p.1)

Two separate interview operations will be conducted to To address these research questions we will conduct approximately: (a) 230 10-minute interviews using a structured interview protocol and neighborhood observations (“structured interviews”) and (b) approximately 80 100 of 90-minute interviews using a semi-structured an unstructured, qualitative interview protocol and home observations (“qualitative interviews”). Due to the differences in interview protocols, data from these two sets of interviews (b) 750 address and neighborhood observations will be analyzed separately. This document discusses the importance of conducting this study and provides an overview of its methods.

(pp.3-5)

2. The proposed study

The proposed In-Person Study of NHES:2019 Nonresponding Households will use structured and qualitative in-person interviews, as well as address and household neighborhood observations to assess whether the existing theories surrounding household survey nonresponse are applicable to NHES. The study is designed to provide additional actionable information about how to combat the growing nonresponse problem in NHES and other federal government household surveys that use mail to contact sample members. It is expected that the results of this study will be used to improve the design of NHES, with the goal of increasing the response rate and the representativeness of the respondents. Potential changes could include modifications to the nonresponse follow-up protocol, the type of incentives offered, and the presentation of contact materials.

1. Structured interview methodology

The primary goal of the structured interviews is to understand who lives in nonrespondent households and the household members’ primary reasons for nonresponse.

Approximately 230 structured interviews will be conducted with NHES screener nonrespondents. Screener nonrespondents, rather than topical nonrespondents, are the focus of the study because of the impact that screener nonresponse has on both the screener response rate and each NHES survey’s overall response rate and because NHES screener response rates are lower than topical response rates.⁶ Larger numbers of screener nonrespondents are anticipated to be clustered in metropolitan areas than in more rural locales. Exhibit 1 plots the location of NHES:2016 screener nonrespondents. As expected, screener nonrespondents are widely dispersed, but there is some clustering around large cities.

About 350 screener nonrespondents will be sampled to yield these 230 completed interviews. Sampled cases will be clustered around 2 to 4 U.S. cities and will be located within 70 miles of the center of those cities. To evaluate the representativeness of the sample drawn from the cities under consideration (Chicago, IL; Washington, DC; Atlanta, GA; Austin, TX; and San Mateo, CA), we have conducted a preliminary analysis using NHES:2016 data to identify potential biases in cases reached under this design compared to cases that are located outside of these five city areas. Focusing on statistically significant biases of five percentage points or more, this analysis suggests that the following groups would be overrepresented in this study sample:

• Addresses in high percent Hispanic strata (and addresses that are in strata that are neither high percent Black nor high percent Hispanic would be underrepresented);

• Addresses where the head of the household is Hispanic (and those where the head of household is White would be underrepresented);

• Addresses that received bilingual mailings (and those that did not would be underrepresented);

• Addresses in block groups that are 100 percent urban (and those that are located in block groups that are 50 percent or less urban would be underrepresented); and

• Addresses with Census low response scores (LRS) in the fifth (lowest propensity) quintile (and those in the second highest quintile would be underrepresented).

Exhibit 1. Location of NHES:2016 Screener Nonrespondents

Though cases located near cities overrepresent certain subgroups of nonrespondents (such as those that are Hispanic or urban dwellers), these are also the groups that tend to be less likely to respond to NHES. In Moreover, in NHES:2016, household in tracts with 40% of more Hispanic persons, high rise apartments, and renters had lower weighted screener response rates, 55.9%, 58.9%, and 56.6% respectively (see Attachment A in this document for screener response rates by additional subgroups) (McPhee et al. 2018). This study is an opportunity to learn more about the specific response barriers faced by these specific types of sample members. For example, are sample members that need bilingual mailings getting them, and are sample members living in multi-unit or high-rise buildings receiving their mailings? While the resulting sample will not be nationally representative, it will be large enough to provide rich data about these important subgroups of nonrespondents.

Analyses of the resulting data are expected to illuminate some of the factors associated with nonresponse and inform revisions to the content and design of recruitment materials to be used in NHES:2022.

2.1.1 Recruitment

Sampled cases will be sent an advance letter that includes $2 or $5 cash⁷, invites them to participate in the in-person study, and indicates that someone will be visiting their home to ask them to complete an interview. The advance letter does not ask for a response or action from the household member. Its purpose is to inform the household that an interviewer will visit their house. All sampled cases will be offered an additional $25 cash incentive for completing the interview. This incentive is designed to demonstrate to participants that their time and participation is valued and takes into account that the sample is comprised of households that have already shown reluctance to participate in the survey. Recruitment materials will be branded as coming from the National Center for Education Statistics (NCES). The participant will receive a consent form to read and sign before the interview starts, and will be able to keep a copy of the form for their records. The details of obtaining consent and the consent forms will be submitted for OMB’s review with a 30-day public comment period in fall 2018.

2.1.2 Address observations

When field interviewers first reach a sampled address, they will complete a brief observation of the neighborhood and address as unobtrusively as possible. The main goal of this observation task is to better understand the characteristics of nonresponding addresses (e.g., whether they appear to be occupied or whether there are objects present that suggest children live there such as children’s outdoor toys). The observations also will assess quality of the United States Postal Service (USPS) return service (for example, are there addresses that actually appear to be vacant or nonexistent? Is it possible to locate the addresses that had at least one mailing returned as undeliverable, and, if so, what are their characteristics?).

2.1.3 Structured interview protocol

Upon completion of the address observation, the field interviewer will ring the bell or knock on the door of the home. If no one answers the door, the interviewer will leave a “Sorry I Missed You” card at the household. The card will indicate that he or she stopped by and will return at a later day and time. It also will contain a phone number the sample member can call to learn more about the study and/or schedule an interview. If someone answers the door, the field interviewer will introduce him- or herself, show their identification badge and forms to indicate that they are part of the study, mention the advance letter, and attempt to speak with the adult living at this address who usually opens the mail.⁸ If that person is not available, the interviewer will attempt to interview another available adult living in the household. Interviews will be voluntary and will last approximately 10 minutes. Interviews will be carried out by trained field interviewers who will be further trained for this specific study before data collection begins. Some interviewers will be Spanish bilingual speakers so that they can conduct interviews in Spanish if needed.

The goal of the interview is to understand the reason(s) for the household’s lack of response to the screener, with a focus on understanding at which stage this household’s response process was interrupted (e.g., did the NHES:2019 survey package reach the household, did the household members open it?). The interviewer also will collect basic demographic household characteristics; for example, whether any household members are eligible for the NHES topical surveys (generally, children in 12^th grade or younger).⁹ Finally, the interview will include questions about characteristics that may be drivers of the nonrespondent status of the household, such as privacy and confidentiality concerns, busyness, topic salience, and attitudes toward surveys or the federal government (Brick and Williams 2013; Groves et al. 2004; Kulka et al. 1991; Singer and Presser 2008). Interviews will be conducted in both English and Spanish. Data quality control procedures will be undertaken; several options, such as audio recording or manager follow-up with a subset of participants, are currently being evaluated. The structured interview protocol and contact materials are currently being developed (see Attachment B in this document for an outline of the protocol’s content). The finalized procedures, interview protocol, and contact materials for the In-Person Study of NHES:2019 Nonresponding Households will be submitted for OMB’s review with a 30-day public comment period in fall 2018.

1. Qualitative interview methodology

Qualitative research is particularly useful for trying to understand how people make sense of their lives and experiences. To gain a deeper understanding of why some sample members do not respond to the NHES screener, approximately 80 semi-structured 100 unstructured, qualitative interviews will be conducted with screener nonrespondents. These interviews are designed to be longer and provide richer rich and more nuanced information than the structured interviews described above about survey nonresponse.

The NHES:2019 screener nonrespondents may will be stratified and sampled to focus on a handful of key household characteristics available on the sampling frame that are likely to drive differences in reasons for nonresponse and could be used for targeting NHES materials and contact protocols in future administrations, such as (1) whether the household has children living in it, (2) whether the household is Hispanic or is expected to prefer to respond in Spanish, and (3) whether the household 2) in what part of the United States the address is located an urban area or a rural area. A final sample design . Cases will be sampled from three or more sites and sampling strata will be specified in the fall 2018 submission. In addition, the cases sampled for each site will be clustered near 2 to 4 U.S. cities and will be located within 70 miles of the center of those cities. The 70 an area with approximately a 30-mile radius.¹⁰ This design will allow for sampling of some rural addresses (e.g., villages and towns such as Sollitt, IL, which is 45 miles south of Chicago, IL and is, while still maintaining a rural community with fewer than 100 people within the community). manageable driving distance for the interviewers. Approximately 750 addresses will be sampled from among NHES:2019 early screener nonrespondents for address and neighborhood observations. Of these, 480 500 households will be sampled selected for and invited to take part in the qualitative interview study (to yield approximately 80 100 completed 90-minute interviews). If nonresponse clustering patterns result in lower than estimated cost of observing and interviewing the target numbers of addresses, more addresses will be selected for the address/neighborhood observations.

The cases included in the nonresponse follow-up study will be selected using frame variables that will be identified based on the observed patterns of nonresponse to early NHES:2019 screener mailings. Because NHES:2019 will be the first NHES administration that uses a web-push mixed-mode design (whereas recent NHES administrations used a paper-only design), we are not certain what patterns of screener nonresponse will be observed and thus must wait to determine the actual selection criteria for the nonresponse follow-up study until NHES:2019 screener data collection has begun and patterns of the early NHES:2019 screener nonresponse become known. However, patterns of screener nonresponse to NHES:2016 give some indication of address characteristics that are likely to drive screener nonresponse in 2019, such as: being in an area with a higher percentage of Black or Hispanic residents, being in an area with a higher poverty rate, being in the south, being in a rented (vs. owned) dwelling, being in a multiunit (vs. a single unit) dwelling, not having a phone number available on the frame, or lacking demographic information about the head of the household (such as gender, age, or marital status).

Once we determine the subgroups of address characteristics that show the highest likelihood of nonresponse to early NHES:2019 screener mailings, we will identify three or more geographic clusters, of approximately 30-mile-radius each, that will include large enough numbers of nonrespondents from the address characteristics subgroups to meet the recruiting targets for the nonresponse follow-up study. Due to differences in mail delivery challenges in urban versus rural areas (which are key for understanding potential reasons for nonresponse to a mailed survey such as NHES), urbanicity will be one of the factors used in the selection of sites. In addition, we will ensure that each of the at least three clusters will include a diverse group of addresses by characteristics such as region, address characteristics, demographics, etc. Initially, in each selected cluster, address/neighborhood observations will be conducted with the nonresponding households selected from that cluster. From the observed addresses, across all clusters, a subset of 500 addresses will be selected for recruitment for the target 100 in-person qualitative interviews. At this time, the cost of observations and interviews will be determined and, budget permitting, additional NHES:2019 screener nonrespondent addresses may be selected for additional address/neighborhood observations.

One possible design for the study of nonresponding households is to also follow up with a small sample (n= 50-100) of responding households for the 90-minute semi-structured, qualitative interviews. Data from responding households would provide a comparison to the data from the nonresponding households. Respondent sample cases could be stratified on whether they were early or late responders and on the same demographic characteristics used to identify cases for nonrespondent sampling. If differences are found between the responding and nonresponding groups, this will aid analyses by helping to hone in on the themes in the data from nonrespondents that may be related to nonresponse. If no differences are found between the responding and nonresponding groups, it will be a signal to look at some of the less common themes in the interview data for clues about nonrespondents’ unique characteristics or attitudes. In turn, these less common themes would provide future researchers with better information on which to base interview protocols for research into nonresponse.

Bates (2007) found that debriefing both respondents and nonrespondents to a mail survey yielded considerably more data from respondents than nonrespondents and that nonrespondents commonly did not remember receiving the mailed survey materials. Collecting data from respondents in addition to nonrespondents would help us to understand whether or not differences in household practices with mail are the linchpin to ABS-survey nonresponse in which contact materials are sent through the mail. Budget may limit our ability to add a sample of responding households. Final decisions and details about the sample design, and whether or not responding households will be included, will be provided for OMB review in fall 2018.

2.1.1 Recruitment

The following recruitment protocol will be used after the NHES:2019 screener data collection closes in late May 2019:

• Sampled cases will be sent an advance invitation letter that includes $2 or $5 cash¹¹, invites them to participate in the in-person study, and indicates that someone will informs them of the contingent incentive and provides them with NHES staff contact them to schedule an interview. The advance letter does not ask information for a response or action from the household member. Its purpose is to inform letting the household researchers know that they will be contacted to schedule an interview at their house would like to participate. All sampled cases will be offered an additional $120 cash incentive for completing the interview and this incentive will be mentioned in all recruitment materials. This incentive is designed to demonstrate to participants that their time and participation is valued and takes into account that the sample is comprised of households that have already shown reluctance to participate in the survey (see below for additional discussion of the incentive amount). Recruitment materials will be branded as coming from the National Center for Education Statistics (NCES).

• Up to two reminder post cards will be sent to each sampled household that has not yet responded to the nonresponse follow-up study invitation. The first will be sent in the early part of the recruitment period, approximately one week after the invitation letter. A second will be sent to the remaining follow-up nonrespondents about halfway through the recruitment period. These postcards will invite sampled households to participate in the study, remind them about the contingent incentive, provide them with NHES staff contact information for letting the researchers know that they would like to participate, and inform them that an interviewer will be visiting their home to try to schedule an interview.

• Recruitment phone calls will occur in two windows for sampled households that have phone numbers available on the sampling frame. One window will occur at and have not yet responded to the start of nonresponse follow-up study contacts. The first will be early in the recruitment period (shortly after the first reminder postcard) and the other will be about halfway through the recruitment period. During each window, each household will be called twice up to two times. One voicemail will be left per calling window if the household does not answer the phone.

• An in-person recruitment period will also take place, where all households that have neither agreed nor declined to participate over the phone (or do not have a phone number available) will be visited by a recruiter. Recruiters an interviewer. Interviewers will have an identification badge, which they will show to each household along with study forms to indicate that they are part of the study.

The phone and in-person recruiters recruitment will encourage the household to participate in the in-person interview, screen for eligibility (e.g., the participant needs to have been living at the address when the NHES:2019 materials were sent), and set up a time for a trained qualitative to do the interview. The interviewer to do so. The recruiter will attempt to speak with the an adult living at this address who usually opens the mail. Should that person not be available, they the interviewer will attempt to speak to another available adult living in the household. If, when the interviewer arrives, no one answers the door, the interviewer will leave a “Sorry We Missed You” card at the address. The card will indicate that someone stopped by and will return at a later day and time. It will also contain a phone number and e-mail address the sample member can contact to learn more about the study and/or schedule and interview.

All materials will be branded as coming from the National Center for Education Statistics (NCES). For sampled addresses that are expected to have a high likelihood of being Spanish speaking, the mailed recruitment materials will include both English and Spanish; bilingual phone and in-person staff will be assigned to these cases.

To maintain engagement and increase the likelihood that recruited households keep their scheduled appointments, the recruiters interviewers also will ask recruited households to provide contact information that can be used to keep them engaged and aware of their upcoming interview appointment. Depending on the type of contact information provided, recruited households will receive: (1) a confirmation email or text shortly after scheduling the appointment, and (2) a reminder email or text and a reminder phone call in the days leading up to the interview. The interviewer will then call the participant the morning of the interview to confirm he or she is still available to meet. If when the interviewer arrives, no one answers the door, the interviewer will leave a “Sorry I Missed You” card at the household. The card will indicate that he or she stopped by and will return at a later day and time. It also will contain a number the sample member can call to learn more about the study and/or schedule an interview. All interviewers will have an identification badge they will show to the household to indicate they are part of the study.¹² Interviews will be carried out by trained qualitative interviewers who will be further trained for this specific study before data collection begins. Some interviewers will be Spanish bilingual speakers so that they can conduct interviews in Spanish if needed. All materials will be branded as coming from the National Center of Education Statistics (NCES). (or an appointment reminder card if they are recruited in-person), and (2) a reminder email and/or phone call the day before the interview.

The participant will receive a consent form to read and sign before the interview starts, and will be able to keep a copy of the form for their records. The details of obtaining consent and the consent forms will be submitted for OMB’s review with a 30-day public comment period in fall 2018.

(pp.7-8)

2.1.2 Qualitative interview protocol

Qualitative interviews will be voluntary and will last 90 minutes to allow time to obtain consent, give participants sufficient time to reflect on the more open-ended questions included in this protocol, and conduct observations. Because this is exploratory research (trying to find out why something is happening as opposed to testing a hypothesis) maximizing contact time with the participants and giving them enough space to discuss their experience is crucial. Interviews will be conducted in both English and Spanish. All interviews will be audio-recorded with participants’ permission¹³. They will consist of the following kinds of questions:

• Qualitative interview questions that aim to gain a deep understanding of the household’s reasons for nonresponse and what might convince them to respond (covering domains such as understanding of surveys, privacy concerns, attitudes toward the government, attitudes toward education, time use, and civic engagement);

• Questions adapted from the More structured interviews for the qualitative interview protocol questions, such as conducting a household roster by asking about the other members of the households participant’s demographic characteristics, or asking if the participant remembers receiving the NHES mailings;

• A mail interaction discussion, using a packet of example mail where the interviewer asks the participant to do things like “show me what you do with your mail when you get it” or “show me model how you decide whether to open a piece of they sort mail and make decisions about what to keep, read, or throw away, while the interviewer probes about that decision process; and

• A request for feedback on NHES:2019 screener mailed materials.

2.1.3 Observations

The interview will conclude with the interviewer using a brief check list to make making observations about the home and/or household members, beyond any observations recorded while interacting with the participant during the interview. Observations are important to include in qualitative research because interview questions can often only address attitudes or actions of which participants are cognitively aware. There are likely other observable aspects of participants’ lives that they may not explicitly connect to their lack of participation (e.g., how they organize their mail), as well as observable qualities that nonresponding households may have in common (e.g., whether they display children’s educational achievements). These observations will be used to capture such more nuanced indicators.

The qualitative finalized interview protocol and contact materials are currently being developed (see Attachment B for an outline of the protocol’s content). The finalized procedures, interview protocol, and contact materials for the In-Person Study of NHES:2019 Nonresponding Households qualitative interviews can be found in the Attachment B of this document.

2.2. Address/neighborhood observation

Approximately 750 addresses will be sampled for address and neighborhood observations. Of these, 500 will be also invited to take part in the qualitative interview study (see above), and for the remaining 250 households, only observations will be collected. The objective of these observations is to determine the types of addresses that are prone to nonresponse or having their NHES mailings be undeliverable, and to assess the accuracy of the information available on the frame for such addresses.

The objective of these observations, which will be carried out by staff contracted for NHES, is to determine the types of addresses that are prone to nonresponse or having their NHES mailings be undeliverable and to assess the accuracy of the information available on the frame for such addresses. The instrument has been programmed to be administered on a tablet or computer to ease data collection efforts and minimize error. Each observer will receive a tablet to complete their observations. The 7-minute observation instrument is designed to evaluate variables from the address-based sample frame. Several of the items are based on existing instruments (e.g., the Neighborhood Observation Instrument (NOI), Contact History Instrument (CHI), the Los Angeles Family and Neighborhood Survey). The instrument also includes a series of characteristics that interviewers will look for and mark off when they observe them for that address or neighborhood. There is also space for entering notes that might provide helpful context about the address or neighborhood. Exhibit 2 below displays a summary of the proposed measures, the frame variables that will be evaluated, and the potential impacts those findings could have on the design of future NHES administrations. To increase efficiency, observers may audio record their observations and transcribe them onto the instrument later.

Exhibit 2. Summary of proposed measures, frame variables evaluated, and potential impact on NHES design

Measure	Frame Variable(s) evaluated	Potential Impact
Structure type	Mailing address type Dwelling type Address vacancy status Seasonal address Delivery point usage code	Mailing method (e.g., use courier service such as FedEx)
		Contact materials (e.g., use larger font for address to minimize sorting issues for multi-unit addresses, tailor language of contact materials)
		Incentive use (e.g., use varied incentives by structure type)
		Frame accuracy (e.g., evaluate frame information to improve adaptive design models)
Occupancy	Address vacancy status Seasonal address	Frame accuracy (e.g., evaluate frame information to improve adaptive design models)
		Eligibility (e.g., confirm whether the postmaster ‘vacant household’ status is accurate)
Eligibility	Presence of children	Topical Eligibility (e.g., assess whether the nonresponse process differs based on topical eligibility)
		Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
		Contact materials (e.g., tailor language used in survey invitation to alleviate privacy concerns of certain households such as households with children)
Mail Access	Mailing address type Drop point Delivery point type code	Mailing method (e.g., using courier services that provide door-to-door service)
		Contact materials (e.g., use bolder envelope designs such as larger font for addresses where mail mix-up is likely)
		Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
Household income	Household income	Incentive use (e.g., use varying incentive structures by income level)
		Frame accuracy (e.g., evaluate frame information particularly to improve adaptive design models)
Additional address / neighborhood observations	None (these observations will be used to assess whether additional census planning database variables should be used for making data collection protocol changes)	Contact materials (e.g., tailor emphasis of individual (incentive) versus communal (policy impact) advantages of NHES participation)

As part of the 7-minute observation, observers will also take a picture of the sampled address to validate that the observer was indeed present at the address, to document a time and location of the visit, and to clarify any issues related to the observation the observer may have recorded. The photos will only be taken from public spaces such as the street, and will not be taken while on someone’s private property. The finalized observation protocol for the In-Person Study of NHES:2019 Nonresponding Households is provided in Attachment B.3 of this document

All materials on pages 14-36 and 48-51 of Appendix 4 are newly added. Please also note that Attachment B.2. of Appendix 4 (pages 37-47) will provide the Spanish versions of the materials that will be used to interact with households sampled for the qualitative interview. These materials will be provided to OMB as a change request in November 2018.

1 The final base weighted NHES:2016 screener response rate was 66.4%. ECPP yielded a 73.4%, PFI a 74.3%, and ATES a 73.1% topical response rate. These rates resulted in overall response rates (the product of the screener response rate and topical response rate) of 48.7%, 49.3%, and 48.5%, respectively. The increased use of the web survey mode in 2019 is expected to increase topical response rates.

2 The incentive amount of $2 vs. $5 is currently under consideration. One amount will be selected for data collection and provided to OMB for review in fall 2018.

3 The incentive amount of $2 vs. $5 is currently under consideration. One amount will be selected for data collection and provided to OMB for review in fall 2018.

4 Ineligible addresses are those that are undeliverable. Screener mailings for an address where one or more mailings are returned as a postmaster return (PMR) and no mailings are returned completed or refused will lead to an address being coded as ineligible.

5 Address eligibility and response rates are estimated based on NHES:2016 and are calculated to account for expected differential response rates within sampling strata and experimental treatment groups.

6 The final base weighted NHES:2016 screener response rate was 66.4 percent. The Early Childhood Program Participation survey yielded a 73.4 percent topical response rate. The Parent and Family Involvement in Education survey yielded a 74.3 percent topical response rate. The Adult Training and Education Survey yielded a 73.1 percent topical response rate. These rates resulted in overall response rates (the product of the screener response rate and topical response rate) of 48.7 percent, 49.3 percent, and 48.5 percent, respectively. The increased use of the web survey mode in 2019 is expected to increase topical response rates.

7 The incentive amount of $2 vs. $5 is currently under consideration. One amount will be selected for data collection and provided to OMB for review in fall 2018.

8 Only one field interviewer will be at the household. To ensure interviewer safety, a security escort will be provided when field interviewers are visiting unsafe neighborhoods.

9 While this basic demographic data is similar to information requested in the screener, these data will not be incorporated into the NHES:2019 dataset or used to send the household a topical NHES:2019 survey.

10 P.O. Boxes that are included in the NHES:2019 sampling frame will be excluded from the nonrespondent sampling frame due to not having a street address associated with the P.O. Box to visit.

11 The incentive amount of $2 vs. $5 is currently under consideration. One amount will be selected for data collection and provided to OMB for review in fall 2018.

12 Only one field interviewer will be at the household. To ensure interviewer safety, a security escort will be provided when field interviewers are visiting unsafe neighborhoods.

13 Note that due to their brevity and structured nature, the structured interviews will not be audio recorded.

550 12^th Street, S.W., Washington, DC 20202

Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.