Survey Administration Plan

his document presents Insight Policy Research’s (Insight) draft Survey Administration Plan for the ConnectHome Internet Subscribers Telephone Survey. The survey is part of the evaluation of the U.S. Department of Housing and Urban Development’s (HUD) ConnectHome pilot initiative to increase high-speed Internet access among families with school-aged children in public housing. Results of this survey will be a critical component of assessing the effect of the ConnectHome pilot initiative in 28 communities across the United States. The purpose of the survey is to collect contextual information about subscribers’ Internet experiences, including use of the Internet to improve subscribers’ quality of life and to potentially create opportunities that did not exist without Internet access.

Insight drafted the survey instrument, drawing on existing survey questions that have been implemented in other surveys of Internet use and digital literacy and incorporating feedback from HUD, ConnectHome stakeholders, and pretest participants.¹ The survey will be administered by Insight’s subcontractor ICF International (ICF), a major provider of telephone surveys for Federal agencies, including HUD.

The survey will be fielded to a sample of up to 2,500 households, which HUD will identify through its partnerships with Public Housing Agencies (PHAs) as recent ConnectHome Internet subscribers. PHAs will provide Insight with a frame of telephone numbers, and Insight will sample from that frame as needed. ICF will conduct the telephone surveys using computer-assisted telephone interviewing (CATI) software. At HUD’s discretion, PHAs may conduct outreach to sampled households to encourage their participation in the survey.

Chapter 2 of this plan provides an overview of the development and design of the materials, including the advance notification letter and the survey instrument. Chapter 3 describes the survey administration approach including sampling, CATI programming and training staff, data collection and nonresponse follow-up procedures, and final data file development. Chapter 4 provides a detailed timeline for all survey administration activities. The survey materials appear in appendices A and B.

Chapter 2. Survey Materials

his chapter describes the materials that will be used in the ConnectHome Internet Subscribers Telephone Survey administration (section A) and provides an overview of how they were developed (section B).

Overview of Survey Materials

The materials used to administer the ConnectHome Internet Subscribers Telephone Survey will include an advance notification letter (section 1) and the survey instrument, which will be available in English and Spanish (section 2).

Advance Notification Letter

All households sampled to participate in the ConnectHome Internet Subscribers Telephone Survey will receive an advance notification letter 2 to 3 weeks prior to receiving a telephone call to complete the survey. The advance notification letter will provide a detailed description of the ConnectHome initiative and the importance of participating in the survey. The letter will notify participants that the information they provide will be kept confidential and that their housing benefits will not be affected by taking part in the survey. Insight recommends using a $15 incentive for participation to boost response rates; the advance notification letter will describe such an incentive.

The advance notification letter will be signed by Everyone On, a partner of the ConnectHome initiative, in an effort to assure potential respondents of the legitimacy of the survey data collection. The letter appears in appendix A.

Survey Instrument

The draft ConnectHome Internet Subscribers Telephone Survey includes approximately 70 questions. Initial estimates suggest the instrument should take roughly 20 minutes to complete, but this estimate will be confirmed during the pretest phase in July. The survey covers the following topics: households’ use of Internet and technology in general; households’ use of Internet for education, employment, and health purposes; digital literacy; and demographic characteristics. HUD will translate the survey into Spanish as needed (see chapter 3, section B). The draft survey instrument is shown in appendix B.

Development of Survey Materials

Insight provided the initial draft of the baseline access survey to HUD on February 17, 2016. When developing this instrument, Insight consulted several surveys for existing survey questions related to Internet use and digital inclusion, including the Pew Internet and American Life Survey, the Seattle Residential IT Survey, the Current Population Survey, the Panel Study of Income Dynamics, the Smart Communities Survey, the Opportunity for All Survey, and the ConnectHome Baseline Access Survey.

HUD reviewed the instrument and provided comments on February 24, incorporating feedback from stakeholder communities and entities. Insight provided a revised instrument for review on February 29, along with a draft advance notification letter.

In early March, HUD worked with two PHAs (Tampa and Kansas City) to recruit potential participants for a pretest of the survey instrument. The Tampa PHA provided a list of 10 participants who were willing to participate in a pretest interview over the phone, and who were in households that had recently received high-speed internet access through ConnectHome. The pretest was delayed, however, due to delays in implementation of ConnectHome precluding the administration of the survey until fall 2016.

Insight will conduct the pretest in July 2016 with up to five public housing residents to obtain feedback on the materials’ clarity, relevance, and design. The pretest interviews will last 45 minutes to an hour, and Insight will provide a $50 Walmart gift card as a token of appreciation for participants’ time. Insight will use the feedback from the pretest to recommend revisions to the instrument and cover letter and will summarize the findings in a memo provided to HUD by July 25. Following feedback from HUD on July 27, Insight will finalize the instrument by July 29, and HUD will translate the instrument and cover letter into Spanish by August 1. HUD will be responsible for obtaining OMB approval for the ConnectHome Internet Subscribers Telephone Survey, and will submit the materials to OMB, including the English and Spanish versions of the instrument, by August 2. HUD anticipates the OMB review process will take approximately 60 days following submission, which would mean an estimated receipt of clearance in early October. Insight will provide the instruments to the call center on August 17; note the instruments must be final because the programming of the instruments will start immediately.

Chapter 3. Survey Administration

his chapter describes the sampling frame and approach, CATI programming and training procedures, data collection procedures and practices, and data file development for the ConnectHome Internet Subscribers Telephone Survey. To ensure acceptable standards are met for protection of respondent privacy, ICF will obtain institutional review board approval.

Sampling

HUD—or a partner of HUD, such as a PHA in a participating community—will provide Insight the sampling frame of phone numbers and names for the heads of household for households that have recently obtained high-speed Internet access in their homes through ConnectHome. If possible, a secondary phone number for these households will be provided to increase the likelihood of reaching selected households.

HUD will provide a preliminary estimate of the size of the frame to Insight by July 13. If more than 2,500 households are included in the frame, Insight will develop a sampling approach to draw a sample of up to 2,500 households. Insight will provide a memo summarizing this approach by July 27. Although the frame itself (i.e. list of names, address, and phone numbers for recent Internet subscribers) will not be needed until August, Insight will need to know the total size of the sample frame by July 13 to accurately design the sampling approach, receive the frame by August 17 to conduct the sampling and QA process, and provide the sample file to ICF by September 7. Phone numbers provided later in the fielding period would reduce the time available for the call center to attempt to reach the households and complete the surveys.

ICF Survey Programming and Training Procedures

Once the instrument is finalized and the sampling is completed, ICF will prepare to field the survey. This section describes the process for programming the survey (section 1) and training call center interviewers (section 2) for the ConnectHome Internet Subscribers Telephone Survey.

Programming the Survey

Upon receipt of the final survey document, the first step in CATI programming will be to reformat the questionnaire to ensure all questions are included in the survey for the location specified. Logic statements will be added at the beginning of each question to make it clear to the survey programmer which respondents should be asked each question and for which respondents certain questions should be skipped. Once the questionnaire document is prepared for CATI programming, the document will be sent back to Insight for final approval. After receiving approval, ICF will program the survey.

ICF has Spanish-speaking interviewers; the survey may be conducted in Spanish as needed. We assume up to 15 percent of interviews will be conducted in Spanish.

New Hire Training

Superior data collection performance depends on call center staff prepared with training beyond basic question-by-question reviews. This training begins with new hire training. New interviewers participate in a rigorous 2-day training that—

Gives them an excellent foundation in proper interviewing techniques
Teaches them techniques for gaining respondent trust and cooperation
Instructs them on how to work efficiently within the CATI program software
Emphasizes the importance of survey work and their role within the project

New interviewers then receive extensive hands-on practice with the CATI system and work through an exhaustive series of practice interviews and interviewing situations.

On the second day of training, trainers discuss techniques for—

Handling difficult situations
Probing for answers in challenging situations
Identifying a member of the household
Handling additional refusal aversion/conversion techniques

All interviewers participate in monthly interviewer enhancement courses. ICF has developed 12 refresher courses, one for each month, on topics for which skills can continually be honed and improved, such as bypassing gatekeepers and countering refusal aversion.

Fielding the Survey: Data Collection

The advance notification letters will be mailed to sampled households 2–3 weeks prior to ICF initiating data collection. The survey will be fielded over a 3-month period, approximately mid-October until mid-December.²

The survey instrument will use a screener question at the beginning of the survey to identify appropriate respondents. ICF’s CATI software package will release sample records as needed throughout fielding of the phone survey to allow for up to 10 attempts by telephone to reach each sample household. The CATI software will be programmed to adhere to the following best practice dialing protocols:

Setting allowable calling times
Defining maximum number of contact attempts
Rotating the day of the week call attempts are made
Prioritizing scheduled callback appointments
Suspending interviews as necessary
Calling busy lines back at a minimum of 10-minute intervals
Automatically attempting a second call for partial completes
Disposing of fax lines

ICF will employ a proven strategy for achieving excellent response rates. Tactics will include the following:

Displaying caller identification
Prioritizing scheduled appointments
Allowing appointments outside usual calling hours
Focusing on completing first contacts
Streamlining interviewer screens
Ongoing training
Tracking response and cooperation rates
Using an optimal interviewing protocol

Final Data File Development

Insight will prepare electronic tallies of survey responses by disposition code and submit these to HUD monthly throughout the 3-month data collection period. Insight will review, clean, and tabulate the final data.

Chapter 4. Timeline

able 4.1 provides an estimated timeline of survey administration events, based on chapters 1 through 3 and receipt of the final sampling frame from HUD on August 17. Bold steps correspond to steps for which HUD is responsible.

Table 4.1. Telephone Survey Administration Timeline

Task	Time Needed To Complete Task	Expected Date
Insight submits draft survey administration plan (Task 6.4.3) to HUD		February 29 (complete)
HUD provides comments to Insight		July 2
Insight submits final survey administration plan to HUD		July 9

Final decision from HUD on any non-English languages for the phone survey		February 26 (complete)

Develop the survey instrument
Insight provides rough draft survey to HUD		February 17 (complete)
Conference call to obtain feedback from stakeholders	1 week	(complete)
HUD provides initial comments	1 week	February 24 (complete)
Insight provides revised instrument to HUD	1 week	June 29 (complete)
HUD provides final comments	1 week	By July 2
Insight finalizes revisions	1 week	By July 7

Advance letter to survey participants
Insight provides draft advance letter to HUD		June 29
HUD provides comments		July 2
Insight finalizes advance letter		July 7

Pretest	3-4 weeks total
HUD or partner recruits pretest participants (or confirms we can call the previously recruited participants from 4 months ago)	1 week	By July 1
Insight conducts pretest	1 week	By July 11
Insight summarizes results and revises instrument	1-2 weeks	By July 25
HUD provides comments on revisions		July 27
Insight finalizes instrument		July 29
HUD translates instrument and cover letter into Spanish		August 1

OMB
HUD begins drafting Supporting Statement Part A and Part B		June 24
HUD conducts internal privacy assessment		June 29
HUD internal PRA review		June 29
HUD incorporates PRA review feedback into package		July 6
HUD submits final materials to OMB, including translations¹		August 2
Estimated receipt of OMB clearance (assuming 60 days)		~October 2

Sampling
HUD provides Insight with the preliminary frame size, by community if possible		July 13
Insight provides draft sampling memo to HUD (Task 6.4.2a)²	2 weeks	July 27
HUD provides comments to Insight		August 3
Insight provides final sampling memo to HUD		August 10
HUD provides frame to Insight		August 17
Insight reviews the frame file and confirms sampling approach	1 week	August 24
Insight conducts sampling and QA	1-2 weeks	By September 7
Insight provides sample to ICF		September 7

ICF prepares to field the survey	5-6 weeks total³
Insight provides English and Spanish instruments to ICF		August 17
ICF’s IRB application/approval	3 weeks
Program and test the instrument (both ICF and Insight)⁴	4 weeks
Train the interviewers (both ICF and Insight)	2 weeks
Insight provides sample to the call center		September 7
ICF prepares the sample	2+ weeks

Advance notification letter sent to sampled households	2-3 weeks in advance of fielding period	October 2 (or whenever OMB clearance received)
Insight/ICF field the phone survey	3 months	Starting October 16, 2016 – December 16, 2016

¹ Our understanding of what is needed for OMB:

Summary of sampling approach
Recruiting approach
Incentive plan
Description of any non-English languages for the survey
Advance letters and phone scripts
Draft instruments

² If fewer than 2,500 phone numbers are available, Insight will conduct the survey using a census instead of a sample.

³ Assumes a fairly simple survey, with minimal skip patterns, etc.

⁴ Assumes no changes to the survey instruments based on OMB feedback. Instruments provided to the call center for programming must be final.

Appendix A. Advance Notification Letter

[Placeholder for Final Advance Letter]

Appendix B. Telephone Survey Instrument

[Placeholder for Final Survey Instrument]

1 The instrument will be pretested in July, pending identification of pretest participants by HUD and PHAs.

2 Dates pending OMB approval by October 2.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Daven Ralston
File Modified	0000-00-00
File Created	2021-01-23