ConnectHome Baseline Survey PRA SS(Part B) Final 8_17_16

Part B

Statistical Methods

The U.S. Department of Housing and Urban Development will perform an evaluation of ConnectHome, a national initiative that will bring the opportunity for high-speed home Internet access to low-income families across the nation. The first stage of this evaluation is a baseline survey of public housing households that will be offered free or significantly discounted at-home Internet access in the 28 participating communities. The survey design will consist of a stratified systematic random sample of public housing units located in the 28 communities targeted for ConnectHome services.

The baseline survey will provide HUD with baseline measures of in-home high-speed internet access, barriers to access among those without access, and types of devices used to access the internet. Upon establishing baseline measures, HUD’s ConnectHome team will use this information to support local efforts in closing the digital divide. Following the baseline, participating Internet Service Providers (ISPs) will provide stakeholder organization EveryoneOn with updates on ConnectHome subscription figures. The baseline survey and information from the ISPs will enable HUD to establish a reliable point estimate of how many additional public housing households gain at-home Internet access through ConnectHome.

The baseline survey will only include public housing households: the vast majority of communities are focusing on public housing as they begin to implement ConnectHome. Based on conversations with communities, HUD anticipates that most communities will begin to serve households with housing choice vouchers or in multifamily buildings in summer or fall of 2016. HUD plans to perform a baseline survey of housing choice voucher and multifamily households

We expect to achieve a 50 percent response rate among households, based on feedback from participating communities. HUD is setting the sample size at 5,600 households (estimated 200 per community) to reach a goal of 2,800 total responses. HUD’s contracted research organization, Insight Policy Research, will draw the sample from HUD Public Housing Information Center (PIC) data.

The baseline survey and the ISPs’ performance data is part of HUD’s larger evaluation of ConnectHome. PHAs will provide HUD with ad hoc updates on how many of their assisted buildings and units are accessible for high-speed Internet service: HUD’s conversations with communities indicate that residents in most buildings can already access service. HUD also plans to perform telephone surveys and focus groups that will consider ConnectHome subscribers’ use of their high-speed, at-home Internet connections and self-reported benefits, including for education and employment. The follow-up telephone surveys and focus groups will be the subject of separate submissions for OMB clearance.

Ronald M. Hill, a Social Science Analyst in HUD’s Office of Police Development and Research, is the current project director of this initial data collection. He will serve as the Government Technical Representative (GTR) of the study.

B1 Potential Respondent Universe

The sampling frame for the internet access survey will be built from a list of eligible public housing households located in areas targeted for ConnectHome services in 28 communities provided by HUD, which is the agency that administers public housing in the communities. HUD will extract a list of public housing units in targeted communities immediately prior to sampling. The unit of analysis for the survey is a public housing unit as opposed to a public housing resident to eliminate duplicate responses for the same unit.

The sampling frame will be restricted to public housing units (as of November 2015) to minimize recall bias during the survey. Responses will be requested from the main owner or tenant in the public housing unit in order to elicit the most accurate information regarding Internet access.

B2 Statistical Methods

This section describes the conceptual approach to conducting a baseline assessment of high-speed Internet access by public housing residents in 28 ConnectHome communities. The baseline assessment data will be collected through a mail survey administered to a probability-based sample of households living in HUD public housing properties. The survey will have 7 questions and take 5 minutes to complete. Public reporting burden for this collection of information is estimated to be .083 hours (5 minutes) per PHA resident, and includes time for reviewing the instructions, and completing and reviewing the responses. The completion of this information collection is voluntary. The survey will ask whether respondents have home Internet access; if so, how they access the Internet and what devices they use; and if not, reasons for not having Internet access.

Below is a description of key activities to conduct the access survey data collection.

The survey sampling frame of public housing households will be based on a list of current public housing units in areas targeted for ConnectHome services provided by HUD. The survey design will consist of a stratified systematic random sample of 2,800 respondents obtained through a sample of 5,600 public housing households located in the 28 communities targeted for Connect Home services.

Target Population. The target population for the internet access survey is individuals residing in public housing units targeted to receive ConnectHome services to increase high-speed home Internet access in 28 communities participating in the ConnectHome initiative.

Survey Eligibility. All individuals in the survey target populations are eligible for the study, so no screening will be needed beyond verification that the respondent is a resident in the sample household in the targeted public housing development.

Statistical Methodology for Stratification and Sample Selection. We plan to select a stratified systematic random sample of 5,600 public housing units from the sampling frame of all units in the 28 ConnectHome communities provided by HUD. Prior to sample selection, we will first stratify the data based on the 28 ConnectHome communities. We will then sort the units by zip code and property and perform systematic sampling within each of the 28 strata. This method involves numbering the public housing households in the population from 1 to N (= total records in population). To select a sample of households, we take a household at random from the first K households and every kth household thereafter until the designated sample size of households is achieved in the stratum. In this way, each household in the sampling frame will be given a known, nonzero probability of selection so that weighted inferences can be made about the entire population of public housing households. This will ensure a random sample within the 28 strata.

The sampling procedures are designed so study findings can be used to make statistically defensible inferences about the entire population of public housing units targeted for ConnectHome services in the 28 communities. Goals of the sample design include (1) the ability to produce accurate estimates of the true public housing population in the 28 communities, and (2) the ability to make comparisons between various subgroups: each of the 28 communities, units with school-age children, units with elderly members, and units with Hispanic members. This sample size was selected to measure the overall access to the Internet with—at most—a 1.9-percentage-point error rate and a 95-percent confidence level.

Reliability of Estimates. Estimates of percentages (such as the percentage of public housing units with high-speed Internet connection) will have 95-percent, two-tailed confidence intervals of between 0.3 and 1.9 percentage points, and the community-specific estimates will have 95-percent, two-tailed confidence intervals of between 5.9 and 9.8 percentage points. The confidence interval becomes larger for subgroup estimates. For example, with a sample size of 100 in a particular subgroup, the survey will be able to detect differences in percentages of larger than 9.8 percent, as compared to the sample as a whole.

Estimation Procedures. The primary purpose of the Internet access survey is to assess the high-speed Internet access of public housing units’ in the 28 ConnectHome communities. We will calculate the final survey response rate and adjust the initial sample weights for nonresponse based on relevant variables available from the public housing database from which the sample frame was drawn. Following data collection, sample weights for public housing units will be prepared and adjusted to account for 1) the initial probability of selection, 2) unit nonresponse, and 3) multiple selection opportunities (if needed). The product of these three weights will result in final weights suitable for use in analysis of responses. This weighting scheme inflates the respondents' data to represent the entire universe of public housing units targeted for ConnectHome services in the 28 communities.

Using Microsoft Excel software, electronic tallies of survey responses by disposition code and resultant response rates by ConnectHome site will be prepared. Next, we will populate table shells, analyze the survey data, and provide written survey results (using charts/graphs and cross tabulations, as appropriate). Subgroup data will be presented as sample sizes allow.

B3 Maximizing Response Rates

Our goal for the internet access survey is to achieve an overall response rate of 50 percent. We feel that this is a likely response rate for this survey for numerous reasons. First, we plan to use a proven data collection methodology (mailer survey with locating and follow-up with second mailing to nonrespondents). Second, to reduce the respondent burden, we have kept the length of the questionnaire to a minimum (5 minutes). Third, stakeholder organization EveryoneOn has provided Chromebooks for a lottery anrticipating households, which HUD anticipates will significantly boost participation. PHAs are also planning to perform targeted outreach to households identified as part of the sample. Participation in the baseline survey is voluntary.

B4 Tests of Procedures or Methods

The baseline survey was designed by HUD’s Office of Policy Development and Research with in consultation with HUD’s Office of Public and Indian Housing; HUD’s contracted research organization, Insight Policy Research; EveryoneOn, a national nonprofit working to eliminate the digital divide; and the communities participating in ConnectHome. These stakeholders reviewed to ensure the instrument is clear, flows well, is as concise as possible, and covers the essential baseline information. At the recommendation of OMB/OIRA a pre-test was done on the instrument designed in December 2015. We administered to a small sample of residents at the Washington, DC housing authority (DCHA). This same instrument was also administered from January 2015 to June of 2016 to a random sample of ConnectHome residents in the 28 targeted communities.

B5 Statistical Consultation and Information Collection Agents

HUD’s Office of Policy Development and Research is designing the evaluation in coordination with communities and HUD’s contractor, Insight Policy Research. Ronald M. Hill (202) 402-7073 will serve as GTR and Chase S. Sackett (202) 402-2405will be the GTM over the contractor with the Program Director: Carol S. Star who is Director of the Program Evaluation Division in PD&R (202) 402-6139. Insight team members have experience in qualitative and quantitative research, survey design, and data collection and analysis related to low-income populations.

2