FINAL_OMB Supporting Statement_Part B_Follow-up Survey_1_11

B. Statistical Methods

B.1 Potential Response Universe

A sample of 125 public housing agencies that responded to the PHA census will be invited to participate in follow-up interviews for the Study of PHAs Engagement with Homeless Households. The contractor will select 75 PHAs that have homeless preferences and 50 PHAs that do not. The balance between PHAs with and without preferences may be revised based on findings from the census. The Final Report, and all other publications resulting from this research effort, will clearly note that the findings from the telephone survey are based on a purposive sample of 125 public housing agencies, and therefore, the results cannot be generalized to a broader population. Findings from the telephone survey are intended to be descriptive and illustrative in nature.  

B.1.1 Study Participants

The primary respondent from each PHA will be the Executive Director. However, in many instances the research team anticipates that multiple respondents may be necessary to complete the follow-up interview. Particularly in larger agencies, it is likely that the Executive Director will complete some parts of the interview, but staff from within the Public Housing and Housing Choice Voucher programs may need to complete the parts of the interview related to their respective programs. Thus, it is possible to have from one to three respondents per PHA.

B.2 Statistical Methods

B.2.1 Sampling Plan

Upon completion of the census data collection, HUD’s contractor will examine the data to select a sample for the follow-up survey. During this review, the research team will identify: whether the PHAs administer Public Housing, HCV, or both; which PHAs have homelessness preferences and which do not; whether PHAs with homeless preferences have formed partnerships with homeless assistance providers; and whether PHAs without preferences operate within CoCs that have large numbers of homeless people. Using sampling strata defined below, a sample of 125 PHAs will be selected. In order to maximize response to the follow-up survey while still allowing for some refusals, the team will also identify a number of PHAs in each sampling strata to be used as replacement cases.

The sampling strata will be based on a combination of these factors:

• Whether or not the PHA uses homeless preferences (including waitlist preferences and set-asides for public housing and/or vouchers);

• What type of homeless preferences the PHA uses—general preferences or partnerships with homeless service providers based on partnerships;

• Whether the PHA has preferences in Public Housing only, in HCV only, in both Public Housing and HCV programs, and through other programs serving homeless households;

• Whether or not the PHA modifies screening rules to make it easier to service homeless households;

• Whether or not the PHA administers other programs to serve homeless households;

• Whether or not the PHA is in one of the 25 priority communities for HUD technical assistance;

• The size of homeless population in the CoC that includes the PHA's jurisdiction; and

• The PHA’s program size (number of HCV and public housing units).

The research team will purposively select 75 homeless-engaged PHAs that together identify a range of approaches to serving homeless people. Selection of the remaining 50 PHAs in the sample will focus on agencies in areas with large homeless populations that through the census reported they have no preferences, nor any other targeted engagement with homeless families. HUD may also prioritize PHA jurisdictions that fall into one of 25 HUD-identified priority communities for technical assistance. It would also be desirable to select four to six PHAs that have Moving to Work (MTW) designation from HUD.

Even with purposive selection, the research team will use a formal process of allocating PHAs to sampling frame cells by the final criteria and nominating agencies by cell for inclusion in the sample. The table below provides an initial approach for allocating the sample based on the factors listed above and type of housing programs operated by the agencies.

Table B.1 Initial Sampling Approach for Purposive Sample

	Priority Community or Large Homeless Population	Other large PHAs—aim for geographic diversity
Strong general preference or set-aside of vouchers or public housing N=75
Strong general preference	15	10
Set-aside of vouchers	15	10
Set-aside of public housing	10	5
Alters screening	5	5
No strong general preference or set-aside of vouchers or public housing N=50
Vouchers and public housing	20	10
Public housing only	2	2
Vouchers only	5	5
Other programs serve homeless	3	3

B.2.2 Justification of Level of Accuracy

The 125 follow-up telephone surveys will gather qualitative information about each agency’s preferences, set-asides, and targeted programs for homeless households, and about the motivations, challenges and barriers in establishing them. In response to open-ended questions, the PHA Executive Director (or other designated staff) will confirm responses from the census survey and will be asked to provide additional detail based on those responses. The information will provide contextual reference and examples of the trends that are observed in the census survey results for all PHAs. The qualitative data will be analyzed according to categories or themes.

B.2.3 Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems associated with this sample. The study seeks responses from a purposive sample of Public Housing Agencies, to supplement information from the census previously approved by OMB.

B.2.4 Any Use of Periodic (less frequent than annual) Data Collection Cycles to Reduce Burden

Not applicable to this study.

B.3 Maximizing Response Rates

HUD’s contractor will administer the follow-up interviews with a two-person team—an interviewer and a research assistant. The interviewer will administer the questions and guide the respondent through the interview instrument, probing for additional information where appropriate. The research assistant will record all responses independently. This will ensure that there are complete notes on all items collected in the interview and will aid in the subsequent analysis stage.

Maximizing the response rate to the follow-up interview requires an approach that acknowledges the challenges of reaching busy PHA directors and staff and encourages the participation of the sampled PHAs.

Following the completion of the PHA census, HUD’s contractor will compile a database of key contact information for all PHAs that may be included in the sample and include the values of the variables needed to assign PHAs to sample strata for the follow-up survey. Once the sample is selected, these flags will allow the research team to tailor the initial outreach letter for the follow-up sample survey.

The approach to completing the follow-up interviews has four key steps:

1. Letter from HUD referring to the census and explaining the further purpose of the follow-up survey. The letter will encourage cooperation and alert PHA to expect contact from the HUD’s contractor team;

2. Email follow-up from the contractor to introduce the senior interviewer and explain more about the survey;

3. Phone outreach and scheduling; and

4. Survey administration.

The contractor will help draft a set of letters for the Assistant Secretary for Public and Indian Housing to send to PHA executive directors, introducing the second part of the study. The letters will thank the PHA for their participation in the census and explain the general nature of the follow-up interview. For example, one version would be for agencies with homeless waitlist preferences in their Public Housing programs; a different version would be for agencies with no public housing units but with HCV set-asides for homeless people. By tailoring the letters, the effort that the PHAs have already put into the study is acknowledged and shows that this second step builds on the information already provided. Letters will be generated for a substantially larger number than 125 PHAs, so as to have available letters for potential replacement PHAs in each of the sampling strata. Previous experience has shown that not all selected PHAs can or will agree to participate. The research team will predetermine a list of potential replacement PHAs in each stratum to try to maximize the response to this effort. Having a letter ready to go to replacement sample in advance will allow the replacement process to be expedited.

The introductory letter will also indicate that the researchers may need to speak to more than one respondent. The actual interview time for the PHA follow-up survey is estimated to be about one hour per PHA, on average. Within three business days of sending the introductory letter, the lead interviewers will email each PHA to introduce themselves and alert respondents that they will be calling to schedule interviews. The email will also contain study team contact information, in case PHA staff members have questions. Within 48 hours of the email, interviewers will begin telephone outreach efforts to schedule interviews, identify all potential respondents within the PHA, and then complete the surveys.

Staff will send up to four email reminders, to invited PHAs who fail to respond after the telephone outreach call. All non-responders after the second email attempt will receive a reminder phone call from the interviewer. Subsequent contact attempts will alternate between telephone and email invitation, until an interview is completed or a determination is made to move on.

This will be treated as a quota sample, with PHAs assigned priorities within the sample strata. Thus, if the researchers are unable to achieve a completed interview with a particular agency, they will move to begin outreach efforts with the next PHA in the same stratum. Once a replacement PHA is activated for the survey, interviewers will follow the same steps to engage the new respondent and complete the interview.

B.4 Tests of Procedures or Methods

HUD personnel, staff from Contractor Abt Associates, and other project team members, the U.S. Interagency Council on Homelessness, National Association of Housing and Redevelopment Officials and Council on Large Public Housing Agencies’ staff reviewed draft versions of the follow-up telephone survey. Their comments are reflected in the version of the follow-up instrument included in this package.

During the review period of the request for OMB clearance, the study team will complete a pretest with fewer than 9 PHAs. Feedback from the PHAs that participated in the pretest will be reflected in the final version of the request for clearance. At the time of submission of this document to OMB, the pre-test has been completed with 3 PHAs. Results of this pre-test thus far indicate that the time required to complete the survey is approximately one hour, as estimated. As a result of the pre-test, we have made wording clarifications to the survey.

B.5 Statistical Consultation and Information Collection Agents

HUD’s Office of Policy Development and Research will work with the contractor, Abt Associates, to conduct the proposed data collection. Anne Fletcher, a Social Science Analyst in HUD’s Office of Policy Development and Research, Program Evaluation Division, serves as Government Technical Representative (GTR). Within Abt Associates, Dr. Jill Khadduri (the study’s Principal Investigator) and Dr. Judith Feins (the study’s Project Quality Advisor) contributed to the review of the PHA follow-up interview and this information request.

B. Statistical Methods ▌pg. B-3