Final - Part B PHA Waivers Study

Evaluation of Public Housing Agencies (PHA) Coronavirus Aid, Relief, and Economic Security (CARES) Act Waivers:

Supporting Statement for Paperwork Reduction Act Submissions

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

On behalf of the U.S. Department of Housing and Urban Development (HUD), 2M Research is administering the evaluation of PHA CARES Act Waivers to understand how and why PHAs implemented CARES Act waivers during the pandemic. 2M Research plans to gather and analyze feedback from PHA stakeholders on the utility of these waivers on their general operations and assisted households.

2M Research plans to conduct three semi-structured interviews with three stakeholders (PHA leadership, PHA operations staff, and members of Resident Advisory Boards) from a sample of 45 PHAs that adopted waivers to better understand the successes, challenges, and changes resulting from the waivers. 2M Research will also conduct one semi-structured interviews with three stakeholders (PHA leadership and PHA operations staff ) from a sample of five PHAs that did not adopt any waivers to understand their reasons for taking this course of action. Please note that the interviews with PHAs that adopted waivers, we will conduct separate interviews between leadership and operational staff on a case-by-case basis, depending on the size and capacity of the PHA. For small PHAs with few staff members, for example, we will consider conducting one group interview with leadership and operational staff. In all, 2M Research plans to conduct up to 140 interviews for this study (three interviews per 45 PHAs that adopted waivers, and one interview per five PHA that did not adopt any waivers).

Exhibit A provides the sample description, expected sample size, and expected response rate for each data collection source. All of the data collection in this information collection report (ICR) will be done by 2MResearch. 2M Research expects to collect data from all the stakeholders identified above and shown in Exhibit A with a 100 percent response rate.

 Exhibit A | Sampling Plan

Primary data collection source	Sample description	Potential respondent universe	Expected respondent sample size per year	Expected response rate
Semi-structured interviews with PHA leadership from PHAs that adopted waivers	PHA’s executive leadership that were involved in the decision for the implementation of PHA waivers	Leadership staff with relevant information at the 45 selected PHAs that adopted waivers	45 (1 staff per PHA)	100%
Semi-structured interviews with PHA staff from PHAs that adopted waivers	PHA staff (such as PHA residents and tenants outreach staff or other staff that were involved or impacted by the implementation of PHA waivers)	PHA staff with relevant information at the 45 selected PHAs that adopted waivers	45 (1 staff per PHA)	100%
Semi-structured interviews with PHA’s Resident Advisory Board members from PHAs that adopted waivers	Members from the PHA’s Resident Advisory Boards	PHA Resident Advisory Board members with relevant information at the 45 selected PHAs that adopted waivers	45 (1 staff per PHA)	100%
Semi-structured interviews with leadership and staff from PHAs that did not adopt a waiver	PHA executive leadership PHA operations staff	Leadership and staff at PHAs that did not adopt any of the waivers offered by the CARES Act.	15 (one interview with up to 3 staff per five PHAs)	100%

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Below we highlight our plan to select a purposive sample of PHAs.

Sample Selection

We will select a purposive sample of 50 PHAs. In consultation with HUD, 2M Research will construct the sample using a stratified purposeful sampling procedure. The purpose of a stratified purposeful sample is to capture common core experiences and major variations within a fairly homogeneous sample. In line with this procedure, 2M Research will select the sample of 50 PHA sites based on several criteria including geographic regions, the size of the PHA, the types of programs administered, and Moving to Work (MTW) status.

The first stage of constructing the stratified purposive sample will be to define the population from which the sample will be taken. We have determined two groups of PHAs for the study:

1. PHAs that adopted at least one CARES Act waiver.

2. PHAs that did not utilize any CARES Act waivers.

While the total sample size for the qualitative study is 50 PHAs, we plan to select 45 PHAs that adopted at least one waiver and five PHAs that did not utilize any waivers. Below, we describe our criteria for selecting PHAs that utilized waivers and those that chose not to adopt a waiver.

Selection Criteria for PHAs that Utilized Waivers

We will divide the sample of PHAs that utilized the waivers into strata using HUD’s guidance in the CARES Act Waivers Reporting Tool (CAWRT) Dashboard User Guide (see Exhibit B). This will help ensure we are appropriately representing each stratum. Next, we will determine the sample size for each stratum. Additionally, we will strive to ensure diversity across the different categories of waivers offered under the CARES Act.

Exhibit B | Criteria for Identifying PHAs

Criteria	Strata
Region	Northeast Southeast Upper Midwest Southeast Midwest Far West and Beyond
Program Type	Combined (PHAs that have both Public Housing and HCS programs) HCV Only Public Housing Only
PHA Size*	Extra Small (0-249) Small (250-549) Medium (550-4,999) Large (5,000-57,999) Extra-Large (58,000 or Larger)
MTW Status	N Y Y-Expansion

Notes: *We may combine certain groups to make sure we have an appropriate number of PHAs in each stratum.

Selection Criteria for PHAs that did not Utilize Waivers

Our preliminary analysis shows that most (about 75 percent) of the PHAs that did not adopt a waiver are small PHAs, with the rest comprising medium and large PHAs. As a result (and because our proposed sample size for this group will be relatively small), our selection criteria for PHAs that declined to adopt a waiver would have favored small PHAs. After discussions with HUD COR and SMEs, we have determined to select three small PHAs, one medium PHA, and one large PHA for this group. We will aim to ensure that these PHAs are in close proximity to PHAs from the sample of PHAs that utilized a waiver. This ensures we are able to control the environment within which PHAs operate, especially given that there may have been local or regional conditions that may have influenced PHAs in similar ways, regardless of PHA size. This may have been the case especially during the early stages of the pandemic as localities had different policies on lockdowns, social distancing, and other COVID-19 prevention strategies that likely impacted PHAs and assisted households. Such an approach will offer limited but useful insights into why PHAs operating in the same localities or regions had different or similar outcomes because of the utilization of waivers.

We will maintain a list of up to ten alternative PHAs should any of the original PHAs be unavailable or unresponsive to our recruitment efforts; this will ensure we interview stakeholders from 50 PHAs within the data collection period. Once we select the sample, we will provide HUD with the list of selected PHAs and secure HUD approval before beginning the recruitment of the PHAs for the study.

Sample Recruitment

After finalizing the sample, HUD will provide the contact information or 2M Research will identify a point of contact for each sampled PHA (where possible). We anticipate this contact will be in a leadership position or a staff member from selected PHAs. To facilitate the recruitment of these PHAs, HUD will send an outreach email to each point of contact from the 50 selected PHAs. 2M Research will follow the outreach email by scheduling a call with each point of contact during which a 2M Research liaison will introduce the study, respond to any questions or concerns, and begin to work with them to identify other relevant stakeholders from the PHAs, such as PHA operations staff, members of Resident Advisory Boards, or other stakeholders that were involved or impacted by the implementation of PHA waivers. 2M Research will work with each PHA’s point of contact to identify the most relevant staff to answer the study’s research questions. For the members of the Resident Advisory Boards, 2M Research anticipates assistance from PHA leadership and operations staff to help recruit for the interviews. Following the introduction email from the PHA staff, 2M Research will send a scripted email to the members of the Resident Advisory Boards introducing the study and the nature of questions that we hope to ask them.

2M Research will designate two analysts as our lead points of contact for a minimum of 25 PHAs each, and they will serve as the primary points of contact for all respondents within each PHA to ensure the process is streamlined and coherent for stakeholders.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Every data collection instrument and question has been careful scrutinized to make sure it aligns with the research questions, is needed for the evaluation, and does not duplicate other existing data sources. 2M Research will also clearly communicate the reason for collecting the data to the respondents and make it as easy as possible for respondents to provide the data.

PHAs may have varying levels of interest in participating in the semi-structured interviews. The non-waiver PHAs may be least likely to want to participate—knowing that they did not adopt the CARES waivers and perhaps thinking they have nothing to offer the evaluation. Anticipating some reluctance on the part of PHAs, 2M Research has tried to minimize the burden of the primary data collection by only asking PHAs for data that is not reported to HUD elsewhere and using the least time intensive approach, using virtual meeting software, such as Zoom, Teams, etc. We also hope to incentivize PHAs’ participation by being clear up front about what 2M Research will need from the PHAs and by offering something meaningful to PHAs in return, such as explaining to them about the importance of this study to improving policies for PHAs and assisted households.

Each PHA will have a dedicated liaison from 2M Research who will be the primary point of contact for the PHA and will be responsible for conducting all data collection activities and achieving high response rates.

We will maintain a list of up to ten alternative PHAs should any of the original PHA sites be uninterested or unresponsive to our recruitment efforts; this will ensure we interview stakeholders from 50 PHAs within the data collection period.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

No pretesting will occur for this data collection. 2M Research will try to reduce respondent burden by reviewing all available information for each PHA (e.g., the CARES Act Waiver Reporting Tool [CAWRT] survey data as well as the administrative data collected by HUD). 2M Research will also pre-populate the interview guides with information collected from PHAs through HUD’s online CAWRT surveys and a review of HUD documents to minimize the respondent burden on PHA stakeholders.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

HUD contracted with 2M Research to conduct the data collection. Dr. Hiren Nisar (Principal Investigator), Dr. Dennis Okyere (Project Manager), and Dr. Lan Hu (Quantitative Analysis Lead) developed the statistical aspects of the design. HUD’s Contracting Officer’s Representative (COR), Justin Brock, reviewed all the procedures and had them reviewed by other subject matter experts at HUD. If there are any questions about this submission, please call either the HUD COR, Justin Brock (202-402-4348, or more frequently available by email at [email protected]) or the 2M’s Research Project Manager, Dennis Okyere (817-856-0892).

In addition, the 2M Research staff are supported by two subject matter experts, Eric Oberdorfer and Tushar Gurjal, from the National Association of Housing and Redevelopment Officials (NAHRO) who will review and provide feedback on all deliverables including the management and work plan, research design/data collection and analysis plan, instrument development, analysis of qualitative and quantitative data, and reports.