S811_PRA_OMB_Part_B_07022015

Section 811 PRA Demonstration Evaluation – Phase I

Office of Management and Budget Submission Part B – Statistical Methods

Office of Management and Budget Submission - Part B

Table of Contents

B. Statistical Methods 4

Potential Response Universe 4

Statistical Methods 4

Maximizing Response Rates 5

Tests of Procedures or Methods 6

Statistical Consultation and Information Collection Agents 6

B. Statistical Methods

Potential Response Universe

The Section 811 PRA Demonstration Process Evaluation will include the entire universe of the 12 state grantee sites. The contractor will select up to 79 total lead agency representatives and key partners to participate in the interviews, averaging between 6 and 7 respondents per site.

B.1.1 Study Participants

The primary respondents from each grantee state will be the lead housing agency (grantee agency) and the lead health and human services agency (Medicaid agency). The specific respondent from the grantee agency will be determined based on who is identified in the grant application as the primary contact. However, in many instances the research team anticipates that multiple respondents may be necessary to complete the interview. Particularly in larger agencies, it is likely that a senior administrator will complete some parts of the interview, but other staff from within the agency may need to complete the parts of the interview related to their respective programs. Thus, it is possible to have from one to three respondents per lead agency. For the lead Medicaid Agency, we will similarly contact a senior administrator or other lead person identified in the grant application. Again, it may be necessary to include up to 2 other staff persons to complete portions of the interview related to their particular agency functions. Additional interviews will be conducted with respondents from other state and/or local agencies that are implementing the PRA Demonstration and with up to three service providers and/or referral sources and with one property owner or manager.

Statistical Methods

B.2.1 Sampling Plan

Interview respondents will be selected purposively. Given the nature of the research questions for the study and the resources available for data collection, the research team will seek a limited set of respondents who can best describe how the demonstration was expected to operate and how it is actually being implemented. Upon completion of the initial screening of the lead housing agency, it may be clear that particular service providers or property owners/managers are better candidates for process interviews than others. The research team will select a small sample of service providers, referral partners, and property owners/managers for interviews. To identify knowledgeable partner representatives for interviews, the research team will identify the extent to which the partner is involved in the Section 811 PRA Demonstration program using indicators such as the number of referrals made to the program, number of agency clients in the PRA program, number of PRA units provided for the demonstration, number of PRA units under Rental Assistance Contracts (RAC) or number of housing placements made for clients in the program. In order to maximize response to the process interviews while still allowing for some refusals, the team will also identify a number of partners to be used as replacement interviews if necessary.

B.2.2 Justification of Level of Accuracy

The 79 interviews will gather qualitative information about each agency’s role in the state, role in the PRA Demonstration program, existing partnerships, motivations for participating, challenges and barriers to program success, programmatic design characteristics that are associated with greater success, and ways in which the program implementation differs from the planned version of the program. This number is based on information from grant applications that identify potential respondents and the content of process evaluation interviews that align with particular roles.

B.2.3 Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems associated with this sample. The study seeks responses from all grantee sites.

B.2.4 Any Use of Periodic (less frequent than annual) Data Collection Cycles to Reduce Burden

Not applicable to this study.

Maximizing Response Rates

Maximizing the response rate to the process interview requires an approach that acknowledges the challenges of reaching busy program administrators and staff and encourages the participation of all grantee sites.

Following the completion of the initial screening calls, the research team will compile a database of key contact information for all potential lead agency and key partner respondents.

The approach to completing the process evaluation interviews has five key steps:

1. Review of grantee information.

2. Initial screening call to grantees.

3. Tailoring the process evaluation instruments to each grantee site.

4. Sending invitation emails and scheduling site visits.

5. Attending site visits and holding interviews.

Invitation emails will be sent to all lead representatives and key partners identified as critical to the process evaluation. This invitation will describe the study effort and indicate that the researchers may need to speak to more than one respondent. The actual interview time for the process evaluation interview depends largely on role. The study team estimates five hours for the grantee agency, five hours for the HHS/Medicaid agency, and between one and two hours for each partner agency. Within three business days of sending the invitation, the site visitor will email each potential respondent to introduce themselves and alert respondents that they will be calling to schedule interviews. The email will also contain research team contact information, in case Section 811 PRA Demonstration partners have questions. Within 48 hours of the email, interviewers will begin telephone outreach efforts to schedule site visits and interviews. If identified key partners are unable to be interviewed during the timing of the site visit, the research team will identify another partner that addresses that particular role in the program or will schedule telephone interviews with key partners at another time.

Tests of Procedures or Methods

HUD personnel and staff from BCT Partners and Abt Associates reviewed draft versions of the process evaluation interview protocols. Their comments are reflected in the versions of the instruments included in this package.

Since the interview guides will be tailored to each individual respondent, we will not pretest the interview guides with potential respondents before conducting the site visits. We will however revisit and review the data collection instruments after the first visit and before conducting remaining visits.

Statistical Consultation and Information Collection Agents

HUD’s Office of Policy Development and Research will work with the contractors, BCT Partners and Abt Associates, to conduct the proposed data collection. Teresa Souza in HUD’s Office of Policy Development and Research, Program Evaluation Division, serves as Government Technical Representative (GTR). Within the research team, La Tonya Green and Gretchen Locke (the study’s Principal Investigators) and Dr. Jill Khadduri (the study’s Project Quality Advisor) contributed to the review of the Section 811 PRA Demonstration process evaluation interview protocols and this information request.

105 Lock Street

Suite 207

Newark, NJ 07103
(973) 622-0900 (Phone)

(973) 622-0655 (Fax)

[email protected] (E-Mail)

www.bctpartners.com (Web)