S811_PRA_OMB_Part_A_01072016_Revised_Clean

Section 811 PRA Demonstration Evaluation – Phase I

Office of Management and Budget Submission Part A – Justification

Office of Management and Budget Submission - Part A

Table of Contents

A. Justification 3

Circumstances that Make the Collection of Information Necessary 3

How and by Whom the Data will be Collected and Used 4

Use of Improved Technologies 8

Efforts to Avoid Duplication 8

Involvement of Small Entities 8

Consequences of Less Frequent Data Collection 9

Special Circumstances 9

Consultations Outside the Agency 9

Payment to Respondents 9

Arrangement and Assurances Regarding Confidentiality 9

Sensitive Questions 9

Estimate of Annualized Burden of Hours 9

Estimated Record Keeping and Reporting Cost Burden on Respondents 10

Estimated Cost to Federal Government 10

Reasons for Changes in Burden 10

Tabulation Plan, Statistical Analysis and Study Schedule 10

Exceptions to Certification 12

A. Justification

The Department of Housing and Urban Development’s (HUD) Section 811 program has historically provided funding to support the development and operation of project-based housing (group homes and independent living projects) for very low income people with disabilities. In 2010, the Frank Melville Supportive Housing Investment Act introduced a number of reforms to the Section 811 program, including the Project Rental Assistance (PRA) option, which provides subsidies for scattered site units located in affordable housing developments financed by other funding sources and occupied by a mix of people with and without disabilities. This new option puts emphasis on achieving the goals of the Olmstead decision to allow people with disabilities to live in the least restrictive settings possible that meet their needs and preferences. It also gives states opportunities to respond to incentives from the Affordable Care Act for rebalancing toward community-based housing and care.

The first round of Section 811 PRA Demonstration grants were awarded to, and are being implemented by, 12 state-level grantees. In each state, the lead housing agency (grantee) works in close collaboration with the agency administering Medicaid in the state. Grantees proposed a variety of service approaches based on their target populations and the services resources available in the state. Based on the service approach, a variety of other partners may be involved in the PRA program. Other partners include other state or local agencies, community-based service providers, property owners or managers, Public Housing Authorities, and centralized intake agencies.

The Melville Act requires that HUD report to Congress on the progress of the PRA Demonstration in January 2016. In February 2015, BCT Partners and their partner Abt Associates were selected to implement the first phase of what is expected to be a two phase evaluation of the PRA Demonstration for HUD’s Office of Policy Development and Research. The aim of the Phase 1 Evaluation is to document the early implementation of the PRA Demonstration in the 12 states that are implementing PRA programs under the grants awarded in February 2013.

In order to assess how the PRA Demonstration programs have been implemented across the 12 grantee sites and whether the programs are making progress toward their goals, this first phase of the evaluation includes three reports:

• Preliminary Outcomes Report;

• Case Studies Report; and

• Process Evaluation Report.

OMB approval is required for the data collection involved in the process evaluation component. The preliminary outcomes report will be based on administrative data reported to HUD through data collection instruments that have been previously approved by OMB, such as the Owner’s Certification with HUD Tenant Eligibility and Rent Procedures (HUD Form 50059, OMB Approval Number 2502-0204) and the Section 811 PRA Logic Model/Reporting Instrument (HUD Form 92241-PRA, OMB Approval Number 2502-0608). The case study data collection involves highly tailored discussions with representatives from six grantees and does not involve standardized data collection involving more than nine respondents.

Circumstances that Make the Collection of Information Necessary

The PRA evaluation provides an important opportunity to assess the early implementation and outcomes of a new approach to providing community-based housing and services for persons with disabilities. In the short term, the process evaluation will document the variety of approaches and tools the 12 states are using to implement the demonstration. Some states are using the PRA demonstration to build on established efforts to address Olmstead consent decrees or to enhance system reform efforts. Others will focus on the population targeted for the Money Follows the Person (MFP) program, which allows people in in-patient facilities to receive support services in their own homes, providing important new housing options that have been lacking in the past. The process evaluation will describe the planned and actual program structures, strategies the grantees developed to implement the PRA option, and ways the programs changed or evolved in the two years following grant awards.

The Process Evaluation will provide a descriptive overview of the planned and actual structures and strategies the grantees developed to implement the PRA option and the reasons, if applicable, programs changed or evolved. Much of the information for the study will come from in-depth, semi-structured interviews conducted with state agency and partner staff involved in the PRA Demonstration in each state. The study team will also review information in the PRA demonstration from grantee applications (in response to HUD’s 2012 Section PRA demonstration NOFA) and grantee administrative data. However, the application data may be dated and the administrative data only includes quantitative data that would be insufficient to adequately describe how the PRA demonstration was implemented.

How and by Whom the Data will be Collected and Used

Project Overview

This submission requests OMB approval for the process evaluation site visit interviews that will be conducted in January and February 2016. The process evaluation will include in-person, semi-structured interviews with multiple stakeholders in all 12 grantee states. The goal of the process evaluation is to describe the extent to which the demonstrations in each state are being implemented as described in the grant applications, and whether unit production and occupancy are proceeding as expected. The process evaluation will address the following key questions:

• How many units did grantees propose to assist and occupy, and what were the proposed characteristics of these units? How many units did grantees actually assist and occupy, and what are the main characteristics of these units?

• How many people did grantees propose to reach, refer and place in housing by target population? How many people did grantees actually reach, refer, and place in housing by target population?

• What are the characteristics of the partnerships between state housing and health and human services or Medicaid agencies?

• What is the PRA project cycle and what are grantee procedures to refer, place, and occupy PRA units? Where do delays occur?

• What services are being offered to program participants and how are the services managed and coordinated? What are the accountability measures?

• What are the program implementation’s challenges and successes and what approaches have achieved the Section 811 PRA program expected goals and outcomes?

The identification of respondents will rely on information in grantee applications, cooperative agreements between the grantee and the HHS or Medicaid partner, and from discussions with key grantee staff. The process evaluation data collection involves the following five steps:

Step 1: Review of Grantee Materials

To streamline both the screening calls and the process evaluation interviews, the research team will review all grantee materials (grant application, cooperative agreements, grantee quarterly reports, etc.).

Step 2: Initial Screening Calls to Grantees

Screening calls will be held with grantees approximately one to two months before the site visits. The calls will last approximately 60 minutes, and will:

• Identify key partners in the PRA Demonstration program;

• Describe the roles of key partners and identify key respondents based on the scope of the interview protocols;

• Confirm program status details such as the current number of occupied units, total number of referrals to program, number of referrals by source, and total number and location of leased units.

Step 2: Tailoring Interview Guides and Identifying Stakeholders

Based on information collected from background documents and screening calls, the research team will create a spreadsheet of key partners for each of the 12 grantees, identifying their organization or agency, role, and contact information. The study team will then tailor the interview protocols to each site, taking into account partnership structures, key partners, and program features.

Step 3: Scheduling Site Visits

Once the lists of interview respondents are created, the research team will send invitations to potential respondents and schedule interviews. We expect scheduling to begin approximately one month prior to site visits. If respondents are unable to meet in person during the scheduled site visit, we will schedule a time to interview them by phone.

Step 4: Conduct Site Visits

Purpose of Data Collection

The data collected will be used to compare and contrast program designs across states; identify differences between proposed approaches and the program as implemented at the time of the site visit; compare actual performance against planned performance, and the reasons for the variations; and identify themes in terms of design approaches, successes, challenges, and strategies to overcome challenges. Within each state, interviews will be conducted with the grantee agency, the Medicaid partner agency, and any other state agencies that have partnered with the grantee agency to implement the PRA Demonstration. In addition, at least three partner organizations will be interviewed for each grantee site. The anticipated outcome of each interview is described below.

• Grantee - Lead Housing Agencies: The interview will result in a comprehensive understanding of the grantee agency, its role in providing housing to extremely low-income non-elderly persons with disabilities, and the PRA program design and how it was developed. We will also identify how the PRA program is administered and monitored, including the financial management of the program.

• State Health and Human Service/Medicaid Agencies: Interviews with staff from Medicaid lead agencies will result in an understanding of the statewide Medicaid programs operated by the agency, the role of the agency in the PRA Demonstration, the types of services available to individuals while transitioning to and occupying PRA units, and the funding sources of these services. We will ask about how the state Medicaid agency coordinates and monitors services and resources for clients in PRA units through individual case management or other methods.

We will ask both the grantee and the lead Medicaid agency about reasons for participating in the PRA program and the process for developing the Inter-Agency Partnership Agreement, and determining each agency’s role in the program. We will ask how partners were identified and chosen and whether those identified in the grant application continue to implement the PRA program. We will ask the grantee agencies and lead Medicaid agencies about the history and extent of their partnership for providing housing and services for the target populations and about other partnerships that may exist between the two agencies and other partner agencies to determine whether the PRA program is building from existing state programs and partnerships.

Partner Interviews. The process evaluation site visits will also include interviews with key partners involved in identifying eligible clients, providing transitional and on-going services, and property management. These partners may be other state agencies or departments, community-based service providers, centralized intake and assessment entities, and property owners or managers. Because of the varied potential respondents, we developed interview guides by role instead of entity. Topics covered in the partner interview protocols are:

• Outreach and Referral to the Program: Interview questions on outreach and referral to the program will help to map applicant flow from learning about the PRA program through applying and being referred to the program. We will ask how clients learn about the availability of PRA assistance. Specifically, we will ask which organization or agency refers clients to the program and whether the lead agencies do any outreach. We will document the outreach process for each type of target population (i.e. applicants institutionalized or at-risk for institutionalization, homeless applicants or at-risk for homelessness, applicants leaving group homes) and the waiting list selection policy. We will ask about any pre-screening conducted prior to referral to the program. If there is a centralized statewide or community-wide referral system, we will ask how referrals to the program are coordinated. Interviewers will ask how the target populations were determined and whether the target population has changed as the program has been implemented.

• Participant Eligibility Determination: The study team will review when eligibility pre-screening happens, which agency does the pre-screening, what tools are used and how often people are screened out of the program. We will ask about the reasons households are found ineligible and the reasons applicants may decline the offer of a PRA unit.

• Service Provision and Coordination: The study team will interview at least two service providers for each grantee. Service providers will be asked to describe the services they provide to individuals while transitioning to and living in PRA units, how services are accessed, how services are funded, and the length and intensity of services. We will also ask service providers to describe how service needs are identified, how service plans are developed and monitored, and how service providers identify and address barriers to independent living. We will ask how the service providers work with the property managers or owners in the provision of services and how they work with the lead Medicaid agency in the coordination of services available to PRA tenants.

• Property Selection: The study team will ask about the site selection process for each type of housing strategy (e.g., providing PRA rental assistance for units in existing developments with affordable housing restrictions, or units in developments already or planned to be in the pipeline of the state’s LIHTC or other funding program). We will ask who is involved in site selection, the criteria for property selection, and the schedule for the selection process.

• Property Leasing and Management: The study team will also interview at least one property owner or manager of PRA-occupied units selected by the grantee agency. These interviews will not exceed 90 minutes and will include discussions about their role in the PRA program and their experience referring and transitioning people to housing, working with service providers, working with the target population, and adhering to the administrative requirements of the program. We will ask about property features such as range of unit sizes, accessibility, common areas, and services available to residents. We will also ask about challenges of participating in the PRA program.

All respondents, including the grantee lead agency and the lead Medicaid agency, will be asked about their role in the state outside of the Section 811 PRA program (populations served, programs administered, etc.) and their role in the PRA program. We will also ask about their reasons for participating in the demonstration and whether the planned role was the same as the implemented role.

Who Will Use this Information

HUD is the primary beneficiary of the planned data collection and will use the information from the study to understand the how the PRA demonstration has been implemented across grantee sites, as listed in section A.2.2. HUD will also use this information to inform Phase 2 of the PRA Demonstration evaluation which is expected to examine the effectiveness of the Section 811 PRA program compared to alternative housing options for non-elderly people with disabilities.

Instrument Item-by-Item Justification Chart

Exhibit A-1 describes the target respondents, content, and reason for inclusion for each data collection activity. Copies of the data collection instruments are provided as Appendices.

Exhibit A-1. Item-by-Item Justification of Data Collection Instruments

Data Collection Instrument	Content and Reason for Inclusion
Grantee Interview Protocol	Respondents: 12 grantee agencies. We expect more than one person in the agency to be responsible for the 8 grantee activities listed below for each state. If so, we will interview all people responsible for the grantee activities. Content: Section 811 PRA Demonstration Planning Overall Strategy for Application PRA Program Characteristics Partnerships and Organization Property Selection Tenant Participant Selection Outreach and Referrals to the Program Application and Eligibility Referrals to PRA Units/Owners Supportive Services Housing Transition Ongoing Supportive Services Grant Administration and Fiscal Oversight RAC Administration HUD Data Systems Leveraged Vouchers and Other Resources Program Monitoring, Reporting, and Oversight Reason: Interviewing the grantee agency is necessary to gain a comprehensive understanding of the administration of the PRA demonstration program, program structure, program scale, partner roles, whether program is being implemented as planned, and challenges and advantages of the PRA demonstration program.
HHS or Medicaid Partner Interview Protocol	Respondents: 12 lead Medicaid agencies in each of the grantee states. We expect more than one person at the lead Medicaid agency may be responsible for the 4 activities listed below for each site. If so, we will interview staff responsible for all of the activities listed below. Content: Section 811 PRA Demonstration Application Planning Overall Strategy Housing Strategy Targeted Population and Services Partnerships and Organizational Structure Tenant Participant Selection Outreach and Referrals to the Program Application and Eligibility Determination Referrals to PRA Units/Owners Supportive Services Housing Transition Services Ongoing Support Services Services Coordination and Funding Program Monitoring, Reporting, and Oversight Reason: HHS interviews are critical to understanding the service component of the Section 811 PRA Demonstration program.
Partner Organizations Interview Protocol	Respondents: 55 partner respondents across 12 grantee sites. Interviews will be conducted with 24 service provider agencies and 12 property owners or managers across 12 grantee sites. In 7 grantee sites, state agencies other than the grantee agency and Medicaid agency are partners to the inter-agency PRA demonstration agreement, and they also will be interviewed. We will also interview any other key partners identified by the grantee by phone if the respondents are not available to be interviewed in person during the site visit or if the interviewer learns of additional key partners during the site visit. Other key partners could include the source of coordinated or central intake and referrals, other state agency partners, community-based service providers, and other owners and landlords. On average, we expect one additional partner respondent in each site, bringing the total number of partner organization respondents to 55 across 12 grantee sites. Content: Role of Partner and Organization Outreach Referrals Eligibility Determination Housing Location Assistance Transition into Housing Assistance Supportive Services for PRA Participants Housing Transition Services Services Provided to PRA Tenants Service Coordination Services Funding Property Management Partner Perspective on the PRA Demonstration Reasons: Information collected from partner organizations is critical to understanding the implementation of the PRA Demonstration program. Partner organizations will provide critical insight into program effectiveness.

Use of Improved Technologies

Improved information technology will be used to organize the qualitative information collected through the site visit interviews in a way that allows for easier analysis. The study team will use an online database to support our analysis and to interpret and collate interview responses. The study team will use the database to assemble data from multiple sources in one place and organize the information using a coding tree.

Efforts to Avoid Duplication

To avoid duplicate data collection and data entry, the research team will use information from grantee application materials, quarterly reports, and other information submitted by the grantee—for example, administrative data on the characteristics of assisted units and rental assistance amounts —instead of collecting this information again. This will minimize burden on respondents and take advantage of existing data that are already entered in electronic databases.

Involvement of Small Entities

The research team may work with small state or community agencies that play key roles in the PRA Demonstration programs. Given that housing and services agencies are often small in size, it is likely that small entities will be included. The team will take great care to ensure minimal burden for all agencies—particularly those small in size—participating in this evaluation.

Consequences of Less Frequent Data Collection

Process evaluation interviews will be conducted a single time only.

Special Circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6

(Controlling Paperwork Burden on the Public, General Information Collection Guidelines). There are no circumstances that require deviation from these guidelines.

Consultations Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and Urban Development (HUD) published a 60-Day Notice of Proposed Information Collection in the Federal Register on July 1, 2015. The docket number was FR-5837-N-03. The Federal Register Notice appeared on pages 37649 and 37650. The notice provided a 60-day period for public comments, and comments were due by August 31, 2015. A copy of the notice is included in Appendix D.

HUD received comments for the proposed information collection from a state housing agency participating in the Section 811 Project Rental Assistance program and from one non-profit organization. The following highlights the major comments and concerns received and HUD responses. A copy of the comments received and HUD responses are included in Appendices E and F, respectively.

• One commenter inquired about the necessity of interview questions that were already spelled out in the Section 811 grant application.  The commenter suggested that the burden to grantees could be reduced if interviewers obtained information from the grantees’ application already submitted to the extent feasible. 

  • HUD response. This is indeed the intent of the proposed information collection.  Since this appears to be unclear in the data collection instruments, it was clarified in the revised interview protocols that interviewers will complete as much as the interview protocol as possible in advance of the site visit from available data sources including the 2012 Section 811 grant application for funding, quarterly grantee reports, and HUD administrative data. The protocols clarify that interviewers will only ask questions that are not filled in prior to the site visit or if the filled-in responses require updates or clarification. HUD noted, though, that this is a process evaluation and, as such, its purpose is to learn about changes that might have taken place between the applicants’ 2013 program design and the implementation taking place currently.   To that end, it is important for the interviewer to check if areas of the program design have changed, even though it may come at the cost of some amount of repetition.

• A commenter expressed concern about the insufficiency of the estimated time to collect the information.  The commenter came to this conclusion based on the amount of time that took four agency staff to complete Sections A and B of the Interview Protocol for Section 811 Grantees.

  • HUD response. HUD made two clarifications. First, the estimated time is an average, and as such, it is possible that interviews in some states, with more complex program structures, can take longer, while interviews in other states, with less complex program structures, might be shorter.  Second, grantees will not be asked to complete the interview protocol in writing.  Instead, the information will be collected through verbal responses which HUD anticipates will take substantially less time.  That said, in the interest of caution, and based on this feedback and additional information received, HUD added one hour per response to the estimate to ensure that HUD is not underestimating the burden of the proposed information collection.

• One commenter suggested that HUD considers sending the interview protocols to grantees ahead of site visits. 

  • HUD response. HUD has accepted this proposal and is sending interview protocols ahead of site visits. HUD is making clear that submission of the interview protocols is not meant to impose an additional burden on grantees. Submission of the protocols ahead of time is meant to help grantees understand the scope of the data collection and grantees are being encouraged to avoid completing the interviews in writing by themselves before the site visits. The interviews are designed to have interviewers draw as much information as possible from grant applications, using site visits to expand upon questions and verify changes to the original program design. The information will be collected through verbal responses which HUD anticipates will limit the estimated burden of collection on grantees.

• A commenter suggested that HUD considers interviewing other State Human Services agencies involved in the Section 811 Project Rental Assistance Program implementation and not only Medicaid partner agencies.

  • HUD response. This is indeed the intent of the proposed information collection. As described in the Office of Management and Budget Submission Part A “within each state, interviews will be conducted with the grantee agency, the Medicaid partner agency, and any other state agencies that have partnered with the grantee agency to implement the PRA Demonstration” (OMB Part A, pp.5).

• A commenter suggested clarifications, revisions and changes in terminology to the interview protocols to better align the interview to the program.

  • HUD Response. HUD accepted the suggestions and incorporated the proposed revisions and changes in terminology in the revised interview protocols.

Payment to Respondents

The respondents to the PRA Demonstration process evaluation will not receive payments for participating in the interviews, because the response burden is moderate and the respondents are agency staff.

Arrangement and Assurances Regarding Confidentiality

The confidentiality procedures followed for this evaluation will be appropriate to the nature of the respondents and the type of information sought. Lead agencies and partner organizations will be told that the information requested under this collection will be used for research purposes only and will not be used for compliance monitoring. Respondents will also be told before each interview that the research team will make every effort to protect your confidentiality, but that it is not possible to guarantee complete anonymity given the high level of HUD involvement in the PRA Demonstration effort. Individual respondent names will not be used in reporting what we have learned during the onsite visits, and responses will be combined with those of other grantee and partner respondents.

Prior to the start of the process evaluation site visits, the study protocols will be reviewed by Abt Associates’ Institutional Review Board (IRB) – the PRA Demonstration Evaluation is being implemented by a partnership between BCT Partners (the prime contractor) and Abt Associates. Detailed plans for data security procedures are described below.

Data Confidentiality Protections

The process evaluation interviews will be conducted in-person. The information will be analyzed using an online database. Process evaluation interviews will be uploaded to the database and will use unique study identifiers to identify respondents so that interview responses can be matched to grantees or properties without the use of names or contact information. Every precaution appropriate to the type of information collected for this study will be taken to ensure that the data remain both secure and confidential.

Sensitive Questions

The process evaluation interviews do not include any questions of a sensitive nature.

Estimate of Annualized Burden of Hours

A total of 79 participants will participate in the process evaluation interviews across the 12 grantee sites. The interviews with grantee representatives will take an average of 6 hours. Interviews with HHS or Medicaid agency representatives and other state agency partners will take an average of 6 hours each. The estimated number of hours for the grantee, HHS or Medicaid partner, and other state agency may be spread across multiple respondents if more than one person is responsible for distinct activities related to the PRA Demonstration grant. Prior to the interviews, we will conduct screening calls with each grantee to tailor the conversations and identify participants to include in the process interviews.

The length of interviews with partner organizations will vary based on the roles they have in the PRA Demonstration. We expect the interviews to take between 120 and 180 minutes based on the responsibilities of each partner. Exhibit A-2 below provides a list of all data collection tools, the number of respondents per tool, and the burden hours of the interviews.

For the state housing agency staff and state health and human service agency or state Medicaid agency staff, researchers will administer interviews on the implementation of the Section 811 PRA Demonstration for an average of six hours. An additional 2 hours will be needed for agency staff to compile material needed on the PRA program in order to answer the research questions. The total burden for state housing agency and Medicaid respondents is 192 hours. The average interview for PRA Demonstration Partner Agency/Property Owner staff is 90 minutes long, with an additional hour to compile information needed to complete the answers to the interview questions. The total burden for PRA Demonstration Partners is 329.5 hours.

Exhibit A-2: Burden Hours by Respondent Type

Respondents	Number of Respondents	Average burden/response (in hours)	Average burden/ data collection (in hours)	Total burden hours
State housing agencies	12	6	2	96
Medicaid agencies	12	6	2	96
PRA Demonstration Partners	55	1.5	1	137.5
Total	79			329.5

Estimated Record Keeping and Reporting Cost Burden on Respondents

This data collection effort involves no recordkeeping or reporting costs for respondents other than the time burden to respond to questions on the data collection instruments as described in item A.12 above. There is no known cost burden to the respondents.

Estimated Cost to Federal Government

The total costs to the government of all other activities described by this submission is $66,270 if all contract options associated with these data collection activities are executed.

Reasons for Changes in Burden

This submission to OMB is an original request for approval; there is no change in the burden estimate.

Tabulation Plan, Statistical Analysis and Study Schedule

This information collection request is for in-person interviews of 24 lead agencies and up to 55 partner agencies across 12 grantee sites. The specific use of the data collected through these interviews is described below.

The analysis of the interview responses will focus on the following topics:

• Partner roles and staffing: What were the planned roles and responsibilities across partner agencies as described in grantees’ applications? Did the roles change during implementation? If so, what changes occurred and why? How are partners held accountable?

• Property selection: What approaches are being used for property selection, unit occupancy and leveraging in terms of existing versus new development, unit sizes, rent levels, incentive system in QAPs (Qualified Allocation Plans), or other factors? How have the approaches changed from what was proposed?

• Rent levels and subsidy assumptions: Did assumptions change from application to implementation? If so, how and why?

• Numbers of PRA units: Did the types and number of units supported by the program or assumptions for leveraging other resources change from application to implementation? What changed and why? What approaches or assumptions were effective? Were any approaches or assumptions problematic, and if so, why? How did the approaches affect property selection and occupancy?

• Service funding and coordination: How are service needs identified and services provided for different target populations? What services are provided for different target populations and how are these services funded? How do states coordinate the availability of services for PRA tenants? What worked and did not work and why? What changes were made to overcome challenges?

Use of Analytical Database. Responses to the interviews will be recorded into the interview protocol at the time of the interview, capturing the answers in response to open-ended questions as close to verbatim as possible. Site visitors will review the completed interview guides as soon as possible after the interview to ensure all questions were addressed and responses are clear. Once final, the responses in Word will be uploaded into the study’s database.

Using the typed interview responses, the research team will organize and analyze responses within the study database. The database will allow researchers to generate reports that group both close-ended and narrative responses to individual study questions and to combinations of questions across all of the interviews. The research team will create codes for questions or themes so they can be sorted quickly across all interviews and then synthesized by the research team. For example, analysis codes may be created for any of the questions in the interview protocol, as well as for specific topics that the research team wants to analyze across the responses to different interview questions.

Some examples of how data will be compared by theme include:

• The use of centralized or coordinated referral systems, and whether they are effective in meeting occupancy goals;

• Strategies leading to faster occupancy as identified by respondents, and any variance by target population, referral source, or location of housing units;

• Differences in program designs and outcomes for PRA programs in states with Olmstead plans or settlement agreements;

• Coordination with other state and local efforts to serve people with disabilities in the community, and whether that led to differences in the characteristics of individuals living in PRA units;

• Service delivery approaches for different target populations.

• The use of existing housing or service models and whether it led to differences in PRA outcomes; and

• Challenges and successes of the different types of housing strategies.

Analyzing the Interview Responses. Creating the analysis codes will be an iterative process. The team will begin by creating codes based on broad categories of research questions the evaluation is intended to answer and implementing that coding in the study database. After members of the team have implemented the coding and begun to pull information from database to answer the research questions, the team will meet to discuss what they are finding and to identify additional codes and sub-codes that may be useful for identifying patterns of interview responses. This additional coding will then be implemented, and the analysis will proceed, possibly followed by a further round of discussion and additional coding of the data.

Study Schedule

Under the current schedule, process evaluation interviews will be conducted over a two month period in January and February 2016. Commencement of the site visits will follow the initial screening calls to lead housing agencies.

All data collection instruments will prominently display the expiration date for OMB approval.

Exceptions to Certification

This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 CRF 1320.9).

105 Lock Street

Suite 207

Newark, NJ 07103
(973) 622-0900 (Phone)

(973) 622-0655 (Fax)

[email protected] (E-Mail)

www.bctpartners.com (Web)