Final - 11_Follow-up Site Visit Interviews Treatment PHA Y5

Year 5 Interviews: Treatment PHAs

MTW Evaluation: Landlord Incentives

Last Modified: October 5, 2022

Introduction to Protocol for Reviewers

The PHA interview protocol is a guide for the interviewer to follow a particular approach to interviewing. The interviewer guides the conversation using questions listed in the protocol, following up with probes (shown in bullets) as needed to gather more detail. Interviews will be conversational; the interviewer and interviewee may speak about these topics out-of-order, or the interviewee may address multiple questions in a single response. Interviewers will also have access to information about each PHA prior to the interview, so they can tailor the questions as needed to make the questions more relevant and targeted (for example, asking about a specific incentive or characteristic of the local rental market).

In Year 5, we anticipate interviewing up to 5 key staff at each PHA, such as the Executive Director, Manager of the HCV Program, MTW Coordinator, Landlord Liaison, and Housing Specialists. These interviews will take place in-person during site visits.

On average, we expect interviews to be approximately 60 minutes. Interviews will vary in length according to the information the interviewee has to share; some may be longer than 60 minutes if the interviewee has a lot of information to share, while others may be less than 60 if the interviewee has less experience with the program. The estimated time for each section currently adds up to more than 60 minutes. Interviewers will be trained to make trade-offs and prioritize some questions over others during the interview in order to manage the interview timing.

The following interview protocol will be piloted and adjusted accordingly to eliminate any questions which do not yield useful information. Interviewers will be trained to keep a core focus on the goals of the interview and manage the interview time accordingly.

Note to Interviewer

Note to Interviewer: Before the interview, you should carefully review the Site Summary, which collates information from the MTW Supplement, pre-site visit survey responses, and admin data. Use this information to tailor the questions below accordingly. You should pay close attention to the challenges that they identify, the incentives that they plan to implement, and the way their incentives are targeted or tailored to their market. You should also make note of any Agency-Specific Waivers listed in the MTW Supplement that may lead to variations in implementation of incentives (for example, agencies may seek a waiver to include special programs like HUD-VASH or Mainstream Vouchers in their MTW program).

Interviews will be held with PHA staff potentially including the executive director, director of the HCV program or rental assistance programs, MTW Coordinator, housing specialists, or landlord liaisons. One-on-one interviews are strongly preferred, but group interviews (ideally with staff with similar roles) can be conducted if there are scheduling challenges that require it.

Depending on their role and their length of time with the organization, interviewees may not be able to comment on all of the questions or sections included here. For example, the executive director and program director will likely be able to answer all these questions; front-line staff like housing specialists may not be able to answer questions about program management. If the interviewee indicates that they are not the ‘right person’ to speak about a specific topic, move on to the next section. You should focus on the questions that are most likely to be relevant for the interviewee based on their role with the agency.

Paperwork Reduction Act – Burden Statement

Public Reporting Burden for this information collection is estimated to average 60 minutes per respondent, including time to review the information in the Study Overview and Introduction. The MTW Evaluation: Landlord Incentives is a study of the implementation and impacts of providing selected incentives to landlords to participate in the Housing Choice Voucher (HCV) program. This information is being collected to document implementation of PHA activities that could affect landlord participation in the HCV program. PHAs in the MTW Landlord Incentives cohort are required to respond. The information requested is protected and held confidential in accordance with 5 U.S.C. § 552a (Privacy Act of 1974) and OMB Circular No. A-130.

Study Overview & Introduction

Thank you very much for taking the time to speak with me today. HUD is collaborating with Abt Associates to understand the impact of landlord incentives. The purpose of the study is to understand how PHAs are implementing landlord incentives, and how these incentives affect program outcomes. We’re interviewing all the PHAs that are implementing landlord incentives through MTW.

These questions have been reviewed by the Office of Management and Budget [OMB#]. When your agency received MTW designation, they also agreed to participate in the evaluation – so, your participation in this interview today is required. There may be some questions that you are unable to answer or that are more appropriate for other staff. If you are unable to answer a question, just let me know and we’ll move on to the next one. We expect this interview to take about one hour.

We will be taking notes and recording this interview. The notes, recording, and transcript of this interview will not be shared outside of the study team. The information you share with us will be used for research purposes only, not for any audit or compliance purposes. The study reports will summarize the findings from these interviews and from other data collection activities. We won’t name specific individuals in our reports and publications, but we might name the PHAs who participated in the study, so it’s possible that someone who is familiar with the PHA would be able to tell who was interviewed. 

Do you have any questions before we begin? [pause for questions]

Do you consent to participate in this interview? [get verbal consent before proceeding]

Is it OK if I turn on the recorder now? [get verbal consent before proceeding]

The tape recorder is now on. This is [interviewer name] speaking with [interviewee name] from [PHA name] on [date].

Interview Questions

Context and Background

1. If different person from baseline interviews: Can you tell me a bit more about your role with the HCV program? How long have you been in this role? Do you support any other PHA programs?

2. If same person from baseline interview: Last time we spoke, your role was [describe]. Has that changed at all since the last time we spoke?

3. Tell me a little more about the current rental market here. What does the market look like right now? (tight vs. loose, costs, size and types of units, quality of units)

4. Tell me more about the kind of units that someone with a voucher normally finds in this market right now. Is it easy or tough for them to find a unit? What types of units do they tend to rent? (e.g. single-family/multifamily/mobile homes, size of building, number of bedrooms, quality, neighborhood, tax credit or other subsidized properties)

5. Do some types of households have tougher time successfully leasing with a voucher? If so, which types and why?

6. Are there any laws here that prohibit discrimination against tenants using vouchers? Can you tell me about them?

   • If jurisdiction has SOI protections: How do you think that affects voucher holders? Does it make it easier for them to find a unit?

7. Since [year of baseline interview], been any changes at the PHA that have affected the HCV program? (e.g. leadership changes, new programs, changes in funding or capacity.)

8. Since [year of baseline interview], have there been any changes in the community that could affect the HCV program? (e.g. market changes, new housing programs, legislation)

9. In general, how are things going with the implementation of the landlord incentives? What were the highlights of the last year? What were the challenges?

Implementation of MTW Incentives

I have a list of the incentives you’ve implemented, based on your most recent MTW Supplement. I’d like to walk through each of these and ask you for a little more detail.

10. First, I see here that you are currently offering [list incentives]. Can you tell me if any of these incentives have changed since you submitted the latest Supplement? Is there anything missing from this list, or anything that I mentioned that has been discontinued?

[Repeat for each incentive. For new or discontinued incentives, skip to Q11/Q12.]

11. Tell me a little more about the process of implementing [incentive]. What was that process like?

    • What has worked well with implementing this incentive?

12. Tell me more about any challenges you have had in implementing this incentive.

    • If you have been able to, how have you dealt with those challenges?

    • If you have not been able to, what is preventing you from dealing with those challenges?

13. Was the incentive implemented as initially planned?

    • Were there any changes in the targeting or eligibility? If so, what led to that change?

    • Were there any delays or changes to the timeline? If so, what led to that change?

    • Have there been any other changes to this activity over the past few years? If so, what led to those changes?

14. How did landlords respond to [incentive]?

    • Was the reception different among different landlord segments or neighborhoods?

    • Did landlords have questions or concerns about [incentive]? If so, what were they? How did you address them?

    • Did you hear any other feedback from landlords about the incentive?

15. What about tenants – how did they react to [incentive]?

    • Have the incentives influenced tenants’ experiences of the program?

    • Were there any differences in the reaction between different types of tenants?

    • Did tenants have questions or concerns about [incentive]? If so, what were they? How did you address them?

    • Did you hear any other feedback from tenants about the incentive?

16. Overall, would you say [incentive] had the impact you hoped it would? Why do you think it did/did not? (e.g. landlord willingness, participation, success rates, mobility, tenant experience)

    • Did [incentive] have any other impacts in terms of program performance or other aspects of the program?

17. If the PHA appears to have added new activities since previous supplement: It looks like the PHA implemented [new activity]. What made you decide to add this incentive?

    • Which landlords/programs/areas are eligible for this incentive? How did you decide how to target this incentive?

    • What is the timeframe for implementation?

    • Which staff will be involved? Will any partners be involved?

    • Do you anticipate any impacts on program operations (e.g. staffing, costs, revenues)?

18. If the PHA appears to have discontinued some activities since previous supplement: It looks like the PHA no longer plans to implement [incentive]. Can you tell me why you decided to discontinue this activity?

19. Are there any other changes to how your agency has used its MTW authority that we haven’t mentioned? If so, what are they?

Changes in program and landlord behaviors

Now that the landlord incentives are coming to a close, I’d like to ask you more about how the incentives have changed the HCV program and landlord behaviors. These could be positive, negative, or neutral changes.

20. First, I’m interested to hear whether there are any changes among landlords who are already in the program. Overall, do you think these incentives have changed landlords’ experiences with the program? Can you give me a couple of specific examples of any changes you’ve observed?

    • Have you seen any changes in your relationships with current landlords?

    • Have you seen any change in landlords’ willingness to continue to accept vouchers?

21. Now I’m interested in learning more about whether the changes in the HCV program have led to changes in the way you engage new landlords. I’m interested in hearing a couple of specific examples of changes you’ve seen.

    • Have there been any changes in the number, type, or location of landlords participating in the program?

    • Have you seen any changes in landlords’ perceptions of or attitudes towards the program?

    • Have you seen any changes in landlords’ perceptions of or attitudes towards the PHA?

    • Have you seen any changes in landlords’ willingness to accept vouchers?

22. Have you observed any changes (positive or negative) in the overall performance of the HCV program? [For each of the following areas, probe on what the staff person thinks might have caused those changes.]

    • Have you observed changes in success rate?

    • Have you observed changes in lease-up times?

    • Have you observed any changes in program costs?

    • Have you seen any other changes in program outcomes or performance?

23. What about tenants – how have the landlord incentives affected them?

    • Have you observed any changes in the lease-up process for tenants?

    • Have you observed any changes in the type or quality of homes available to tenants?

    • Have you observed any changes in the feedback that you hear from tenants about the units?

24. Were there any unanticipated consequences of the incentives? What were they?

    • How did they impact the program?

    • How did they affect the experiences of tenants or landlords?

    • How did you address these consequences? Did you make any changes to mitigate or manage these consequences?

25. Of all the landlord incentives that the PHA implemented, which one(s) do you think changed landlord behavior the most? Tell me more about that.

26. Of all the landlord incentives that the PHA implemented, which one(s) do you think changed landlord behaviors the least? Tell me more about that.

27. At the start of the evaluation, your team shared that your PHA goals were [insert goals]. Overall, would you say the PHA’s landlord incentive program was successful in meeting these goals? Can you share more about that?

Close-out of landlord incentives

28. How has the agency monitored the implementation and impact of these incentives?

    • What outcome or output measures have you collected about these incentives? Tell me why you decided to collect these measures.

    • Have you seen any changes in those outcome or output measures since implementing the incentives?

    • How do you use these measures to inform your program implementation?

    • Have you collected any additional data or feedback on the incentives and their impact on landlord engagement and lease-ups? If so, can you describe what data you are collecting and from whom?

    • How do you plan to use that data/feedback moving forward?

    • Will you continue collecting this data after the incentives end?

29. Do you have any plans to continue offering any of these incentives in some form after the Landlord Incentives cohort ends? If so, can you tell me more about those plans (how they are funded, administrative waivers, etc.)?

30. What is the PHA doing to prepare for the close-out of the incentives?

    • How is the PHA communicating with landlords and tenants about the close-out of the incentives?

    • What changes is the PHA making to its operations (staffing, funding, etc) in preparation for close-out?

    • Do you anticipate any challenges with the close-out?

31. Moving forward, do you have any other (non-MTW) plans for improving landlord participation? If so, can you tell me more about these plans?

Lessons learned

32. Reflecting on your experience with the program, if you were implementing it again, would you do anything differently? Would you have set different goals or priorities? Would you have designed or implemented the incentives in a different way?

33. What lessons have you and your staff learned about how to increase landlord engagement and participation in the HCV program?

• What structures or policies need to change within the PHA to make it easier to engage landlords?

• What structures or policies or laws need to change within the city or state to make it easier for the HCV program to recruit landlords to participate in the program?

34. Are there any other reflections you’d like to share about your experiences with the MTW landlord incentives?

Additional cost-related questions for data manager and/or program manager

Before we wrap up, I’d just like to confirm some of the information we gathered from our pre-site visit survey, your MTW Supplement, and HUD administrative data. [Note that they may be unable to provide this information during the interview. If so, ask them to share information by email and follow up as needed.]

35. First, I’d like to confirm your agency’s success rate.

• If success rate data is available from survey or admin data: I see that you reported [#] regular tenant-based voucher issuances between [DATE] and [DATE]. Of those, you reported that [#] successfully leased up within 180 days. Do those numbers seem correct to you? If not, do you think they are too high or low? What are the correct numbers?

• If agency has missing or incomplete success rate data: We are looking for two key metrics: the number of regular tenant-based voucher issuances between [DATE] and [DATE], and the number of those issuances that resulted in a lease-up within 180 days. This excludes special program vouchers like HUD-VASH, Mainstream vouchers, or EHVs. Are you able to provide that information for our study?

36. For signing bonus only: I’d also like to confirm some data related to the signing bonus.

• If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] signing bonus payments in total, averaging [$X] per payment. Does that sound right?

• If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of signing bonuses you’ve issued in the most recent fiscal year, and the total dollar value of those bonuses. Are you able to provide that information for our study?

37. For damage deposit only: I’d also like to confirm some data related to the damage reimbursement incentive.

• If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] damage reimbursement payments in total, averaging [$X] per payment. Does that sound right?

• If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of damage reimbursements you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

38. For front-end vacancy loss only: I’d also like to confirm some data related to the front-end vacancy loss payments. This is the vacancy loss payments that are made when the unit transitions from non-HCV to HCV.

• If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] front-end vacancy loss payments in total, averaging [$X] per payment. Does that sound right?

• If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of front-end vacancy loss payments you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

39. For regular vacancy loss only: I’d also like to confirm some data related to the regular vacancy loss payments. This is the vacancy loss payments that are made when the unit transitions from HCV to HCV.

• If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] vacancy loss payments in total, averaging [$X] per payment. Does that sound right?

• If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of regular vacancy loss payments you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

40. For pre-qualifying inspections only: Since implementing your MTW program, how many pre-qualifying inspections has your agency completed to date? Of those, how many have not resulted in lease-up?

41. For initial inspection waivers only: Since implementing your MTW program, how many initial inspections have you waived to date?

42. If agency has any other missing data from survey or admin data: I noticed that we are also missing [information] from our information on your PHA. Could you provide that information for our study?

Thank you very much for your time today. Do you have any questions for me?

11