Interviews with MTW (Treatment Group) PHAs

Appendix A: Interview Guides for Interviews with MTW PHAs

A.1. Baseline Interview with MTW (Treatment Group) PHAs

Interview respondent is the PHA Executive Director and one or more designated staff. The interviewer will prepopulate information from the Baseline Survey and MTW Plan. The prepopulated information is highlighted in gray.

Introduction and Verbal Consent

Thank you very much for taking the time to speak with me. HUD has hired Abt Associates to study Cohort 1 of the MTW Expansion. The purpose of the study is to understand how PHAs in the cohort use their MTW flexibility and how MTW affects outcomes for the PHA and its tenants. We are interviewing all the PHAs that received MTW designation under Cohort 1.

Your participation in this interview is voluntary and you are free to skip any questions you do not wish to answer. The questions in the interview have been reviewed by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995. We expect the interview to take about an hour. The OMB control number is XXXX-XXXX, expiring XX-XX-XXXX.

The study team will use the information you provide for research purposes only, not for any audit or compliance purposes. We will be taking notes but will not be recording this call. Only members of the study team will see your individual responses. Our reports to HUD will summarize the results from the interviews but will not name individuals. If we would like to highlight your PHA in one of our reports, we will give you the opportunity to review the text in advance.

There may be some questions you may not be able to answer or that are more appropriate for other staff. If you are unable to answer a question or would prefer not to answer, just let me know. You are free to skip any question you do not wish to answer.

Do you have any questions before we begin?

Motivation for Applying for MTW

I’d like to start by talking about what motivated your agency to submit the initial letter of interest for the first cohort of the MTW expansion in [fall 2018 / spring 2019].

1. Can you remember how you heard about the first cohort of the MTW expansion?

2. Can you talk about what motivated your agency to submit the initial letter of interest? (This would have been in the fall of 2018 or early 2019.) Was it an easy decision to apply or did your agency deliberate for some time? (If not an easy decision: What were the factors you were weighing? What convinced your agency to apply?) (If respondent was not involved or cannot remember the application or baseline survey, skip to next section/Q8.)

3. Which statutory objectives were most important to your agency at the time of application? How did you prioritize among and within the statutory objectives of cost effectiveness, self-sufficiency, or housing choice?

4. Were there other local or PHA objectives – outside the three statutory objectives – that were important to your agency at that time?

5. Were there specific programmatic or operational challenges you were seeking to address in applying for MTW?

6. Were there specific outcomes or goals you were seeking to achieve in applying for MTW?

7. Did you have a clear view at the time of completing the Baseline Survey of what policy or program changes you wanted to implement with your MTW flexibility?

Moving from the Baseline Survey to the MTW Plan

Now let’s talk about the MTW plan that you submitted earlier this year as part of the second phase of the application process, and any changes that your agency made in your planned MTW program between the submission of the Baseline Survey and the MTW Plan.

8. Did your agency (management, staff, or board members) have any reservations about submitting the full application? If so, what were they and how did you overcome those reservations?

9. Can you walk me through the process that you undertook to develop the MTW Plan?

   1. Which PHA staff were involved?

   2. Did you seek input from residents?

   3. Did you seek input from other community stakeholders?

   4. To what extent was the Board involved?

10. (If there are objectives in the MTW Plan that were not in the Baseline Survey. Ask for each new objective.) It looks like X was not an objective at the time you complete the Baseline Survey but is in your MTW Plan as an objective. What motivated you to add this objective?

11. (If there are objectives that were in the Baseline Survey but are not in the MTW Plan. Ask for each removed objective.) It looks like X was an objective at the time you complete the Baseline Survey but is not in your MTW Plan. Why is your agency no longer pursuing this objective?

12. (If there are activities in the MTW Plan but not in the Baseline Survey. Ask for each new activity.) I see X activity in your MTW Plan that was not in your Baseline Survey. What motivated you to add this activity?

13. (If there are activities that were in the Baseline Survey but are not in the MTW Plan. Ask for each removed activity.) I see X activity was in your Baseline Survey but not in your MTW Plan. Why is your agency no longer planning to implement this activity?

Plans for Year 1 MTW Implementation

Now let’s talk more about your plans for the coming year. I would like to learn more about the MTW activities your agency has planned for its first year of MTW designation.

MTW Activities Related to Cost Effectiveness

If PHA is planning to implement activities associated with the cost effectiveness objective, ask:

14. What are the factors at your agency or in your community that affect program and PHA costs?

15. (If not addressed in previous questions.) How did your agency determine which activities to undertake with statutory objective of cost effectiveness?

16. Let’s discuss each of the activities that you are planning associated with cost effectiveness? I see X, Y, and Z in the MTW Plan. Can you tell me a little bit more about each of those initiatives?

(Complete for each activity:)

1. Details of the changes the PHA is planning to make and how they are expected to affect costs

2. Which housing programs the change affects

3. How it affects staffing

4. How it affects other aspects of PHA operations

5. Timeframe for implementation

6. How the PHA will track results or outcomes

17. Which of these agency-planned MTW activities do you think will have the biggest positive impact on costs? Why?

18. Do you think you will you be able to isolate cost changes that are a result of a specific MTW activity? If not, why not?

19. I see that you identified the following areas in your MTW Plan/Baseline Survey as areas for reinvestment of cost savings: [AREAS IDENTIFIED]. How did you prioritize these areas? Are there other areas that are important to your agency?

MTW Activities Related to Self-Sufficiency

If PHA is planning to implement activities associated with the self-sufficiency objective, ask:

20. How does your agency define self-sufficiency? Why do you consider that to be a good measure of self-sufficiency?

21. What are the factors in your community that affect residents’ ability to move toward self-sufficiency?

22. (If not addressed in previous questions.) How did your agency determine which activities to undertake with the statutory objective of self-sufficiency?

23. Let’s discuss each of the activities that you are planning to encourage self-sufficiency. I see X, Y, and Z in the MTW Plan. Can you tell me a little bit more about each of those initiatives?

(Complete for each activity:)

1. What are the changes the PHA is planning to make and how are they expected to affect self-sufficiency?

2. Which housing programs and populations will the initiative apply to?

3. If limited to specific public housing sites: How were those sites selected?

4. What is the timeframe for implementation and rollout for the initiative?

5. Does the PHA anticipate creating a hardship policy related to the initiative?

6. Does the PHA expect the initiative to affect PHA staffing or other aspects of PHA operations?

7. How the PHA will track results or outcomes for the initiative?

24. Among the activities your agency plans on implementing for self-sufficiency, which one do you think will have the biggest impact on self-sufficiency? Why?

25. What type of impact do you see these activities having on PHA and program costs?

26. Do you see these activities having any impact on housing choice for residents? If so, describe.

MTW Activities Related to Housing Choice

If PHA is planning to implement activities associated with the housing choice objective, ask:

27. How does your agency define housing choice?

28. How would you describe the extent of choice that your PHA’s residents or applicants have in where they live? What are the factors in your community that affect housing choice?

29. How did your agency determine which activities to undertake under the statutory objective of housing choice?

30. Let’s discuss each of the activities that you are planning to increase housing choices. I see X, Y, and Z in the MTW Plan. Can you tell me a little bit more about each of those initiatives?

(Complete for each activity:)

1. Details of the changes the PHA is planning to make and how they are expected to affect housing choice

2. Which housing programs and populations the change affects

3. Timeframe for implementation and rollout

4. Does it affect PHA staffing or other aspects of PHA operations

5. How the PHA will track results or outcomes

31. Among the activities your agency plans on implementing for housing choice, which one do you think will have the biggest impact on housing choice? Why?

32. What type of impact do you see these activities having on PHA and program costs?

33. Do you see these activities having any impact on resident self-sufficiency? If so, describe.

Local Non-Traditional Programs

If PHA is planning to implement local non-traditional programs:

Let’s discuss each of the local, non-traditional (LNT) programs you are planning to implement.

34. Why did your agency decide to use MTW flexibility to develop a LNT program?

35. I see X, Y, and Z in the MTW Plan. Can you tell me a little bit more about each of those initiatives?

(Complete for each initiative:)

1. Details of the changes the PHA is planning to make

2. Which housing programs and populations the change affects

3. Timeframe for implementation and rollout

4. Does it affect PHA staffing or other aspects of PHA operations

5. How the PHA will track results or outcomes

36. For each LNT program: How did you determine how many households you would serve with the program?

37. For each LNT program: How did you determine the type and amount of assistance to be provided to households through the LNT program?

38. For each LNT program: Are there any services required or offered as part of the program? Who provides those services?

39. For each LNT program: Are you partnering with any outside agencies for the LNT? Or does your PHA have affiliate organizations that provide the services? For each partner or affiliated organization,

1. Describe the partnership and how it came about.

2. What benefits does the partner receive as part of this partnership?

3. Are partners or any other organizations contributing funds or other resources to this program?

Funding Flexibility

40. How do you expect to use the MTW funding flexibility in the coming year?

41. What are your objectives in using your funding flexibility this way?

42. What results do you expect from the use of funding flexibility?

43. How will funding flexibility affect your agency’s ability to meet the MTW statutory objectives?

44. Do you expect to bring in new sources of funding as a result of your participation in MTW? If so, describe the sources of funding and expected uses.

45. Do you foresee any negative effects for your agency of funding flexibility? Perhaps for individual program budgets?

Evaluation

46. Is your PHA planning to do any evaluation of its MTW program, either internally or using a third-party evaluator? If so,

    1. What policies or programs will be evaluated?

    2. What data will be used to evaluate the program? Will the PHA be collecting any new data from residents or about the agency or properties for the purposes of the evaluation?

    3. What is the timeframe for the evaluation?

    4. How will information from the evaluation be shared within the agency and with the broader community?

Wrap Up

Thank you for your time today. We just have a few more questions.

47. Thinking ahead, which MTW flexibilities do you expect to be most critical to achieving the program’s three statutory objectives? Why?

48. [If the PHA is trying to meet other objectives] Which MTW flexibilities do you expect to be most critical to meeting the PHA’s other objectives? Why?

49. Looking forward to the next year, what challenges do you expect to face in implementing your MTW program?

50. Is there any other information you would like to provide about your MTW program or activities?

Thanks very much for your time today. We will speak again around this time next year. Do you have any questions for me?

A.2. Annual Interview with MTW (Treatment Group) PHAs

Interview respondent is the PHA Executive Director, MTW Coordinator, and/or one or more designated staff. The interviewer will prepopulate information from the Baseline Interview and MTW Supplement. The prepopulated information is highlighted in gray.

Introduction and Verbal Consent

HUD has hired Abt Associates to study Cohort 1 of the MTW Expansion. The purpose of the study is to understand how PHAs in the cohort use their MTW flexibility and how MTW affects outcomes for the PHA and its tenants. We are interviewing all the PHAs that received MTW designation under Cohort 1.

Your participation in this interview is voluntary and you are free to skip any questions you do not wish to answer. The questions in the interview have been reviewed by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995. We expect the interview to take about an hour. The OMB control number is XXXX-XXXX, expiring XX-XX-XXXX.

The study team will use the information you provide for research purposes only, not for any audit or compliance purposes. We will be taking notes but will not be recording this call. Only members of the study team will see your individual responses. Our reports to HUD will summarize the results from the interviews but will not name individuals. If we would like to highlight your PHA in one of our reports, we will give you the opportunity to review the text in advance.

There may be some questions you may not be able to answer or that are more appropriate for other staff. If you are unable to answer a question or would prefer not to answer, just let me know. You are free to skip any question you do not wish to answer.

Do you have any questions before we begin?

Introduction

1. In general, how are things going with your MTW program?

1. What were the highlights of the last year?

2. Did you experience challenges in implementing your program?

2. Have there been any changes at the PHA, such as leadership or Board changes, that affect the MTW program?

3. Have there been any changes in the community that could affect the MTW program?

4. Have there been any changes to the level of funding that you receive from HUD for your HCV and public housing programs?

5. Any other changes that could affect your MTW program and that I should be aware of?

Status of MTW Supplement

I see you have/do not have an approved MTW Supplement as of date.

6. [For PHAs without an approved MTW Supplement] What is the status of your MTW Supplement?

7. Can you walk me through the process your agency went through/is going through to develop the Supplement?

   1. Which agency staff were involved in the Supplement’s development?

   2. Was the PHA’s Board involved in the development? Did you make any changes to the Supplement as a result of consultation with the Board?

   3. Did you involve residents in developing the Supplement? Did you make any changes as a result of resident input?

   4. Did you involve any outside consultation (for example, community partners, local government, advocacy groups, general public)?

8. Have you experienced challenges developing the Supplement or getting it approved? If so, describe.

9. Is there training or support that you think your agency would benefit from in developing your MTW Supplement?

Changes in Program Goals and Activities Since MTW Plan [or last MTW Supplement]

Now I would like to review any changes that your agency made in your planned MTW program since the submission of your MTW Plan / latest MTW Supplement.

10. (If the PHA appears to have changed its objectives.) It looks like the PHA has changed its objectives somewhat, from X to Y? Do I have that right? What was the reason for the change?

11. (If the PHA appears to have added new activities.) It looks like the PHA has identified new activities to implement, X and Y? Do I have that right? Can you tell me why you added those activities? How do you plan to implement them?

12. (If the PHA appears to have discontinued some activities.) It looks like the PHA has decided not to implement activities X and Y? Do I have that right? Can you tell me why you decided not to pursue those activities?

13. (If the PHA is using agency-specific waivers) How did you determine that you would need to request agency-specific waiver(s) versus the existing set of waivers covered by the Operations Notice?

14. (If the PHA has gone beyond the existing safe harbors) How did you determine that you would need to go beyond the safe harbors in the Operations Notice?

15. Has the agency had any change in how it plans to use MTW’s funding flexibility? If so, describe the change and the reason for the change.

16. Has the agency had any change in the outside funds it plans to raise? If so, describe the change and the reason for the change.

17. Are there any other changes to how your agency plans to use its MTW authority that we haven’t mentioned? If so, what are they?

Implementation to Date of MTW Waivers and Activities

Interviewers will prepare a list of active, discontinued, and planned MTW waivers and activities based on the MTW Supplement and send it to the PHA in advance of the interview. The interviewer will ask the PHA to confirm or edit the list in advance of the interview and then will use the list to ask the following questions about any current or planned activities. For activities that continue from year to year, interviewers will build on the information collected the previous year.

18. Based on the chart I sent you and that you edited, it looks like you are continuing X, Y, and Z activities from last year and plan to implement A and B activities this year. How did your agency prioritize the activities to implement in the coming year?

Ask for each current or planned MTW activity:

19. Can you explain the connection between the activity and the primary statutory objective(s) [cost effectiveness/self-sufficiency/housing choice]?¹

20. Does the activity meet any other PHA objectives besides the statutory objectives?

21. Do you have any partnerships or affiliated nonprofits helping to implement this activity? If so:

    1. Describe the partner and type of organization.

    2. How did the agency come to develop the partnership?

    3. What is the nature of the partnership?

22. Can you describe what steps you have taken to implement this activity in the past year?

    1. Which agency staff have been involved?

    2. Will any contractors, community partners, or other third parties are involved?

23. What is the timeframe for implementing the activity?

    1. How confident are you that your agency will meet your schedule for implementation for this activity?

    2. What factors could affect your ability to meet the schedule or planned results for this activity?

24. Has the agency met its milestones for this activity in the past year?

25. If the PHA has experienced delays: What are the reasons for the delays? What steps has the PHA taken to address these challenges? What is the revised timeline for implementing your MTW activities?

26. Have there been any changes to how you are currently implementing this activity (compared to when you started)?

27. When do you expect the activity to show results related to the statutory objective(s)?

    1. What are the specific results you would expect to see?

    2. How are you measuring and tracking the results?

28. Why does the PHA think this activity will have a [positive/negative/neutral] cost implication? Have you seen any impacts on cost to date?

29. Has this activity had any effect on PHA staffing to date? If so, when did these changes happen? How are you measuring changes in PHA staffing? (For activities not yet implemented, ask about future expected effects.)

30. Has this activity changed how you interact with tenants or other aspects of your work as a PHA? If so, how? (For activities not yet implemented, ask about future expected effects.)

31. Will this activity or has this activity required any changes to any of the PHA’s policies or procedures in the PHA’s Administrative Plan (HCV) or ACOP (public housing)? If so, describe.

32. It appears that this activity has/does not have a hardship policy (refer to MTW Supplement). Am I correct? If needed: What is the hardship policy?

33. It appears that you received X hardship requests in the past year and approved Y requests? Is that correct? What are the main reasons requests are denied?

Local Non Traditional (LNT) Programs

For each LNT identified in the MTW Supplement, ask:

34. What are the goals of the program?

35. How do you plan to implement this program in the next year?

36. How will you select households for participation in the program? What are the eligibility requirements? Has this changed over time?

37. How much assistance can households receive? How was this amount determined? Has this changed over time?

38. How long can households can receive assistance? How was this determined? Has this changed over time?

39. Are there any services required or offered as part of the program? Who provides those services? Has this changed over time?

Lessons Learned and Wrap Up

40. Thinking back over your experiences with MTW thus far, which MTW flexibilities do you view as most critical to achieving the program’s three statutory objectives? Why?

41. [If the PHA is trying to meet other objectives] Which MTW flexibilities do you view as most critical to meeting the PHA’s other objectives? Why?

42. Can you identify any lessons that you have learned in implementing your MTW activities in the past year?

43. Is there training or support your agency would benefit from as you implement your MTW program?

44. Based on what you’ve done so far, what advice would you give a new MTW Agency?

45. Do you have any recommendations for how the MTW program could be changed to facilitate meeting its statutory objectives or your objectives as a PHA (if different)?

Thanks very much for your time today. We will speak again around this time next year. Do you have any questions for me?

1 It may not be necessary to go through this in all cases. Some activities will be associated with one objective only and the connection will be obvious.

16