Public reporting burden for this collection of information is estimated to average 2 hours for the HUD TA Assessment interviews. HUD may not conduct or sponsor, and a person is not required to respond to, a collection information unless that collection displays a valid OMB control number 2528-xxxx, expiring xx-xx-xxxx.
All the information we collect will remain confidential. The information requested under this collection is protected and held private in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130.
Estimated Length: 90 minutes
Introduction
Good morning/afternoon, I am [your name] and this is my colleague, [name]. We want to thank you for your willingness to talk with us today. As you may know, Congress has mandated an assessment of HUD TA programs funded through the Community Compass NOFA. HUD’s Office of Policy Development and Research (PD&R) has awarded a Cooperative Agreement to the Urban Institute to conduct an evaluation of HUD’s technical assistance (TA) programs. The evaluation focuses on the reorganization of the TA programs under the Community Compass umbrella and the tracking of awardees and activities in administrative data systems specific to HUD TA. The evaluation assesses what TA is provided, how it’s provided, and how effective it is. This is not an evaluation of your work or of your office.
For this evaluation project, we are conducting interviews with HUD Headquarters staff and field staff, along with a sample of TA providers and TA customers. Your participation is completely voluntary. If you agree to speak with us about your TA experience and program work, you may choose not to answer any particular questions. We also want to let you know that we will be taking notes of your responses so we can be sure to capture them accurately.We will not attribute your responses to you or name you in any reports. However, you should be aware that due to your position and program area, it may be possible for others to identify your comments despite our efforts.
We know that you are busy so we will focus our discussion as much as possible and promise to keep to the 90 minute timeframe. Along with our written notes, we would like to audio record this interview with your permission. The recording will only be used as back-up for the notes and will not be shared with anyone outside of our evaluation team.
Will you permit us to record this interview? [If consent is given, turn recorder on.]
Do you have any questions before we begin?
BACKGROUND AND ORGANIZATIONAL INFORMATION
We would like to start with a few background questions about your organization/company, your roles, and general experience with providing technical assistance.
Can you confirm your title?
Please describe your roles and responsibilities.
How does this relate to the TA your organization/company provides?
Where does TA fit within your organization’s business model?
For clarification, is providing TA a central aspect of your organization’s services?
Overall, how many people in your organization/company are involved in providing TA?
How are staff allocated among TA roles? (e.g., Support/Logistics, Instructors, Designers, Evaluators, etc.).
Do you use subcontractors to deliver TA? If so, what roles do they generally play?
We are aware that you have been awarded a contract/cooperative agreement to provide TA for certain HUD programs:
What HUD TA contracts and/or cooperative agreements does your organization currently manage? Can you confirm the year awarded, the total amount, and the program office?
Which program offices at HUD have funded your TA activities (since 2014)?
For which program was the most recent application?
Do you typically apply to provide TA through the same HUD programs? Why, or why not?
Who do you collaborate with as part of your TA engagements since 2014? Staff at HUD Headquarters? HUD field offices? Other TA providers?
Can you describe the nature of these collaborations?
Have they helped or hindered the TA process and/or delivery of the TA?
Does HUD’s TA approach encourage or hinder these collaborations?
Looking at the TA Matrix we sent you, can you list the types of TA you provide since 2014? (Refer to our TA Matrix emailed ahead of the interview; for example, direct TA, on-call TA, needs assessments, trainings, or product creation).
Of the TA, you provided since 2014, have you worked on TA projects that HUD required the customer to take rather than TA requested by the customer?
Of the TA, you provided since 2014, what percentage was for TA required by HUD vs. what percentage was voluntary?
Did you provide TA before 2014?
For which programs, did you provide this earlier TA?
What types of TA did you provide? (Probe: on-call TA, direct TA, needs assessment, trainings and self-guided learning online, or tools and products such as manuals or handouts accessible through HUD Exchange)
Response Tracking Table
Types of TA Provided |
Since 2014 (Community Compass) |
Before 2014 (OneCPD) |
Targeted TA |
|
|
Direct TA |
|
|
On-Call TA |
|
|
Needs Assessment |
|
|
Universal TA |
|
|
Trainings and Self-Guided Learning |
|
|
Tools and Products |
|
|
Ask-A-Question Feature |
|
|
[Based on their responses] Let’s first focus on your TA experience with [select one of the types of Universal TA or Targeted TA] and go through these questions.
[Internal Guidance: focus first on the TA types that we have not heard as much about from previous interviews and then go to other types depending on time, etc.]
THE TA PROCESS
TA REQUESTS
How many staff within your organization support this type of TA (insert TA type here)?
Can you briefly describe their roles and responsibilities?
What is your role on this HUD TA program?
Describe your organization’s process for receiving TA requests from HUD.
Is there a primary point of contact at HUD who sends the TA requests or do the requests come from multiple avenues depending on the type of TA?
How do you get the requests? Through the TA Portal or other mechanism?
Which TA requests come through the TA Portal?
Do you see any trends between the types of requests, either by topic or by TA type that come through the TA Portal and those requests that come through other mechanisms?
What do you do once receive a request? Please outline the various steps.
How many TA requests do you get each quarter from HUD?
How often did your HUD program officer in review your activities during the processing of the TA requests?
Is your relationship primarily with HUD Headquarters staff? Or field office staff?
Any challenges you and/or your team experienced during the TA request process?
Any changes you would make if you could to the TA request process?
DELIVERY OF TA
For the (insert TA type), you deliver, how long does it generally take (on average) from receipt of the HUD TA request to sending a notice to the TA customer that they have been selected?
Who is involved in this process? (Prompt: HUD Headquarters staff, field staff, others)
[Ask if they are not in the interview] Does your TA approach involve subcontractors? If so:
Why do you work with subcontractors?
What types of services or expertise do they provide—e.g., support/logistics instruction?
How many subcontractors are involved with this type of TA?
Is your organization responsible for notifying and scheduling the TA activity with the TA recipient(s)?
If so, please describe the roles and the process.
Is your organization responsible for developing the work plan (e.g., curriculum and instructional goals for the TA)?
If so, please describe the roles and the process.
What role, if any, does your HUD program officer have in reviewing your delivery of the TA, such as input regarding the format and/or curriculum and goals?
How do the reviews take place? (phone calls, emails, visits, conferences, meetings, etc.)
How often do the reviews happen?
Are the reviews helpful?
Do you get input or feedback from others at HUD (e.g., the field office)?
Any challenges you and/or your team experienced during the TA delivery process?
Any changes you would make if you could to the TA delivery process?
REPORTING AND OVERSIGHT OF TA
Can you describe the process for reporting to HUD?
What types of information do you track?
Do you use the DRGR system? If yes, what data is entered in the DRGR?
What other data systems do you use, if any?
What, if any, outcomes or results from the TA have you documented?
Do you track or document these outcomes? Why or why not?
Any examples that you can share?
How often are you required to enter data?
Are there any repercussions if you do not complete data entry?
Are you required to provide written reports to HUD?
If so, what are you required to report on and how? (e.g., data submission, standardized performance tracking measures, a qualitative template, an informal check in).
Who do you report to at HUD?
How often do you speak with your HUD program officer?
How often do you meet with them?
Has your HUD program officer or other HUD staff observed your TA activities in the field?
Any challenges you and/or your team experienced during the reporting process? [e.g. TA portal, DRGR, with data to be reported, etc.]
Any changes you would make to the reporting and TA oversight process?
FEEDBACK ON TA ACTIVITIES
These sections relate to receiving feedback from TA customers and providing feedback to HUD staff about TA delivery.
Soliciting Feedback
Do you solicit feedback from the TA subcontractors and/or instructors?
If so, what type of feedback did you request [the overall TA process, the instructors’ approach, the course itself, the product, etc.]
How do you get the feedback? Can you provide an example?
Do you soliciting feedback from TA customers involved with the TA activity/project?
If so, what type of feedback did you request [the overall TA process, the instructors’ approach, the course itself, the product, etc.]
How do you get the feedback? Can you provide an example?
Do you revise and improve the TA [materials, approach, workshops, etc.] based on feedback?
Can you provide an example?
Do you provide regular feedback to HUD about your TA activities/projects?
If so, how is the feedback conveyed to HUD? [emails, HUD trainings, quarterly meetings, conference calls, etc.]
Can you provide an example?
If so, how often/how regular is the feedback?
If not, would you like the opportunity to give (more) feedback to HUD about the TA programs?
In what formats, would you like to give feedback? (Probe: through direct check-ins, focus groups, email, an anonymous survey/web Portal)
ASSESSMENT AND REFLECTIONS ON THE TA
Did the [type of TA] you provide meet the customers’ need or lead to better program or organizational coordination?
Why or why not?
How do you determine this?
Has your agency/organization tried to assess the outcomes or impact of the recent TA you’ve provided?
If so, how did you assess it?
If so, what was found to be the short-term outcomes or impacts from the TA?
Did/will the TA facilitate any longer-term outcomes or impact?
If so, please describe.
If you have other noteworthy experiences with HUD TA, please briefly describe the supported project, type of TA, and highlight any main successes or challenges with this experience.
WRAP UP
Is there anything else you would like to add about HUD’s TA program and your experience as a HUD TA Provider, or anything we’ve discussed today?
Do you have any questions for us?
Thank you for taking this time to meet with us. If there’s anything you think of later that you’d like to share, please don’t hesitate to e-mail or call us.
Public reporting burden for this collection of information is estimated to average 2 hours for the HUD TA Assessment interviews. HUD may not conduct or sponsor, and a person is not required to respond to, a collection information unless that collection displays a valid OMB control number 2528-xxxx, expiring xx-xx-xxxx.
All the information we collect will remain confidential. The information requested under this collection is protected and held private in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130.
Estimated Length: 90 minutes
Introduction
Good morning/afternoon, I am [your name] and this is my colleague, [name]. We want to thank you for your willingness to talk with us today. As you may know, Congress has mandated an assessment of HUD TA programs funded through the Community Compass NOFA. HUD’s Office of Policy Development and Research (PD&R) has awarded a Cooperative Agreement to the Urban Institute to conduct an evaluation of HUD’s technical assistance (TA) programs. The evaluation will focus on the reorganization of the TA programs under the Community Compass umbrella and the tracking of awardees and activities in administrative data systems specific to HUD TA. The evaluation will also assess what TA is provided, how it’s provided, and how effective it is. This is not an evaluation of your work or of your office.
For this evaluation project, we are conducting interviews conducting interviews with HUD Headquarters staff and field staff, along with a sample of TA providers and TA customers. We are required to inform you that your participation is completely voluntary. If you agree to speak with us about your TA experience and program work, you may choose not to answer any particular questions. We also want to let you know that we will be taking notes of your responses so we can be sure to capture them accurately.We will not attribute your responses to you or name you in any reports. However, you should be aware that due to your position and program area, it may be possible for others to identify your comments despite our efforts.
We know that you are busy so we will focus our discussion as much as possible and promise to keep to the 90 minute timeframe. Along with our written notes, we would like to audio record this interview with your permission. The recording will only be used as back-up for the notes and will not be shared with anyone outside of our evaluation team.
Will you permit us to record this interview? [If consent is given, turn recorder on.]
Do you have any questions before we begin?
BACKGROUND AND ORGANIZATIONAL INFORMATION
We would like to start with a few background questions about your organization, your roles, and general experience with receiving technical assistance.
Can you confirm your title?
Please describe your roles and responsibilities.
How long have you been with this agency / organization?
Which HUD programs does your agency/organization currently administer?
For which HUD programs have you applied for TA since 2014?
For which program was the most recent application?
For which HUD programs, have you received TA since 2014?
What types of TA have your received? For example, on-call TA, direct TA, needs assessment, trainings and self-guided learning online, or tools and products such as manuals or handouts accessible through HUD Exchange. Did you use the Ask-A-Question feature on HUD Exchange?
When were each of these TA projects delivered?
Did you receive TA before 2014?
For which programs did you receive this earlier TA?
What types of TA did you receive? (Probe: on-call TA, direct TA, needs assessment, trainings and self-guided learning online, or tools and products such as manuals or handouts accessible through HUD Exchange)
Response Tracking Table
Types of TA Received |
Since 2014 (in Community Compass?) |
Before 2014 |
Targeted TA |
|
|
Direct TA |
|
|
On-Call TA |
|
|
Needs Assessment |
|
|
Universal TA |
|
|
Trainings and Self-Guided Learning |
|
|
Tools and Products |
|
|
Ask-A-Question Feature |
|
|
Now, let’s focus on your TA experience with ___________________.
TARGETED TA (ON-CALL, DIRECT OR NEEDS ASSESSMENT TA)
TA REQUESTS and APPROVAL
How did you/your agency / your organization learn about the availability of HUD TA for the most recent TA experience?
Was this TA required by HUD or was it voluntary?
If voluntary, what motivated you to request TA?
If required, what were the reasons that it was required by HUD?
Describe the process for requesting TA from HUD for (insert name of TA project).
Did HUD require you/your organization to apply/complete an application?
How was your experience with the application process? (Probe: time taken, effort burden, clarity of instructions/steps?)
Did you have to supply any additional information after your initial submission?
If so, what information did you supply?
Where did you submit your application/request for TA?
Are they sent to a specific office/member of HUD staff?
How long did it take from the time you submitted the request to when you received notice of approval for the TA activity request?
What office or program at HUD approved your TA request/application?
Have you applied for other TA without success, leading to unmet needs for TA?
If so, please explain.
DELIVERY OF TA
How long did it take from the time you received notice of the TA approval for (name of project) for the TA work to begin?
How long did it take from the time you started the TA work to the completion of the project?
Describe the TA activity that you and/or others in your agency/organization attended or received.
What was/were the topic(s) of the TA?
How many people from your office or organization participated in or received TA from this request?
Who delivered/who was involved with the TA?
What were their credentials/expertise on the topic?
Did they have previous experience delivering TA?
How did they deliver the TA? (e.g., in-person workshop or convening, on call coordination/meetings, onsite coordination/meetings, etc.?)
How often did you have contact with the TA provider team? (Probes: over email, over the phone, in person)
[If they have received TA from multiple program offices] How does the process compare to TA provided through other HUD programs?
UNIVERSAL TA (TRAININGS AND SELF-GUIDED LEARNING, TOOLS AND PRODUCTS, AND ASK-A-QUESTION FEATURE)
TRAININGS AND SELF-GUIDED LEARNING [IF APPLICABLE]
How did you/your agency / your organization learn about the availability of the training or self-guided learning opportunity?
Will you briefly describe the training/self-guided learning opportunity and who participated in it?
What motivated you to participate?
Did you find the activity to be helpful in meeting your goals?
Are there any topics you would like to see covered in future trainings or self-guided learning that you do not have access to now?
TOOLS AND PRODUCTS [IF APPLICABLE]
How did you/your agency / your organization learn about the tool/product?
Will you briefly describe the tool/product you used and how you used it?
Did you find the tool/product to be helpful in meeting your goals?
Are there any tools or products you would like to see that you do not have access to now?
ASK-A-QUESTION FEATURE [IF APPLICABLE]
What did you use the Ask-A-Question feature for?
Were you able to get a resolution to your question/answer?
Do you have any suggestions for how the Ask-A-Question feature should change?
REPORTING AND OVERSIGHT OF TA
Are you required to report on the TA activities?
Can you describe the HUD reporting process?
What type of data do you report regarding the TA that you receive?
What data systems does your team use to report TA progress or progress towards goals that the TA is assisting with?
How often does your team enter information into the databases?
FEEDBACK ON TA ACTIVITIES
Did you have the opportunity before or after the TA to provide feedback about your TA?
If so, how did you provide the feedback? (Prompt: through what mechanism and to whom)
Did you see any response or changes made based on this feedback?
Would you like the opportunity to give (more) feedback to HUD about the TA programs?
In what formats would you like to give feedback? (Probe: through direct check-ins, focus groups, email, an anonymous survey/web Portal)
To whom would you like to give feedback—directly to the provider/instructor or HUD?
ASSESSMENT AND REFLECTIONS ON THE TA
Did the most recent TA meet your agency’s / your organization’s expectations and goals?
Why or why not?
Are there other activities or issues within your organization or community that could benefit from this or other types of HUD TA?
What difference did the TA make in addressing the issue or challenge that prompted you/your organization to seek HUD TA?
Did the TA you received lead to better program or organizational coordination? Why or why not?
Did the TA you received lead to better program or organizational collaboration? Why or why not?
Did you participate in the TA program before 2014 (when Community Compass began)?
If so, has the process changed for you before and after 2014?
If so, what kind of changes did you experience (positive/negative)?
What recommendations would you make to improve HUD TA processes and systems?
Probe: Such as improvements to the TA Portal, applications, feedback system, review process, assessment process, or the accessibility and availability of the TA process.
Has your agency/organization assessed the outcomes or impact of the recent TA you’ve received?
If so, how did you assess it?
If so, what was found to be the short-term outcomes or impacts from the TA? (e.g., for you individually, for colleagues/staff, for program/project, for agency/organization, for the community?)
If not, what do you think was the short-term outcome or impact from the TA? (e.g., for you individually, for colleagues/staff, for program/project, for agency/organization, for the community?)
Did/will the TA facilitate any longer-term outcomes or impact?
If so, please describe.
If you have other noteworthy experiences with HUD TA, please briefly describe the supported project, type of TA, and highlight any main successes or challenges with this experience.
Wrap up
Is there anything else you would like to add about being a HUD TA Customer, or anything we’ve discussed today?
Do you have any questions for us?
May we follow-up with you if any other questions arise?
[If yes, exchange contact information for follow-up opportunities]
Thank you for taking this time to meet with us. If there’s anything you think of later that you’d like to share, please don’t hesitate to e-mail or call us.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Lo, Lydia |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |