12-20-18_HUD TA Supporting Statement A

Supporting Statement for Paperwork Reduction Act Submissions

HUD Technical Assistance Assessment

(OMB# 2528-XXXX)

A. Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The U.S. Department of Housing and Urban Development (HUD) is conducting this study under cooperative agreement with the Urban Institute to assess its Technical Assistance (TA) programs under the Community Compass structure. HUD TA enables housing and community development providers to be more effective stewards of HUD funding by equipping them with the knowledge, skills and tools to better manage HUD programs. In 2002, the Government Accountability Office (GAO) evaluated HUD’s existing TA programs, and recommended that HUD streamline and coordinate its TA programs, and request its TA providers to establish performance measures and report on their intended outputs and outcomes. Since then, HUD has made significant changes to the structure, implementation, and data systems used in administering its TA programs, culminating in a unified Fiscal Year 2014 Community Compass Notice of Funding Availability (NOFA) and selection process for awarding TA cooperative agreements to providers across five major appropriation accounts: Departmental TA, McKinney-Vento Homeless Emergency Assistance and Rapid Transition to Housing (HEARTH) TA, McKinney-Vento National Data Analysis Project TA, Public Housing Receivership and Recovery, and Native American Housing Assistance and Self Determination Act TA.

This will be the first assessment of these and other changes made to HUD TA in response to GAO recommendations. This study is authorized by title V of the Housing and Urban Development Act of 1970 (12 U.S.C. 1701z–1 et seq.) on “Research and demonstrations; authorization of appropriations; continuing availability of funds” which states “The Secretary of Housing and Urban Development is authorized and directed to undertake such programs of research, studies, testing, and demonstration relating to the mission and programs of the Department as he determines to be necessary and appropriate.” Full language can be found in the appendix.

This project helps fill the gap in understanding HUD’s TA delivery system since Community Compass began in FY 2014. The Urban Institute is conducting an assessment focused on answering three research questions: 1) what TA does HUD provide, 2) how does HUD provide TA, and 3) how effective is HUD TA? To answer those three questions, this study will include in-depth interviews with TA providers and TA customers to supplement interviews completed during the design of the research.

TA providers play the central role in navigating the Community Compass process and implementing TA activities. To gain insights into how providers engage in TA through Community Compass, we plan to conduct 5 group interviews that will target front-line TA staff and subcontractors that deliver the TA. Also, this project will also interview customers receiving TA under Community Compass. To better understand the customer experience, we will conduct interviews with key staff at organizations that received TA.

Out of these interviews, combined with other data collections methods (administrative data and case studies) the Urban Institute will create a final report to HUD on the answers to the above research questions. The findings in the final report will inform HUD’s future delivery of TA under the Community Compass structure.

2. Indicate how, by whom and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

This is a new collection. HUD’s Office of Policy Development & Research plans to use the data collected by the Urban Institute: 1) to improve the structure of HUD technical assistance (TA) under Community Compass, 2) to provide guidance to TA providers and customers on best and promising practices in working with each other, and 3) to better collect and use data to track performance and make adjustments to TA programs. More targeted uses may include modifying the NOFA requirements or process for TA providers, communicating available TA resources and processes to HUD grantees, and enhancing coordination of TA across HUD, TA providers, and TA customers.

This collection will gather information from TA providers and customers, asking them to report details of their current work across a number of areas through group phone interviews. Questions will focus on:

• For which HUD grant programs and since when the organization has administered or received TA

• Types of TA provided or received

• Procedures, regulations, and processes for requesting or providing TA

• Content and format of what was received or delivered

• Reporting requirements on TA activities

• Feedback solicitation, timing, mechanisms, and follow up for TA activities

• Assessment and reflections on TA activities’ value, impact, changes since 2014, and general quality

The interview protocols are attached in the appendix. All interviewees will be asked similar questions by group type (customer or provider) and the interview will take no more than 1.5 hours. Any interview participant will be asked to participate in only one interview. Group interviews will be conducted by phone by two Urban Institute researchers. The interviews will be scheduled ahead of time by email using an online scheduling tool.

The Urban Institute will use the data from the interviews to draw out insights around the type, level of intensity, and duration of the TA engagements. In-depth interviews with TA providers and customers should help us identify and compare the differences and similarities in TA administration, delivery, and assessment by different HUD programs.

Analysis of qualitative data from TA customer interviews will enable us to better understand their motivations for requesting or being required to receive TA and how that might affect the TA process and results. The interviews also can be used to draw out customers’ perspectives on TA effectiveness—did it meet their expectations and address their needs? As recipients of TA, they will have insights on the roles of providers and field staff in the selection and delivery of TA to respond to the needs of TA customers.

We will use NVivo qualitative data software to organize data for analysis. After cleaning interview notes for clarity, the notes will be entered into NVivo. The research team will develop a codebook of key themes related to the research questions that also will include topics or issues uncovered during data collection. The codebook will be used to identify and mark segments of notes relevant to each topic and theme. Notes also will be coded by type of respondent to support the identification of commonalities and differences within and across respondent types.

Analysts will prepare brief summaries of findings by research questions. In addition to using interview data analytically, we will mine them for quotes to use with permission for illustrating findings throughout the final research report.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

For the interviews, staff will use electronic communication (emails) and an online scheduling platform to book times with the interviewees. Data will be collected through phone surveys administered by the Urban Institute. The interviews will only be available for completion by phone to reduce the time and cost burden of participation in interviews. Research staff will take notes from the interviews using laptops and will audio record the interviews to serve as a back up to the computer notes and capture exact language used by respondents.

Urban will develop a final report that will use the information collected in the interviews, which is intended to be made available to the public over the internet by HUD.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

As this is the first comprehensive evaluation of HUD’s TA programs since their reorganization under the Community Compass NOFA, the information collected in this effort has not previously been collected, nor is it available in any written or electronic source. While HUD requires TA providers and recipients to track and report their TA needs/outputs annually, there has been no systematic collection and analysis of the process and outcomes of TA service provision to understand how to improve the effectiveness of the program in meeting its goals. Both interview protocols are designed to focus exclusively on collecting information unavailable through existing reports or administrative data.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I) describe any methods used to minimize burden.

This information collection will not have a significant economic impact on a substantial number of small businesses or other small entities.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

As outlined in the response to Question 1, HUD needs this assessment of their Technical Assistance programs under the new Community Compass structure in order to understand what TA is provided, how it is provided, and what effect it has. As HUD’s TA programs have not been evaluated since 2002, and since their structure and content has changed under the new Community Compass, HUD requires an updated assessment to know whether its investments are fulfilling their intended purpose. This will be the first assessment of the changes made to HUD TA since FY 2014. Failure to conduct this investigation could result in continued spending without evidence that program goals are being met under the new Community Compass structure.

We expect that the results of this evaluation will be used to inform the future of HUD’s TA program design.

The group interviews are one-time events, designed to provide the information on TA customers and providers on their activities and perspectives. There is no other comprehensive source for this information.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• Requiring respondents to report information to the agency more often than quarterly;

• Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Requiring respondents to submit more than an original and two copies of any document;

• Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than 3 years;

• In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or that unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Requiring respondents to submit proprietary trade secrets or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320 (Controlling Paperwork Burdens on the Public). There are no special circumstances that require deviation from these guidelines.

• Under this ICR, HUD will not conduct any data collection requiring respondents to report information to the agency more often than quarterly;

• Under this ICR, HUD will not conduct any data collection requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Under this ICR, HUD will not conduct any data collection requiring respondents to submit more than an original and two copies of any document;

• Under this ICR, HUD will not conduct any data collection requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

• Under this ICR, HUD will not conduct any data collection in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• Under this ICR, HUD will not conduct any data collection requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• Under this ICR, HUD will not conduct any data collection that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Under this ICR, HUD will not conduct any data collection requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

• Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping disclosure, or reporting format (if any) and the data elements to be recorded, disclosed, or reported.

• Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years -- even if the collection of information activity is the same as in prior periods. There may be circumstances that preclude consultation in a specific situation. These circumstances should be explained.

On September 27, 2018, a 60-Day Federal Register Notice was published at Volume 83, No. 188, page 48859. No public comments were received.

The Urban Institute has consulted with a team of four experts – former HUD staff and a TA program evaluator – review the research design and data collection and analysis plan. This included verification of the availability of data, advice on the frequency and design of data collection, the clarity of our interview selection criteria and protocols, security of record keeping, and plan for the final report. Additionally, we have sought input from TA customers and providers to obtain their feedback on the instruments to be used and format of the information collection. This advice was used in revising the research protocol represented in this supplemental material.

Draft protocol were tested with representatives of those from whom information is to be obtained – TA providers and TA customers – to ascertain the reasonableness of the questions used in data collection. In response, revisions were made to protocol to improve clarity and reduce the burden of recall.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payments, remunerations, or gifts will be provided to interviewees.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation or agency policy.

TA providers and customers will be invited to interviews via an email explaining the purpose of the TA assessment study. Before beginning the interview, group interviewee participants will again be provided an explanation of the purpose of the assessment and how their responses will be used. We cannot guarantee complete confidentiality in the group interviews, given the chance that other group interviewees may share information mentioned during the discussion. However, we will ask all interview participants to keep shared information private. We will also inform participants that Urban itself will keep information shared private, and only report information summarized in aggregate and/or with the specific details masked/changed to protect their identity, and that information shared will not affect their access to future TA services and/or funding. Participants will be then asked to affirm their consent per Institutional Review Board (IRB) guidelines for human research. These communications and explanations are attached in the appendix.

The interview instrument will be reviewed and approved by the Urban Institute’s Institutional Review Board prior to initiating any research, which operates according to the Common Rule on the Protection of Human Subjects found in Title 45 of the Code of Federal Regulations, Part 46 (45 CFR 46). The instrument is also attached in the appendix.

Urban’s research plans for the group interviews will be subject to federal human subject research standards to protect the confidentiality of all research subjects, including all persons interviewed. The interview research protocols have been reviewed and approved by the Urban Institute’s Institutional Review Board. All researchers involved in this information collection and data analysis have received training and are certified to conduct human subject research according to federal guidelines. They have signed a Pledge of Confidentiality specific to this project. All raw and summarized data will be securely stored according to HUD protocol, including proper password-protection and encryption as required for files containing any personally identifiable information.

Authority to offer confidentiality, to the extent permitted by law, is made on the basis of:

1. Section 3(b) of the Department of Housing and Urban Development Act, as amended, 42 U.S.C. 3532, authorizes the Secretary to “conduct continuing comprehensive studies, and make available findings, with respect to the problems of housing and urban development.”

2. Section 7(r)(1) of the Department of Housing and Urban Development Act, as amended, 42 U.S.C. 3535, provides that appropriated funds “shall be available to the Secretary for evaluating and monitoring of all such programs . . . and collecting and maintaining data for such purposes.” Subsection (r)(4)(a) of the act further provides that the Secretary “may provide for evaluation and monitoring under this subsection and collecting and maintaining data for such purposes directly or by grants, contracts, or interagency agreements.”

3. Section 502(g) of title V of the Housing and Urban Development Act of 1970, as amended, 12 USC 1701z-2 (g), authorizes the Secretary “to request and receive such information or data as he deems appropriate from private individuals and organizations, and from public agencies.” It further provides that “[a]ny such information or data shall be used only for the purposes for which it is supplied, and no publication shall be made by the Secretary whereby the information or data furnished by any particular person or establishment can be identified, except with the consent of such person or establishment.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

No questions of a sensitive nature will be included in the interviews.

12. Provide estimates of the hour burden of the collection of information. The statement should:

• indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices;

• if this request covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I; and

• provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead this cost should be included in Item 13.

The interviews will involve the following total numbers of respondents by type: 40 TA provider directors and managers and 25 TA customer chief executives. The interviews will require 2 hours of each interviewees time: 90 minutes for the interview and 30 minutes for scheduling. These assumptions reflect consultation with a sample of potential respondents that tested the interview protocol.

The total estimated burden hours for the interviews is 130 hours: 80 for TA providers and 50 for TA customers. Based on the assumptions on time burden, frequency, and wages, we calculate the total cost to be $8,267.10, as broken down in detail by respondent type, burden and wages below.

Whereas the interviewees from the TA providers will be program-level directors and managers responsible for general program operations, including formulating and implementing policies, managing daily operations and planning and executing the use of program resources, we estimated their cost per interview using the most recent (May 2017) Bureau of Labor Statistics, Occupational Employment Statistics median hourly wage for the labor category General and Operations Managers (11-1021): $48.27.

Whereas the interviewees from TA customers are most likely to be directors of their agency or organization which receives and manages HUD funds and seeks TA from HUD to improve their work, we estimated their cost per interview using the most recent (May 2017) Bureau of Labor Statistics, Occupational Employment Statistics median hourly wage for the labor category Chief Executives (11-1011): $88.11.

All assumptions are reflected in the table below.

Information Collection	Number of Respondents	Frequency of Response	Responses Per Annum	Burden Hour Per Response	Annual Burden Hours	Hourly Cost Per Response	Annual Cost
TA Provider	40.00	1.00	40.00	2.00	80.00	$48.27	$3,861.60
TA Customer	25.00	1.00	25.00	2.00	50.00	$88.11	$4,405.50
TOTAL	65.00	1.00	65.00	--	130.00	---	$8,267.10

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information (do not include the cost of any hour burden shown in Items 12 and 14).

• The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life); and (b) a total operation and maintenance purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s) and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities;

• If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10) utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

• generally, estimates should not include purchases of equipment or services, or portions thereof made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

There are no additional total annual cost burden to respondents or record-keepers beyond the labor cost of burden-hours described in item 12 above.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

		Labor
Staff Role			Estimated task Hours	Hourly Rate	General Schedule Payscale Grade	General Schedule Payscale Step	Total Cost
Principal Research Associate/ Principal Investigator			25.8	$193.11	GS-15	5	$4,972
Senior Research Associate/ Project Director			22.5	$173.44	GS-15	1	$3,902
Senior Research Associate/ Task Lead			33.5	$173.90	GS-15	1	$5,826
Senior Research Associate/ Data Lead			18.5	$170.74	GS-15	1	$3,159
Senior Research Associate/ Method Advisor			8.0	$162.45	GS-15	1	$1,300
Policy Analyst			17.5	$85.95	GS-9	5	$1,504
Research Analyst			48.3	$87.00	GS-9	7	$4,198
Research Assistant			63.5	$63.96	GS-9	1	$4,062
Administrative Staff			6.5	$90.95	GS-9	9	$591
		Other Direct Costs
Computer Network Services							$1,506
Books/Periodicals/Library Services							$12
Reproduction @ $.095/page							$12
Telephone Expenses							$25
Postage/Delivery							$0
Supplies and Miscellaneous							$12
SUBTOTAL							$1,567
Total Expenses							$31,081

15. Explain the reasons for any program changes or adjustments reported in Items 13 and 14 of the OMB Form 83-I.

This is a new program.

16. For collection of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Urban began work for this project in September 2017. Its major components include Reconnaissance (Nov 2017-Apr 2018), Research Design (May 2018- Sep 2018), Administrative Data Analysis (Aug 2018-Dec 2018), In Depth Interviews (Oct 2018-Jun 2019), Case Studies (Jan 2019-Mar 2020), and Final Report Creation (Mar 2020-Aug 2020). This information falls into the in-depth interview section, and will run from October 1, 2018 – beginning with a small series of HUD staff interviews for which OMB approval is not required – and running through June 15, 2019, based on OMB approval. The final report where this information will be delivered to HUD in final publication format by the end of August 2020, after which is will be posted online and publicly disseminated.

A standard interview protocol will guide the conduct of each group interview. We will use NVivo qualitative data software to organize the data from the group interviews for analysis. After cleaning interview notes for clarity, the notes will be entered into NVivo. The research team will develop a codebook of key themes related to the research questions that also will include topics or issues uncovered during data collection. The codebook will be used to identify and mark segments of notes relevant to each topic and theme. Notes also will be coded by type of respondent to support the identification of commonalities and differences within and across respondent types.

Analysts will prepare brief summaries of findings by research questions across data collection methods. For those questions to be addressed with both quantitative and qualitative data, relevant summaries will be shared with the administrative data analysts. Cross-team discussions will support the overall analysis process. In addition to using interview data analytically, we will mine them for quotes to use with permission to illustrate findings throughout the final research report.

The final report will be reviewed in draft form by the same team of external experts to ensure that the data collection and analysis meets high standards, answers research questions appropriately, and notes any limitations encountered during the conduct of the study. An internal Technical Reviewer will review the draft report, as well, prior to submission to HUD to ensure that the work described reflects the research methods and processes employed, and accurately presents all findings. While Urban is allowed to reach independent conclusions based on the evidence evaluated for this research, we acknowledge that HUD may disagree with particular conclusions, and may require a statement inserted to indicate their disagreement. We also acknowledge that any publication of the final report by Urban is embargoed for three months after formal submission of the final report, and must include an attribution to HUD PD&R as funder.

Following a final substantive round of HUD review, the final report will be produced electronically and in printed copy. Urban will work with HUD to ensure formatting is in compliance with the agency’s standard publications, and submit an edited final product. Upon HUD’s acceptance of the final report, the Principal Investigator, the Project Director and other key team members will conduct a meeting at HUD headquarters to brief HUD staff and other federal officials about the evaluation implementation and findings from the research, and to receive final feedback from HUD and others in attendance.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The expiration date for OMB approval will be displayed on written or electronic correspondence used to solicit interview participants. These correspondences are included in the appendix.

18. Explain each exception to the certification statement identified in item 19.

No exceptions are necessary for this information collection.