1290-0NEW Supporting Statement A (08272020)_updated

part a: justification for eVALUATION OF THE HOMELESS VETERaNS’ REINTEGRATION PROGRAM)

omb nO. 1290-0NEW

May 2020

PART A: Justification

The Chief Evaluation Office of the U.S. Department of Labor (DOL) has commissioned an evaluation of the Homeless Veterans’ Reintegration Program (HVRP), a competitive grant program administered by DOL’s Veterans’ Employment and Training Service (VETS). HVRP assists veterans experiencing homelessness to find and hold meaningful employment. It does so by providing employment services and by developing partnerships with other service providers to help address the complex circumstances of homeless veterans. The HVRP evaluation offers an opportunity to build knowledge about the implementation and effectiveness of these grants.

This package requests clearance for four data collection instruments as part of the implementation evaluation:

1. Grantee Survey

2. Key Informant Interview Guide

3. HVRP Veteran Interview Guide

4. Non-HVRP Veteran Interview Guide

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

In this evaluation, DOL seeks to enhance understanding of HVRP with evidence of program effectiveness and implementation. Findings from the evaluation will help DOL make good decisions about what works best for whom and about effective ways to improve the service systems seeking to support veterans experiencing homelessness. The evaluation will estimate program impacts through a matched comparison group (MCG) design using available administrative data. Through the evaluation’s implementation study, which includes the grantee survey and site visits, the study team will describe grantees’ contextual factors (such as the job market and other available services in the community), and programmatic inputs and outputs (such as how participants flow through the program and how case management is provided), and the services available to non-HVRP homeless veterans to help explain the impact findings.

Initially authorized under Section 738 of the Steward B. McKinney Homeless Assistance Act of 1987, HVRP is currently authorized by Title 38 U.S.C. Section 2021, as added by Section 5 of Public Law 107-95, the Homeless Veterans Comprehensive Assistance Act of 2001. Section 2021 statute states that “the Secretary of Labor shall conduct, directly or through grant or contract, such programs as the Secretary determines appropriate to provide job training, counseling, and placement services (including job readiness and literacy and skills training) to expedite the reintegration of homeless veterans into the labor force.” The Title also provides for the Secretary to “collect such information as that Secretary considers appropriate to monitor and evaluate the distribution and expenditure of funds appropriated to carry out this section.”

This is a new collection request for a grantee survey, key informant interview guide, HVRP veteran interview guide, and non-HVRP veteran interview guide. Site visits and interviews may be done virtually, if in-person visits are not feasible due to the COVID-19 pandemic. This package requests clearance for data collection activities that need to start in August 2020 to provide DOL with information related to how grantees that were operating in program year 2019 and received program year 2020 grants continue to adapt their programs as a result of the COVID-19 pandemic.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The data collected through the activities summarized in this request will be used by DOL to comprehensively describe HVRP, including its partnerships, training and support services provided, target population, and common implementation successes and challenges. The evaluation will also assess the impacts of HVRP on participant outcomes. In addition to describing the current set of grantees, the grantee survey will support the impact study by allowing for deeper understanding of grantees’ different program models. The qualitative data instruments for conducting site visits will provide important data to document grantees’ structures and provision of services and collect information on the homeless veterans who receive services through the American Job Centers (AJCs) and the services they receive.

1. Overview of the evaluation

The evaluation of HVRP includes two components: (1) an implementation evaluation to understand program implementation, partnership development, and available community services, and (2) an impact evaluation to measure the effects of HVRP on participant outcomes. It takes place over five years (2017 to 2022), and addresses the following research questions:

1. Do HVRP participants in program years 2019 and 2020 have better employment outcomes up to two and three years after enrollment than similar homeless veterans who sought workforce services through AJCs outside of HVRP?

2. Do the effects of HVRP participants in program years 2019 and 2020 differ for key subgroups defined by participant and local area characteristics?

3. How do HVRP grantees identify and enroll veterans eligible for their programs, and how is eligibility determined? What are the eligibility requirements and screening methods? To what extent do grantees screen for program readiness or other characteristics? What types of assessment tools do HVRP grantees use, and for what purpose? What are the characteristics of HVRP participants? How and when are HVRP participants enrolled in a program at an AJC and entered into the state data system?

4. What are HVRP programs’ key components and the role of partnerships? What services and supports are provided, both directly by grantees and through referrals? What are the primary functions and activities related to case management? How are systems and partnerships developed and maintained? What is the role of HVRP services in these systems and partnerships? How strong are these partnerships?

1. How strong is the contrast between treatment and comparison conditions? How do the homeless veterans who receive services through the AJC only compare to those receiving HVRP services? What services do non-HVRP participants receive, and how do they compare to HVRP services?

The implementation evaluation component will answer research questions 3 to 5. This component includes a grantee survey involving all grantees and site visits to eight grantees and their communities. Site visits will include key informant interviews with grantee, partner, and other community stakeholder respondents, and in-depth interviews with current and former HVRP program participants and with non-HVRP veterans. Site visits and interviews may be done virtually, if in-person visits are not feasible due to COVID-19. The impact evaluation component will use administrative data to address research questions 1 and 2. This means that no primary data will be collected to estimate impacts.

If site visits cannot be conducted due to COVID-19 stay-at-home orders or concerns for the safety and health of interviewees, all attempts will be made to conduct site visits virtually. Key informant interviews with grantee, partner, and other community stakeholders will be conducted by phone or video conference at the preference of the key informant.

Every effort will be made to conduct in-person interviews with HVRP and non-HVRP veterans. Interviews with current and former HVRP program participants and non-HVRP veterans would be significantly more difficult to convert to a telephone or video conference format for multiple reasons: veterans may be less comfortable sharing their story with a person over telephone or via video conference and veterans may be less likely to have the technology necessary to make these connections. To mitigate the first concern, the research team will use the same method of communication (telephone or video conference and the same platform) that the HVRP provider uses to provide virtual case work and services. This will increase the veterans’ fluency and comfort with the media. Additionally, the interview guides could be modified to limit any potentially sensitive topics focusing more intently on veterans’ experiences with the program. It is likely that veterans would still share information on their past homeless history and other potentially sensitive topics, but they would not be asked directly and the choice to share would be entirely theirs. To mitigate the second concern, the research team will work with the HVRP grantee to ensure that HVRP veterans have access to the needed technology to participate either through providing phone or internet access through the program and/or the program hosting the veteran in an onsite office space with access to a phone or computer for the purposes of the interview. For non-HVRP veterans, we would work with a non-HVRP partner agency, likely the AJC or Disabled Veterans’ Outreach Program (DVOP), to ensure non-HVRP veterans could also participate in the same ways. Paper copies of consent forms with research team contact information and incentives for participation would be delivered to the program in advance of the virtual site visit so both could be delivered to the veteran while they were on site for the interview.

In selecting grantees for the site visits, DOL will prioritize grantees that have participants represented in the impact study. For the impact study, we will identify HVRP participants using the Workforce Integrated Performance System (WIPS) administrative data set. Once we identify participants, we will use the administrative data to identify from which of the HVRP grantees they received services. In addition, DOL will aim to select a diverse set of grantees that vary in such factors as, population served, urbanicity, and region. We will administer the grantee survey to all HVRP grantees.

2. Overview of the data collection

Understanding the implementation and effectiveness of HVRP requires data collection from multiple sources. The implementation evaluation data collection covered by this clearance request includes the following:

1. Grantee survey. The purpose of this survey, to be administered to all HVRP grantee directors in late summer 2020, is to collect information about HVRP grantees’ characteristics, program features, program services, partner participation, and implementation challenges and successes as they adapt to COVID-19. The survey is expected to take approximately one hour to complete with an anticipated 90 percent response rate.

2. Key informant interview guide. The guide will be used for interviews during in-person or virtual visits to eight grantees and their communities. The interviews will provide information on the grantees’ HVRP programs and the services available to homeless veterans, including those not participating in HVRP. Using the guide, the study team will customize protocols to conduct interviews with grantee administrators and staff, such as job developers, employment specialists, and case managers. The team will also interview managers and staff at the grantees’ key partner organizations or programs, as appropriate, such as those of AJCs, Veterans Affairs, Continuum of Care (CoC)/housing providers, and other partners. Importantly, to ensure understanding the service environment for homeless veterans, we will interview the following respondents regardless of their collaboration with the HVRP grantee: the Veteran Affairs homeless coordinator, the local DVOP specialist at the local AJC, and a CoC director. Interviews are expected to take an average of one hour. This guide was written with the expectation that interviews would be conducted in person, but would not require changes to be adapted to a telephone or video conference format.

3. HVRP veteran interview guide. The study team will conduct in-depth interviews with eight current and former HVRP participants at each of the site visit grantees. This data collection method will collect detailed information about participants’ pathways to homelessness, the barriers they face to looking for and staying in work, experiences in HVRP and other employment and supportive services, and their post-program outcomes. Most importantly we will ask veterans which parts of the HVRP program were most helpful. This will help us address the key ingredients of the model that may drive outcomes. The interviews would result in further understanding the decisions, lived experience, and life challenges faced by veterans who experience homelessness and unemployment and would provide valuable insights with which to contextualize the impact findings based on the quantitative data. The interview, which will be voluntary, is expected to take an average of 1.5 hours. This guide was written with the expectation that interviews with veterans would be conducted in person. If interviews occur by phone or video conference, the interview guide may be modified to focus more intently on veterans’ experiences with the program as opposed to their employment or homeless history.

4. Non-HVRP veteran interview guide. The study team will conduct in-depth interviews with eight non-HVRP veterans in each of the eight communities. This data collection method will focus on collecting information about veterans’ pathways to services and their experiences in employment-related programs. We will also ask veterans about their knowledge of the HVRP program and the reasons why they do not participate. The interviews will provide valuable insights with which to contextualize the impact findings based on the quantitative data. The interview, which will be voluntary, is expected to take an average of 45 minutes. This guide was written with the expectation that interviews with veterans would be conducted in person. If interviews occur by phone or video conference, the interview guide may be modified to focus more specifically on veterans’ experiences with community programs and services they were able to access and those they wanted, but could not access, as opposed to their employment or homeless history.

Proposed uses for each data collection activity are described in Table A.1. A full version of each instrument is attached to this Justification Statement.

Table A.1. How data will be used, by data collection activity

Data collection instrument	How the data will be used
1. Grantee survey	We will conduct descriptive analyses to describe the HVRP grantees’ characteristics, program features, program services, partner participation, and implementation challenges and successes.
2. Key informant interview guide	We will analyze the data to develop common themes across the eight grantees to answer the research questions and identify services available for non-HVRP homeless veterans.
3. HVRP veteran interview guide	We will use NVivo, a qualitative data analysis software, to create a uniform coding scheme aligned with the interview guide. We will code this data and identify common themes that emerge from the interviews.
4. Non-HVRP veteran interview guide	We will use NVivo to create a uniform coding scheme aligned with the interview guide. We will code this data and identify common themes that emerge from the interviews.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Data collection will be conducted using advanced technology to reduce burden on program participants and grantee and partner staff whenever possible. We plan to obtain information in an efficient way that minimizes respondent burden. When feasible, we will gather information from existing data sources, using the most efficient methods available.

The grantee survey will be administered via web survey, so that it can be accessed from any computer, allowing for the greatest ease of access. The web-based survey will enable respondents to complete the data collection instrument at a location and time of their choice, and its built-in editing checks and programmed skips will reduce the level of response errors and allow respondents to complete the survey as quickly as possible. If in-person site visits and interviews are not feasible due to COVID-19, the study team will conduct key informant interviews by video conference using available platforms, such as WebEx or Zoom. We also will work with the program staff to explore conducting in-depth interviews with veterans using the same technology.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The HVRP evaluation will not require collection of information that is available through alternate sources. The grantee survey, key informant interviews, and in-depth interviews with HVRP and non-HVRP veterans are collecting new data that are not available elsewhere.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

The data collection effort might include small businesses or small, nonprofit organizations. The study team will minimize burden for respondents by restricting the interview length to the minimum required and conducting interviews at times convenient for the respondents.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If these one-time data are not collected, DOL will not have a thorough understanding of the structure and implementation of the current set of HVRP grantees. The implementation study data also provide critical contextual information to help interpret findings from the impact study. Without this information, DOL will not be able to identify those program components that may be contributing to any achieved positive participant outcomes.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

* Requiring respondents to report information to the agency more often than quarterly;

* Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

* Requiring respondents to submit more than an original and two copies of any document;

* Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

* In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

* Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

* That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

* Requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no special circumstances for the proposed data collection.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

The 60-day notice to solicit public comments was published in the Federal Register, 83 FR 40087-4088 on August 13, 2018. No public comments were received.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years - even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The study team consulted with current and former providers of technical assistance to HVRP grantees through the National Veterans Technical Assistance Center. We discussed with them the nature of HVRP to help inform the content of the survey. These experts are listed in Table A.2.

Table A.2. Individuals providing consultation on HVRP evaluation design

John Rio

Deputy Director, Advocates for Human Potential, Inc.

41 State Street, Suite 500

Albany, NY 12207

Cindy Borden

Director TA and Training, National Coalition for Homeless Veterans (NCHV)

1730 M Street, NW Suite 705

Washington, DC 20036

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Respondents to the grantee survey and grantee and partner staff participating in site visits will not receive payments.

Incentives for this data collection are planned for HVRP and non-HVRP veterans who participate in an in-depth interview. Each HVRP veteran who participates in a 90-minute in-depth interview will receive a $50 gift card in appreciation of his or her contributions toward the research. Each non-HVRP veteran who participates will receive a $25 gift card in appreciation, given the shorter 45-minute interview length.

The veteran interviews were originally planned to be conducted in-person, facilitated by the program sites from which they are receiving services. The in-person format would have mitigated some of the barriers to veteran participation, for example, by providing a safe and private environment where participants could speak to a researcher. Due to COVID-19, these interviews will take place by phone, increasing the barriers to participation. For example, for programs that are limiting in-person interactions with clients, phone is the most likely way to reach them to ask them to participate in an interview for the evaluation. If participants have limited phone plans or prepaid cell phone minutes, they may be less likely to participate in an interview because of the cell phone costs and difficulty in reaching willing participants if their minutes have run out. For programs that have offices that are open to participants, transportation costs to the site can also be a barrier to participation in an interview. The planned incentive amount helps to ease the financial burden that many participants will incur as a result of participation, including cell phone costs or traveling to a program site. Considerable experimental evidence has also demonstrated that offering incentives encourages those less interested in the research to participate.^1,2 Therefore, providing an incentive for the veteran interviews can help reduce systematic nonresponse among those for whom cost is a barrier, or who have less interest in study participation.

A previous study assessing the effect of Supplemental Nutrition Assistance Program on Food Security (OMB Control Number 0584-0563, Discontinued 9/19/11) offered a $30 incentive to a disadvantaged target population for a 90-minute in-depth interview, similar to the population targeted for HVRP. As there were difficulties completing in-depth interviews for this study, a subsequent study with a similar population used $50 incentives effectively to recruit demonstration project participants for in-depth interviews (Evaluation of Demonstration Projects to End Childhood Hunger; OMB Control Number 0584-0603; Discontinued 8/31/18). However, the in-depth interviews for both of these studies were done in-person, to facilitate access to hard-to-reach individuals, like the veterans sought for the current study. Shifting to fully remote data collection as a result of the COVID-19 pandemic increases the challenges of engaging program sites to contact and help recruit veterans to participate in a remote interview. In promoting HVRP and non-HVRP veteran participation, the planned incentives for the current study may help limit the amount of time program sites and contractor staff will need to expend to recruit a sufficient number of willing study participants. The Strengthening Relationship Education and Marriage Services (OMB Control Number 0970-0481, Expires 4/30/22) faced lower than expected response rates for the follow-up survey in the spring of 2020 as the pandemic began, and therefore could not utilize in-person field locating to promote participation. In early 2020, the Evaluation of Employment Coaching for TANF and Related Populations (OMB Control Number 0970-0506, Expires 4/30/22) also faced lower responses rates in some program sites, which site staff attributed to increased housing costs and homelessness among study participants. Consequently, OMB approved an increase in the offered incentive for the low-income, hard to reach population targeted by both studies, raising the amount to $40 or $50 for a 45-minute interview. The current study can be expected to face similar challenges in recruiting hard to reach veterans for virtual interviews.

10. Describe any assurance of privacy provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Information collected will be kept private to the extent permitted by law. The study team will protect the full privacy of all individuals who provide data. The study will not have data associated with personally identifiable information (PII), as study staff will assign random ID numbers to all data records and then strip any PII, such as name, from the data records. In addition to the data safeguards described here, the study team will ensure that no respondent or organization names are identified in publicly available reports or findings, and if necessary, the study team will mask distinguishing characteristics.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

No questions of a sensitive nature will be included in the grantee survey or key informant interviews. The HVRP and non-HVRP veteran interview guides do ask questions that could be considered sensitive. These questions seek to understand the challenges that participants have faced resulting in their homelessness. These questions are justified so that the study team can document the circumstances that bring veterans into HVRP and other services, and their understanding for how the programs help them. These interviews will be valuable in that they will add the participants’ perspectives to the research.

12. Provide estimates of the hour burden of the collection of information.

Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

* If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens. *

* Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included under “Annual Cost to Federal Government.”

Table A.3 provides annual burden estimates for each of the data collection activities for which this package requests clearance. All of the activities covered by this request will take place within about a three-year period. To calculate the estimated cost burden for respondents, average hourly wages from the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates for May 2019 were multiplied by the number of hours per respondent type. The following summarizes the annual burden estimates for each of the four data collection activities:

1. Grantee survey. We will administer the grantee survey to 149 awardees, with 134 expected completes (90 percent of 149 awardees) using a web-based format. Each survey will take one hour to complete. The total burden for the grantee survey is 134 hours (134 × 1 hour); the annualized burden is 45 hours.

2. Key informant interview guide. We will conduct interviews with key informants on visits to eight grantees. We assume that we will conduct interviews lasting an average of one hour with approximately 16 respondents per visit. The total burden estimate for the protocol is 128 hours (8 grantees × 16 respondents × 1 hour); the annualized burden is 43 hours.

3. HVRP veteran interview guide. We will use the guide to conduct in-depth interviews with 64 current and former HVRP participants. Each interview will last approximately 1.5 hours. The total burden estimate for the guide is 96 hours (8 grantees × 8 participants × 1.5 hours); the annualized burden is 32 hours.

4. Non-HVRP veteran interview guide. We will use the guide to conduct interviews with 64 non-HVRP participants. Each interview will last approximately .75 hours. The total burden estimate for the guide is 48 hours (8 grantees × 8 participants × .75 hours); the annualized burden is 16 hours.

Table A.3. Estimated annualized respondent hour and cost burden

Data collection activity	Annual number of respondents	Number of responses per respondent	Total number of responses	Average burden per response (in hours)	Annual estimated burden hours	Average hourly wagea	Annual monetized burden costs
Grantee survey	45	1	45	1	45	$59.15	$2,661.75
Key informant interview guide	43	1	43	1	43	$59.15	$2,543.45
HVRP veteran interview guide	21	1	21	1.5	32	$7.25	$232
Non-HVRP veteran interview guide	21	1	21	.75	16	$7.25	$116
Unduplicated total	130	--	130	--	136	--	$5,553.2

^a The average hourly wage was obtained from the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates, May 2019. Estimates are based on the national mean hourly wage for “General and Operations Managers” of $59.15. The hourly wage for the in-depth interviews with veterans is the federal minimum wage.

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

* The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

* If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collections services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

* Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

There are no additional costs to respondents other than their time.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

The total cost to the Federal government over the three years is $3,058,062, and the annualized cost is $1,019,354. Costs result from the following categories:

• The annualized cost to the federal government for the contractor to carry out this study is $999,588³. The total cost of the evaluation for three years is $2,998,764. Therefore, the annualized cost is $2,998,764 / 3 years = $999,588.

• The annual cost borne by DOL for federal technical staff to oversee the contract is estimated to be $19,766. We expect the annual level of effort to perform these duties will require 200 hours for one federal GS 14 step 4 employee based in Washington, D.C., earning $61.77 per hour. (See Office of Personnel Management 2019 Hourly Salary Table at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2019/DCB_h.pdf. To account for fringe benefits and other overhead costs, the agency has applied multiplication factor of 1.6:

200 hours × $61.77 × 1.6 = $19,766.

Thus, the total annualized cost to the federal government for this evaluation is $1,019,354 ($999,588 + $19,766 = $ 1,019,354).

15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.

This is a new information collection.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

1. Analysis plan

Our analysis plans for each component are described below:

• For the grantee survey: We will summarize the quantitative data using basic descriptive statistics. Our analysis will follow similar steps: data cleaning, variable construction, and computation of descriptive statistics. Despite best efforts to encourage full response to the survey instrument, respondents will likely leave some missing or incomplete items. During data cleaning, the study team will look for unusual patterns of item nonresponse. If item nonresponse is less than 10 percent, the study report will simply indicate the proportion missing. If it is greater than 10 percent, the study team will examine the types of respondents that did not respond and determine whether the data item suffers from nonresponse bias. Some items of less significance could be dropped from the analysis. Others could be presented in reports, but the study report will provide clear information on the nonresponse issue and describe any cautions that readers should take in interpreting the results.

To facilitate analysis, we will create variables to address the implementation constructs of interest and then, to prepare the data for analysis, we will run a series of data checks, examine frequencies and means, and assess the extent of missing data. We will use these data to identify key ingredients of the HVRP model and to create a typology of service approaches and to classify HVRP grantees using this framework.

• For the key informant interviews: We will analyze the data to develop common themes from the research questions as well as site profiles. These common themes for site visit interviews will be organized by the interview guide framework: (1) target population and enrollment process; (2) key components of the HVRP program model; (3) HVRP partners; and (4) implementation challenges and facilitators. The common themes for the comparison area interviews will focus on: (1) available services for homeless veterans, (2) community partnerships, and (3) the system for providing services.

• For the qualitative data from the HVRP and non-HVRP veteran interviews: We will use NVivo, a qualitative data analysis software, to create a uniform coding scheme aligned with the interview guide. We will code this data and identify common themes that emerge from the interviews.

2. Publications

In early 2022, we will produce a report on the implementation and impact evaluations as well as other dissemination products such as fact sheets and issue briefs on topics of interest to DOL.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The OMB approval number and expiration date will be displayed or cited on all forms completed as part of the data collection.

18. Explain each exception to the topics of the certification statement identified in “Certification for Paperwork Reduction Act Submissions.”

No exceptions are necessary for this information collection.

1 Groves R. M., M. P. Couper, S. Presser, E. Singer, R. Tourangeau, G. Acosta, and L. Nelson. “Experiments in Producing Nonresponse Bias.”  Public Opinion Quarterly, vol. 70, no. 5, 2006, pp. 720–736.

2 Singer, E., and R. A. Kulka. “Paying Respondents for Survey Participation.” In Studies of Welfare Populations: Data Collection and Research Issues. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs, edited by Michele Ver Ploeg, Robert A. Moffitt, and Constance F. Citro. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press, 2002, pp. 105–128.

3 The annualized cost to the federal government for the contractor includes cost for $50 gift cards paid to HVRP veterans who participate in a 90-minute in-depth interview and $25 gift cards paid to non-HVRP veterans who participate in a 45-minute interview.