1290-0NEW Supporting Statement A_TAP ENPP final_update

PART A: JUSTIFICATION for Evaluation of Transition Assistance Program Employment Navigator and Partnership Pilot

OMB No. 1290-0NEW

February 2022

Part A: Justification

The Chief Evaluation Office (CEO) of the U.S. Department of Labor (DOL) has commissioned an evaluation of the Transition Assistance Program Employment Navigator and Partnership Pilot (TAP ENPP, or ENPP). By establishing the ENPP, DOL aims to provide individualized career counseling and guidance to transitioning service members (TSMs) and military spouses. Under the ENPP model, Employment Navigators will provide individualized career services including self-assessment, interest and aptitude testing, career exploration, and detailed labor market information (referred to as the Assist-Explore-Plan [AEP] model) as well as warm handovers and connections to governmental and nongovernmental partners for additional services. ICF has been contracted by CEO to conduct a formative evaluation of this 12-month pilot program. This evaluation examines how the ENPP was implemented, describes to what extent it was implemented as planned, and explores variation in implementation across each of the 13 pilot sites. This document provides insight into the proposed data collection process, which will be used to inform the formative evaluation:

1. TAP manager focus groups

2. Program partner focus groups

3. Military spouse participant focus groups

4. TSM (post-Navigator) participant focus groups

5. TSM (post-Partner) participant focus groups

6. Program Employment Navigator focus groups

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

Service members who separate from the military often encounter many challenges when re-entering civilian life, and in particular, civilian employment. Veterans reflect that TAP overgeneralizes the transition experience and provides a standard set of resources regardless of individual need and career goals.¹ The need to provide individualized counseling for TSMs and spouses as they begin to transition into civilian lives has been well-documented by veterans and stakeholders. In response to this need, the Fiscal Year 2019 National Defense Authorization Act (NDAA) mandated that the TAP offer individualized initial counseling as part of the transition process for service members. In response to the NDAA, DOL Veterans’ Employment and Training Service (VETS) developed the ENPP, which is intended to assist TSMs and spouses in selecting “good fit” career pathways and to connect them to “good fit” partners to improve their civilian employment outcomes.²

DOL intends for the ENPP to result in better employment outcomes for TSMs and spouses, including higher wages, job retention and advancement, and in finding careers that are more suitable for their unique skillsets and aspirations. The ENPP pilot will be implemented for one year (April 1, 2021, through March 31, 2022). During this year, the evaluation will examine how the program connects with TSMs and spouses, the experiences that individuals have as they progress through the pilot, and the connections they are provided to governmental and nongovernmental partners to provide additional supportive services. Data collection will begin upon receipt of PRA clearance.

Citation of sections of laws that justify this information collection: Through TAP, DOL is tasked to “establish and maintain a program to furnish counseling, assistance in identifying employment and training opportunities, help in obtaining such employment and training, and other related information and services to members of the armed forces under the jurisdiction of the Secretary concerned who are being separated from active duty and the spouses of such members” under 10 U.S.C. § 1144.

Further, 29 U.S.C. 3224a (1), authorizes the Secretary of Labor to conduct ongoing evaluation of programs and activities to improve the management and effectiveness of these programs.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The data collected through the activities summarized in this request will be used by DOL to comprehensively describe implementation of the ENPP, identify perceptions of implementation promising practices and challenges that arise, understand how cooperation and coordination among relevant stakeholder groups can be reinforced to improve program outcomes, and document how trainings, direct services, and warm handovers/connections can be enhanced prior to program expansion to additional military bases.

1. Overview of the evaluation

The evaluation of the ENPP is comprised of formative and early implementation components to assess the fidelity of implementation across approximately 13 sites (military bases inside and outside the continental United States). The study will take place over two years (2021 to 2022) and will address the following research questions:

1. Has the pilot been implemented as designed, and to what extent have the planning and mid-way outcomes been achieved?

2. What adjustments and/or modifications did sites have to make during implementation of the ENPP?

Research question 1 will focus on the degree to which Employment Navigators were prepared for their role, partnerships were developed to meet TSMs’ and spouses’ needs, military base stakeholders were engaged through communications and outreach, warm handovers/connections were made to partners, data systems captured activities and outcomes, and Employment Navigator services helped TSMs and spouses set a course for obtaining employment. Research question 2 will focus on how, if at all, the above components of the ENPP implementation and early outcomes varied across sites.

The evaluation includes document review (such as job descriptions, Employment Navigator training materials, MOUs and agreements, promotional materials), interviews with up to nine central federal and program staff, analysis of extant administrative data, and focus groups with approximately 13 sites implementing the ENPP. This PRA clearance request includes the protocols that will be used for focus groups with TAP managers, program partners, military spouse participants, TSM participants, and program Employment Navigators.

The 13 sites DOL selected for the ENPP meet key criteria of interest to DOL. These criteria include the military service branch (Army, Navy, Air Force, Marine, or Joint), the total number of separating service members annually, and the number and/or proportion of “high risk” TSMs (defined as lacking a post-separation plan, having experienced emotional trauma, being financially unstable, lacking a strong support network, having a dishonorable discharge, or separating on short notice due to an injury or disability).

2. Overview of the data collection

Understanding the implementation and early outcomes of the ENPP requires data collection from multiple sources. The data collection instruments included in this clearance request include the protocols that will be used to conduct virtual focus groups at approximately 13 sites implementing the pilot program. Focus groups will be conducted using semi-structured protocols with open-ended question prompts. This package seeks clearance for focus group protocols for four types of respondents: program staff, program partners providing services to participants, TSM program participants, and TSM spouses who received services.

Consent to participate in the research study will be obtained from all focus group participants before the discussion begins. Consent forms will describe the purpose of the study; outline the information that will be collected; explain the risks, benefits, and voluntary nature of participation; and collect participants’ consent to participate in the focus groups.

1. TAP manager focus group protocol. This protocol will be used to conduct virtual focus groups with TAP managers. This protocol will cover communication and collaboration with DOL and their vendor (Serco), participant recruitment for ENPP at their base, and their assessment of the AEP model. The virtual focus groups are expected to take 90 minutes to complete.

2. Program partner focus group protocol. This protocol will be used in focus groups to collect information about the partner onboarding process, how the process of warm handovers is working, their experiences working with ENPP staff, how closely they feel TSMs and spouses are being matched to their agencies based on the services they provide, and how well the data systems used for tracking have been working. The virtual focus groups are expected to take approximately 90 minutes to complete.

3. Military spouse participant focus group protocol. This protocol will be used to conduct focus groups with TSM spouses who have met with an Employment Navigator. The protocol will gather data on how they became aware of the program, how they would describe the program, their assessment of the AEP model, how the warm handover worked, and their assessment of the match between the services provided by the agency(ies) and their employment needs. The virtual focus groups are expected to take approximately 90 minutes to complete

4. TSM (post-Navigator) participant focus group protocol. This protocol will be used to conduct focus groups with TSMs who have met with an Employment Navigator. The protocol will gather data on how they became aware of the program, how they would describe the program, their assessment of the AEP model, how the warm handover worked, and their assessment of the match between the services provided by the agency(ies) and their employment needs. The virtual focus groups are expected to take approximately 90 minutes to complete.

5. TSM (post-Partner) participant focus group protocol. This protocol will be used to conduct focus groups with TSMs who have met with a partner referred by the Employment Navigator. The protocol will gather data on how they became aware of the program, how they would describe the program, their assessment of the AEP model, how the warm handover worked, their assessment of the match between the services provided by the agency(ies) and their employment needs, and whether they would recommend the program to other TSMs. The virtual focus groups are expected to take approximately 90 minutes to complete.

6. Program Employment Navigator focus group protocol. This protocol will be used to conduct focus groups with Employment Navigators. The protocol will gather data on how Employment Navigators describe the mission and goals of the ENPP, their assessment of the training they received (i.e., their preparation for their Employment Navigator role, how relevant it was to their military branch, the guidance and support they received from Serco, VETS, and base staff), the needs of the TSMs and spouses they’ve worked with, how well the intake process capture those needs, their understanding of how recruitment at the base has been conducted, their assessment of the AEP model, how the warm handover worked, and how well the data systems used for tracking have been working. The focus groups will be conducted virtually and are expected to take approximately 90 minutes to complete.

Proposed uses for each data collection activity are described in Table A.1.

Table A.1. How data will be used, by data collection activity

Data collection activity	How the data will be used
1. TAP manager focus groups	We will conduct virtual focus groups with TAP managers to describe the communication and collaboration between TAP and the ENPP, their base’s recruitment strategies, and their assessment of the AEP model.
2. Program partner focus groups	We will conduct virtual focus groups with ENPP partners to describe the onboarding and warm handover processes, their experiences working with ENPP staff, the appropriateness of the match between participants’ employment needs and their agency’s services, and the data systems they’re using to track activities and early outcomes.
3. Military spouse participant focus groups	We will conduct virtual focus groups with military spouse ENPP participants to understand how they became aware of and would describe the program, how the warm handover process worked, their assessment of the AEP model, and their assessment of the match between the services provided by the agency(ies) and their employment needs.
4. TSM (post-Navigator) participant focus groups	We will conduct virtual focus groups with TSMs after they’ve met with an Employment Navigator to understand how they became aware of and would describe the program, how the warm handover process worked, their assessment of the AEP model, and their assessment of the match between the services provided by the agency(ies) and their employment needs.
5. TSM (post-Partner) participant focus groups	We will conduct virtual focus groups with TSMs after they’ve met with an ENPP partner agency to understand how they became aware of and would describe the program, how the warm handover process worked, their assessment of the AEP model, their assessment of the match between the services provided by the agency(ies) and their employment needs, and whether they would recommend the program to other TSMs.
6. Program Employment Navigator focus groups	We will conduct virtual focus groups with Employment Navigators to learn how Employment Navigators describe the mission/goals of the ENPP, their assessment of the trainings they received, what needs TSMs and spouses have, how well the intake process captures those needs, recruitment practices at their base, their assessment of the AEP model and the warm handover process, and the data systems they’re using to track activities and early outcomes.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Several technological techniques will be used to facilitate data collection and reduce respondent burden. The evaluation team’s use of a video conferencing platform such as Microsoft Teams, eliminates the need for travel time for participants. Focus groups will be video-recorded and then transcribed for analysis. This will allow evaluation team to capture participant responses accurately and efficiently, without needing to slow down the discussion for notetaking purposes.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The evaluation of the ENPP will not require collection of information that is available through alternate sources. The evaluation will use available information from DOL, including pilot progress updates and administrative data sets, to ensure that data collected through focus groups are not available elsewhere. In designing the focus group guides, each question was carefully considered to avoid duplication of data available from other sources.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

The evaluation design does not include any data collection with small business owners or otherwise impact small businesses or entities.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

There is an urgent need from the agency to learn about the pilot implementation of the ENPP. As key decisions are being made about expanding the ENPP beyond the 13 pilot sites, the implementation data are important for future modification of the program. Without the information from this evaluation, DOL’s understanding of pilot implementation will be less thorough and more open to bias (i.e., not the result of systematic data collection). Not knowing if the pilot has been implemented according to plan means that DOL’s assessment of the pilot outcomes will be based on an incomplete understanding of the intervention. Further, not understanding the nuance of program delivery across the 13 pilot sites affects DOL’s ability to implement evidence-based policy for any potential future replication and expansion policy for the Employment Navigator program that takes into account the environments, needs, and opportunities across different types of military installations.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

* Requiring respondents to report information to the agency more often than quarterly;

* Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

* Requiring respondents to submit more than an original and two copies of any document;

* Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

* In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

* Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

* That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

* Requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

No special circumstances apply to this data collection.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A 60-day notice to solicit public comments was published in the Federal Register, 86 FR 27114 on May 19, 2021. No comments have been received.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years - even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The evaluation team has consulted with a technical working group, listed in table A.2, that provides subject matter and methodological input on the evaluation design, execution, analysis, and reporting, to ensure the creation of high-quality research that is applicable, useful, and meaningful to stakeholders. The technical working group provided input on evaluation design topics such as selection criteria for pilot bases, creating balanced focus groups, and efficient coding of data for analysis.

Table A.2. Individuals providing consultation on ENPP evaluation design

Peter Mueser, PhD

Professor, Department of Economics and Truman School of Public Affairs

University of Missouri

Columbia, MO 65211

Meredith Kleykamp, PhD

Associate Professor and Associate Chair, Department of Sociology

Director, Center for Research on Military Organization

University of Maryland

College Park, MD 20742

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

There are no payments or gifts to program or partner staff, as activities are expected to be carried out in the course of their employment, and no additional compensation will be provided outside of their normal pay. TSM and spouse respondents may be offered a $25 gift card for participating in focus groups.

10. Describe any assurance of privacy provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The evaluation team will comply with DOL’s privacy policies and procedures and adhere to the standards for protection of personally identifiable information. All members of the evaluation team have signed DOL Non-disclosure Agreements certifying that they will protect the privacy of data collected and processed under this contract, and all have completed human subjects research training.

The evaluation team will apprise all focus group participants of the degree to which they will shield their personal information in notes and in reports to DOL:

• Not report specific names in reports. May include quotes or summaries of situations, but nothing will be associated with an individual by name.

• Since some respondent groups are derived from limited populations (i.e., a given base may only have three Employment Navigators), some reidentification may be possible by those with familiarity with a respondents’ opinions and/or specific work-related events.

• The evaluation team will video record and take written notes during the focus groups. Notes and video recordings will be kept in private files and only authorized staff at ICF will be allowed to use them.

The evaluation design, including instrumentation and consent forms, have been reviewed and approved by ICF’s Institutional Review Board, which is registered with the U.S. Department of Health and Human Services’ Office for Human Research Protections (IRB No. IRB00001920, expires 8/09/2024; Federalwide Assurance No. FWA00002349, expires 7/12/2023).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

None of the focus group guides contain questions that are sensitive in nature. The questions focus on implementation of the pilot and respondents’ perceptions of the pilot effectiveness.

12. Provide estimates of the hour burden of the collection of information.

* Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

* If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.

* Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included under “Annual Cost to Federal Government.”

Table A.3 provides annual burden estimates for each of the data collection activities for which this package requests clearance. Activities covered by this request will take place over a two-year period. To calculate the estimated cost burden for respondents, average hourly wages were obtained from two sources: the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates for May 2020 and the Office of Personnel Management General Schedule Wage and Salary Information. Hourly rates were multiplied by the number of hours per respondent type. The following summarizes the annual burden estimates for each of the six data collection activities:

1. TAP manager focus groups. TAP manager focus groups will be conducted at all sites virtually. On average, focus groups with TAP managers will take 90 minutes to complete. The total burden for TAP manager focus groups is 45 hours (30 managers x 1.5 hours) over two years; the annualized burden is 22.5 hours.

2. Program partner focus groups. Program partner focus groups will be conducted virtually with approximately 48 program partners. On average, focus groups with program partners will take 90 minutes to complete. The total burden for program partner focus groups is 72 hours (48 program partners x 1.5 hours) over two years; the annualized burden is 36.0 hours.

3. Military spouse participant focus groups. Military spouse participant focus groups will be conducted at all sites virtually. On average, focus groups with military spouse participants will take 90 minutes to complete. The total burden for military spouse participant focus groups is 63 hours (42 spouses x 1.5 hours) over two years; the annualized burden is 31.5 hours.

4. TSM (post-Navigator) participant focus groups. TSM (post-Navigator) participant focus groups will be conducted virtually with TSMs served at all sites. On average, focus groups with TSM participants will take 90 minutes to complete. The total burden for TSM participant focus groups is 87 hours (58 TSMs x 1.5 hours) over two years; the annualized burden is 43.5 hours.

5. TSM (post-Partner) participant focus groups. TSM (post-partner) participant focus groups will be conducted virtually with TSMs served at all sites. On average, focus groups with TSM participants will take 90 minutes to complete. The total burden for TSM participant focus groups is 63 hours (42 TSMs x 1.5 hours) over two years; the annualized burden is 31.5 hours.

6. Program Employment Navigator focus groups. Employment Navigator focus groups will be conducted virtually with ENs at all sites. On average, focus groups with ENs will take 90 minutes to complete. The total burden for EN focus groups is 51 hours (34 TSMs x 1.5 hours) over two years; the annualized burden is 25.5 hours.

Table A.3. Estimated Annualized Respondent Hour and Cost Burden

Data Collection Activity	Number of respondents per year	Number of responses per respondent	Total number of responses	Average burden per response (in hours)	Annual estimated burden hours	Average hourly a	Annual monetized burden hours
TAP manager focus groups	15	1	15	1.5	23	$32.02	$720.45
Program partner focus groups	24	1	24	1.5	36.0	$22.85	$822.60
Military spouse participant focus groups	21	1	21	1.5	32	$20.17	$635.36
TSM (post-Navigator) participant focus groups	29	1	29	1.5	44	$17.00	$739.50
TSM (post-Partner) participant focus groups	21	1	21	1.5	32	$17.00	$535.50
Program Employment Navigator focus groups	17	1	17	1.5	26	$22.85	$582.66
Unduplicated Total	127	--	127		193		$4,036.07

^a The hourly wage of $32.02 is based on the January 2021 Office of Personnel Management Grade 12, Step 1 rate (see https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/21Tables/html/GS_h.aspx); the hourly wage of $22.85 is the May 2020 median wage across all Community and Social Service occupations in the United states (see https://www.bls.gov/oes/current/oes_nat.htm); the hourly wage of $20.17 is the May 2020 median wage across all occupations in the United States (see https://www.bls.gov/oes/current/oes_nat.htm); the hourly wage of $17.00 is based on the January 2021 Office of Personnel Management Grade 4, Step 1 rate for the DC-metro area (this was the highest hourly rate across the continental sites, see https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/21Tables/html/DCB_h.aspx).

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

* The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

* If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collections services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

* Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

There are no direct costs to respondents other than their time.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

The total cost to the Federal government over two years is $216,724, and annualized cost to the Federal Government is $108,362. Costs result from the following categories:

The estimated cost to the Federal Government for the contractor to carry out the focus groups is $175,392.³ Annualized, this comes to $87,696:

= $87,696

The annual cost borne by DOL for federal technical staff to oversee the contract is estimated to be $20,665.65. We expect the annual level of effort to perform these duties will require 200 hours for one federal GS 14 step 4 employee based in Washington, D.C., earning $64.58 per hour. (See Office of Personnel Management 2021 Hourly Salary Table at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2021/DCB.pdf.) To account for fringe benefits and other overhead costs, the agency has applied multiplication factor of 1.6:

200 hours × $64.58 × 1.6 = $20,665.60.

Thus, the total annualized federal cost is $87,696+ $20,665.60= $108,361.60.

15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.

This is a new information collection. The data collected through the activities summarized in this request will be used by DOL to comprehensively describe implementation of the ENPP, identify perceptions of implementation promising practices and challenges that arise, understand how cooperation and coordination among relevant stakeholder groups can be reinforced to improve program outcomes, and document how trainings, direct services, and warm handovers/connections can be enhanced prior to program expansion to additional military bases.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

1. Analysis plan

ICF’s proposed approach involves triangulation of data sources within each of the pilot sites as well as across the 13 pilot sites to ensure accurate and determine the consistency of findings. By comparing findings from the focus groups, interviews with central federal and program staff (fewer than nine), program documents, and existing data, ICF will create a robust and dynamic depiction of ENPP implementation and how it varied across the 13 pilot sites. Analyses will begin with a review of program documents and extant administrative data and the key informant interviews. Focus groups with program and partner staff and TSMs/spouses will provide information about the process. All focus group data will be indexed and coded for descriptive and thematic analyses.⁴ After all transcripts are coded, the evaluation team will identify the themes that emerge and conduct cross-site comparisons of content and textual analysis findings.

2. Publications

In mid-2022, the evaluation team will produce a report with results from the formative evaluation to support DOL’s decision-making about potential future delivery and expansion of the ENPP.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The OMB approval number and expiration date will be displayed or cited on all forms completed as part of the data collection.

18. Explain each exception to the topics of the certification statement identified in “Certification for Paperwork Reduction Act Submissions.”

No exceptions are necessary for this information collection.

1 Appel-Newby, E., Deppa, J., Stern, L. (2019). Preparing to separate: The experiences of transitioning service members: An exploratory review of the research. Fairfax, VA: ICF.

2 Serco Group Plc. (2020, November 23). Employment navigator & partnership pilot (ENPP) update to VETS field staff [PowerPoint slides].

3 The total contractor cost includes the cost for $25 gift cards paid to TSM and spouse focus group participants.

4 Schensul, J. J., & LeCompte, M. D. (2016). Ethnography in action: A mixed methods approach (Vol. 7). Rowman & Littlefield.

1