PART B: Justification for Formative Study of Trade Adjustment Assistance (TAA) Navigators
OMB NO. 1290 - 0043
MAY 2023
Formative Study of Trade Adjustment Assistance (TAA) Navigators
Under OMB No. 1290 – 0043: Formative Data Collections for DOL Research
Supporting Statement
Part A
May 2023
Submitted By:
Chief Evaluation Office
U.S.
Department of Labor
200 Constitution Ave NW
Room
S-4307
Washington, DC 20210
Project Officers:
CEO seeks approval for data to be collected for a Formative Study of TAA Navigators to build evidence about the roles of Navigators more generally, and also how TAA Navigators support the implementation of the TAA program in some states and local areas. The data collected as part of this Formative Study of TAA Navigators will be used to understand which states employ TAA navigators; selected TAA administrators’ perception about why states have chosen to implement Navigator positions or not; how state with Navigators are using them; how Navigator positions have evolved since their inception; how the role of Navigators in TAA are similar to or different from those in other programs, and to inform potential future research on the effectiveness of Navigators in TAA or other programs.
Inform the development of DOL research.
Maintain a research agenda that is rigorous and relevant.
Ensure that research products are as current as possible.
Inform the provision of technical assistance.
This information request includes planned data collections for the Formative Study of TAA Navigator, which is part of a broader portfolio of projects to understand the role of Navigators in the workforce system. The components of the larger research project will be conducted over three years (2022 to 2025).
There are no legal or administrative requirements that necessitate the collection. CEO is undertaking the collection at the discretion of the agency.
DOL will use the data collected through the instruments summarized in this request to describe the implementation of Navigator models in TAA programs. These data and the study team’s descriptive and implementation analyses will provide DOL with a better understanding of the extent and variation in the implementation of TAA Navigators across the country and will help DOL to determine the feasibility of conducting additional research to measure the impact of TAA Navigators on worker outcomes and equity.
The Formative Study of TAA Navigators will address key research questions outlined in Exhibit A.1.
Exhibit A.1. Research questions
|
|
1 The research questions refer to navigators and programs who participate in the discussions, interviews, and observations as part of the study. While, the questionnaire of state TAA coordinators will sample the full universe, the remainder of the study uses purposive sampling of TAA navigators, administrators, partners, employers and participants, which does not generalize to all TAA navigators.
Data collection for the Formative Study of TAA Navigators will primarily consist of nine site visits with states that currently have TAA Navigators. These visits will occur in late spring or summer 2023. These states were selected based on clarifying calls conducted between November 2022 and January 2023 with a subset of states that were known to have had a TAA Navigator at some point. During the site visits, the study team will use four of the six instruments included in this request to conduct the following data collections:
Semi-structured discussions with staff members from the TAA program and other workforce partners
In-person interviews with TAA participants
Semi-structured discussions with employers served by TAA Navigators
Direct observations of TAA Navigators and TAA program participants or potentially eligible participants
Additionally, the study will include the following data collection activities:
A web-based questionnaire in spring 2023 of all state TAA Coordinators
Semi-structured conference calls in late spring or summer 2023 with TAA administrators in eight states that considered developing TAA Navigator positions but chose not to move forward
Understanding the implementation of TAA Navigator models requires collecting data from multiple sources. This clearance request includes the following data collection instruments. See Table A.1 for details on how the study team will use data from these instruments.
Discussion guides for Navigators, administrators, and partners. The study team will conduct 135 semi-structured discussions with state- and local-level TAA staff (state TAA Coordinator and staff who oversee Navigators, petition filing, and worker outreach), state- and local-level WIOA Dislocated Worker staff, Rapid Response staff, and other local AJC staff members who interact with the Navigator. We will have semi-structured discussions with, on average, 15 Navigators, administrators, or partners per state, although the number of each type of staff can vary across the nine states. These semi-structured discussions will be conducted using a master protocol to ensure that similar topics are discussed with all respondents. We expect semi-structured discussions will take place during site visits to state workforce agency offices and to AJCs that Navigators engage with or may be stationed. The semi-structured discussions will last up to 90 minutes.
TAA participant interview guide. The study team will conduct 27 in-person interviews, averaging three interviews per state, using the interview guide, with TAA program participants about their experience with TAA Navigators—including access to services and barriers faced within the program. The study team will work with Navigators and other TAA staff to recruit participants for the interviews. Additionally, the study team will provide live translation if a participant’s preferred language is not English or Spanish.
Discussion guide for employers. The study team will conduct 27 brief semi-structured discussions , averaging three discussions per state and using the discussion guide, with employers about their experiences with a TAA Navigator—including how the Navigator assisted with petition filing or approval, with provision of lists of affected workers, and in addressing placement with new employers, such as with development of work-based learning opportunities. These discussions will take place during site visits and will last up to 30 minutes.
Navigator observation rubric. The study team will collect direct observational data of nine TAA Navigator interactions with potentially eligible TAA participants, current TAA participants, employers, or other partners, averaging one direct observation per state. These may include direct observations of service delivery such as interactions between Navigators and participants, benefits and rights information meetings with potentially eligible TAA participants, or support to an employer on petition filing. Direct observations will be conducted as part of the site visits and will last up to 60 minutes. The study team will use a structured observation rubric to document the content of the interaction and how services are delivered.
State TAA Coordinator questionnaire. The study team will field a web-based questionnaire of all 52 state TAA Coordinators to obtain information about the prevalence of TAA Navigators, their locations, and their basic duties across all states. The study team will program and administer the questionnaire via web. The questionnaire will take approximately 15 minutes, on average, to complete.
Discussion guide for non-participating states. The study team will conduct semi-structured conference calls, using a discussion guide, with eight states that initially expressed interest to federal TAA staff members in creating TAA Navigator positions but confirmed via email that they do not have such positions. These semi-structured conference calls will provide additional information about why these states chose not to implement a Navigator model and will last up to 90 minutes.
Table A.1. Data collection instruments and type of data collected
Instrument |
Mode and respondents |
Purpose |
|
Mode: Semi-structured discussions Respondents: State- and local-level TAA staff, state- and local-level WIOA Dislocated Worker staff, state- and local-level Rapid Response staff, other local-level AJC staff |
Document the core components of Navigator services and context. Navigator interviews will include a discussion of their roles and activities, including how they have spent their time over the most recent week and month. |
|
Mode: In-person interviews Respondents: TAA participants |
Document the experiences of TAA participants as they relate to barriers to access and their experiences with the Navigators. The information will be used to understand program participant experience with TAA Navigators. |
|
Mode: Semi-structured discussions Respondents: Employers that have engaged with Navigators |
Document common experiences of employers with Navigators. The information from these interviews will be used to understand employer experience with TAA Navigators. |
|
Mode: Direct observation Respondents: TAA Navigators and program participants who receive services from Navigators |
Document content of conversation or interaction with Navigators. Data from the observations will inform how TAA Navigators deliver services. |
|
Mode: Web Respondents: State TAA Coordinators |
Inform the prevalence of TAA Navigators, their locations, and roles across all states. It will also provide limited information across all states about why many have chosen not to implement a Navigator model of service. |
|
Mode: Semi-structured conference calls Respondents: State TAA Coordinators in eight states not currently employing Navigators |
Complement questionnaire results by providing more in-depth information about the states’ decision where a TAA Navigator model was not developed. |
1 There is no burden associated with the Navigator observation rubric.
This project will use multiple applications of information technology to reduce burden.
For the semi-structured discussions, in the event an in-person discussion cannot be conducted during the site visit, the discussion will be conducted virtually, as a conference call, using the same protocol to ensure that similar topics are discussed with all respondents. We will also conduct TAA participant interviews virtually if the interviews are not able to be completed in-person.
The state TAA Coordinator questionnaire will have the capability to be hosted on the internet via a live secure web link. To reduce burden, the questionnaires will employ the following: (1) drop-down response categories so respondents can quickly select from a list and (2) dynamic questions and automated skip patterns so respondents see only questions that apply to them (including those based on answers provided previously in the questionnaire).
There is no burden associated with the observation rubric.
This project will not involve collecting information that is available from other sources, as there have been no information collected on TAA Navigators though other research studies or administrative reporting required by the relevant federal agency (the Employment and Training Administration).
Employer partners will participate in discussions as part of study site visits. Some of these employers may be small businesses. To minimize burden on any small businesses that participate, we will request only information required for the intended use of the data collected by each instrument and will minimize burden by restricting the length of interviews to the minimum required time needed to collect the required information based on respondent characteristics and data available on Navigator activities from states—such as TAA manuals, any outreach materials developed by Navigators, and state TAA data. We will also consider the employers’ schedules when making decisions about the timing and locations of the discussions and will conduct discussions as a conference call as needed to further reduce burden. As with all data collection activities, we will remind participants that their participation is completely voluntary.
The is a one-time data collection. This formative study represents an important opportunity for DOL to understand the current implementation of TAA Navigator models and inform potential future research on the effectiveness of TAA Navigators.
There are no special circumstances for the proposed data collection efforts.
No public comments are requested for this information collection.
The study team is coordinating consultation on the research design and data needs. A list of study team members is in Table A.2.
Table A.2. Formative Study of TAA Navigators study team
Organization |
Individuals |
Mathematica
|
Dr.
Jillian Berk |
|
Ms.
Jeanne Bellotti Ms.
Kristen Joyce Ms.
Alicia Harrington |
Social
Policy Research Associates |
Ms.
Kate Dunham |
Navigators, administrators, partners, employers, and state TAA Coordinators will not receive any payments or gifts because activities will be carried out in the course of their employment, with no additional compensation outside of their normal pay.
TAA program participants who complete an interview will receive a $45 incentive. TAA participants who are willing to complete the interview will provide information on lived experience with the TAA Navigator program and will be completing the interview outside of their working hours. They may also be travelling for the interview during the site visit. These respondents may also be less interested in the research on the TAA program. Increasing response rates for populations not motivated to participate otherwise1, improves equity in research, addressing and mitigating nonresponse bias, and providing fair compensation to the participants for their willingness to accept the burdens related to participating in research.2 including which may include time, travel, and child care costs associated with being on site.
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.
As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all federal and departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. All staff must sign an agreement to (1) maintain the privacy of any information from individuals, businesses, organizations, or families participating in any projects; (2) complete online security awareness training when they are hired; and (3) participate in a refresher training annually.
As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall (1) ensure that this standard is incorporated into the Contractor’s property management/control system and (2) establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable federal and departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.
Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individual’s personal identifier.
There are no sensitive questions in this data collection.
Table A.3. Total burden requested under this information collection
Activity1 |
No. of Respon-ndents |
No. of Respon-ses per Respon-dent |
Total Responses |
Average Burden per Response (in hours) |
Total Burden (in hours) |
Hourly Wage Rate2 |
Monetized Value of Time |
Semi-structured discussions with Navigators, administrators, and partners |
45 |
1 |
45 |
1.5 |
68 |
$30.21 |
$2,039 |
TAA participant interviews |
9 |
1 |
9 |
1 |
9 |
$30.21 |
$272 |
Semi-structured discussions with employers |
9 |
1 |
9 |
0.5 |
5 |
$30.21 |
$136 |
State TAA Coordinator questionnaire |
16 |
1 |
16 |
0.25 |
4 |
$30.21 |
$118 |
Semi-structured conference calls with non-participating states |
5 |
1 |
5 |
1.5 |
8 |
$30.21 |
$242 |
TOTAL |
84 |
|
84 |
|
94 |
|
$2,807 |
Note: Numbers are rounded to the nearest whole number for all columns other than the “Average burden hours” and “Hourly wage rate” columns.
1 There is no burden associated with the Navigator observation rubric. This data collection instrument is excluded from the burden table.
2 The average hourly wage is an average of the mean hourly wages for social scientists ($43.70), state government officials ($24.82), local government officials ($26.36), and social service occupation ($25.94), from Occupational Employment and Wage Statistics, May 2021 Occupation Profiles https://www.bls.gov/oes/2021/may/oes_nat.htm. (120.82/4 = $30.21)
Table A.3. includes assumptions about the annual number of respondents expected; the average number of responses per respondent; the average hours of burden per response; the annual burden hours estimated; the time value assumed for respondents; and the total annualized monetary burden hours for the semi-structured discussions with Navigators, administrators, partners, and employers; and TAA participant interviews, as well as for state TAA Coordinator questionnaire. All the activities this request covers are annualized over three years. Here, we summarize the burden estimates rounded to the nearest whole number for each of the data collection activities:
Semi-structured discussions with Navigators, administrators, and partners. The study team will conduct semi-structured discussions with state-level administrators, program administrators, and Navigators. We estimate the discussions will take about 90 minutes to complete. The annualized burden is approximately 68 hours.
TAA participant interviews. The study team will conduct interviews with participants during the site visits. We estimate the interviews will take about 60 minutes to complete. The annualized burden is approximately nine hours.
Semi-structured discussions with employers. The study team will conduct semi-structured discussions with employers during site visits. We estimate the discussions will take about 30 minutes to complete. The annualized burden is approximately five hours.
State TAA Coordinator questionnaire. The study team will administer an electronic questionnaire to state TAA Coordinators and the TAA Coordinators for the District of Columbia and Puerto Rico. We estimate the questionnaire will take about 15 minutes to complete. The annualized burden is approximately four hours.
Semi-structured conference calls with non-participating states. The study team will conduct semi-structured conference calls with states that expressed interest in the Navigator model but opted not to move forward. We estimate the conference calls will involve two state TAA administrators each and will take about 90 minutes to complete. The annualized burden is approximately eight hours.
There are no additional costs to respondents.
A14. Estimate of Cost to the Federal Government
The estimated annual cost to the federal government for the contractor to carry out the data collection activities included in this request is $124,736
This is for an individual information collection under the umbrella formative generic clearance for DOL research (1290-00043).
Data from the state site visits, the TAA Coordinator questionnaire, and state conference calls will be analyzed to create consistent information on Navigator characteristics, roles and responsibilities, time use, worker and participant target populations, and partners.
Analysis of discussions and site observation data will involve coding and triangulating across data sources. The study team will begin by writing up detailed field notes from in-person discussions and conference calls as well as observation rubrics in a structured format. To code the qualitative data for key themes and topics, a coding scheme will be developed and organized according to key research questions and topics following a directed approach to content analysis3 that attends to themes that are predetermined by program goals and research questions as well as emergent themes. The study team will then code the data using qualitative analysis software. To ensure reliability across team staff, all coders will code an initial set of documents and compare codes to identify and resolve discrepancies. These data will be used to explore key research questions and feasibility of designs for a future impact study.
Findings from the Formative Study of TAA Navigators will inform internal DOL decisions about future research and will be summarized in a report about the implementation of TAA Navigator models. The findings from this data collection will be used in a larger research effort to understand the functions of Navigators in multiple DOL programs, which has as its aim improving the effectiveness of such programs. The findings from this collection will likely be used in developing new research but also in informing providers, program administrators, and other interested parties about what has been learned to date about Navigators. For this reason, the findings will likely be provided in reports made available publicly, though such publication is not the primary purpose of the data collection.
All instruments will display the expiration date for OMB approval.
No exceptions are necessary for this information collection.
1 Singer, E. and C.Ye. “The Use and Effects of Incentives in Surveys.” The ANNALS of the American Academy of Political and Social Science, vol. 645, 2013, pp. 112-141.
2 Gelinas, L., E. Largent, G. Cohen, S. Kornetsky, B. Bierer, and H. Fernandez Lynch. “A Framework for Ethical Payment to Research Participants.” New England Journal of Medicine, vol. 378, February 2018, pp. 766-771.
3 Hsieh, H. -F., and S. Shannon. “Three Approaches to Qualitative Content Analysis.” Qualitative Health Research, vol. 15, no. 9, November 2005, pp. 1277–1288.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2024-07-24 |