PIA Supporting Statement Part B

OMB Supporting statement

PART B: COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

In this document, the Department of Labor (DOL) requests clearance from the Office of Management and Budget (OMB) under the Paperwork Reduction Act (PRA) for a new collection associated with the Research to Support the Partnership on Inclusive Apprenticeship (PIA) contract. The Office of Disability Employment Policy (ODEP) of the U.S. Department of Labor (DOL) commissioned the Research to Support the Partnership on Inclusive Apprenticeship (PIA) contract to build the evidence on apprenticeship that is inclusive of people with disabilities and support the PIA initiative.

We discuss here three different instruments that are part of the Research to Support the Partnership on Inclusive Apprenticeship study. The study includes the development of a baseline of knowledge on inclusive apprenticeship, technical assistance on data and outcomes to measure progress and success in inclusive apprenticeship, and a formative process study of PIA, all of which are integrated with and inform each other. The three instruments are:

1. Survey of PIA apprenticeship intermediary organizations (AIO), partners, and community of practice members.

2. Topic guide for site visit interviews with AIO program staff and partners.

3. Focus group guide for apprentices enrolled in inclusive apprenticeship programs.

B.1. Respondent Universe and Sampling

In this section, we describe the respondent universe and sampling for each instrument.

Survey of PIA apprenticeship intermediary organizations (AIO), partners, and community of practice members. The universe for the survey includes all PIA AIOs, partners, and community of practice members identified to the study team by PIA. The survey is designed to provide the breadth of knowledge needed to systematically understand how PIA has structured and implemented its efforts to expand inclusive apprenticeship. The survey will be administered to all identified partners and therefore not require statistical methods for sampling purposes. PIA has estimated that 90 AIOs, partners, and community of practice members will be identified (Table B.1). The study team expects a 100 percent response rate for the AIOs, based on the response rate for the American Apprenticeship Initiative (AAI) Grantee Survey (2021)¹, and a 68 percent response rate for apprenticeship partners and community of practice members, based on the AAI Employer Survey². AIOs are directly involved in the work of PIA and our experience is response rates are high in these cases. Apprenticeship partners, many of whom are employers, are more likely to be non-responsive and our recent experience with the AAI employer survey provides an estimate of the expected response rate.

Table B.1. Estimated population and sample of respondents to the Survey of PIA apprenticeship intermediary organizations (AIO), partners, and community of practice members

Type of respondent	Estimate of population	Respondents sampled
Apprenticeship Intermediary Organization (AIO)	61	6
Apprenticeship partners	78,0002	30
Community of practice members	541	54
Total	78,060	90

¹ Estimated through conversations with PIA program staff.

² The U.S. Department of Labor reports that 26,000 apprenticeship programs are active in the United States. “Apprenticeship partners” include employers, related technical instruction providers, and other supportive service providers. This estimate assumes that all 26,000 programs include one employer partner, on related technical instruction provider, and one supportive services provider. Actual apprenticeship partners may vary and PIA will only work with some apprenticeship partners.

Topic guide for site visit interviews with AIO program staff and partners. The topic guides will be used in semi-structured interviews on site visits with AIO program staff and partners. The study team will conduct site visits with all six anticipated PIA AIOs, so no statistical methods will be used to select AIOs for inclusion in the site visits. If PIA works with more than the anticipated six AIOs, the study team will use purposive sampling to select sites to visit based on information provided by PIA on the importance of the AIO to PIA’s activities and to ensure that a cross section of AIOs are included. In this case, no statistical methods will be used to select sites to visit. For the semi-structured interviews conducted on the site visits, the study team will select twelve AIO and partner staff based on information collected from PIA, work plans and quarterly reports, and recommendations from the AIO. No statistical methods will be used to select the AIO and partner staff to include in the semi-structured interviews as the sample is intended to be neither random nor representative. Instead, the study team will use purposive sampling to select interview subjects based on information provided by PIA and the partner on the importance of the respondent PIA’s activities and to ensure that a cross section of staff are interviewed. Estimated population and samples are provided in Table B.2.

Focus group guide for apprentices enrolled in inclusive apprenticeship programs. The focus group guides will be used in focus groups with apprentices enrolled in inclusive apprenticeship programs supported by PIA AIOs. The study team will conduct a focus group with all six anticipated PIA AIOs, so no statistical methods will be used to select AIOs for inclusion in the focus groups. If PIA works with more than the anticipated six AIOs, the study team will use purposive sampling select apprenticeship programs to include in the focus groups based on information provided by PIA and the AIOs. In this case, no statistical methods will be used to apprenticeship programs to include in the focus groups. For the focus groups with apprentices, the study team will select ten apprentices associated with the identified inclusive apprenticeship programs based on information collected from the selected apprenticeship programs themselves. No statistical methods will be used to select the apprentices to include in the focus group as the sample is intended to be neither random nor representative. Instead, the study team will use purposive sampling to select focus group participants based on the interest of apprentices and information provided by PIA and the partner. Estimated populations and samples are provided in Table B.2.

Table B.2. Estimated population and sample of respondents to site visit interviews and focus groups

Type of respondent	Estimate of population	Respondents sampled
Apprenticeship Intermediary Organization (AIO) program staff participating in interviews	601	12
Apprenticeship partners participating in interviews	78,0002	12
Apprentices participating in focus groups	636,0003	20
Total	714,060	44

¹ Assumes each Apprenticeship Intermediary Organization (AIO) has ten staff members.

² The U.S. Department of Labor reports that 26,000 apprenticeship programs are active in the United States (https://www.dol.gov/agencies/eta/apprenticeship/about/statistics/2020). “Apprenticeship partners” include employers, related technical instruction providers, and other supportive service providers. This estimate assumes that all 26,000 programs include one employer partner, on related technical instruction provider, and one supportive services provider. Actual apprenticeship partners may vary and PIA will only work with some apprenticeship partners.

³ The U.S. Department of Labor reports that 636,000 apprentices active in the United States (https://www.dol.gov/agencies/eta/apprenticeship/about/statistics/2020).

B.2. Procedures for the collection of information

Data for the study will be collected through online surveys, semi-structured interviews, and phone interviews, focus groups, and are all one-time data collection efforts.

The survey will be programmed and administered using Qualtrics. This survey software offers a user interface that is modern, secure, and easy to navigate for respondents. The software will also facilitate generation of tabulations of responses as surveys are completed by respondents and processed. The survey will be hosted on the Internet via a live secure web-link. To reduce respondent burden, it will employ the following: (1) secure log-ins and passwords so respondents can save and complete the survey in multiple sessions; (2) drop-down response categories so respondents can quickly select from a list; (3) dynamic questions and automated skip patterns so respondents only see those questions that apply to them (including those based on answers provided previously in the survey); and (4) logical rules for responses so respondents’ answers are restricted to those intended by the question. The surveys are provided in Attachment A.

The interviews and focus groups will be semi-structured and conducted virtually or, if possible, in-person. We describe the procedures for conducting the interviews and focus groups for each instrument in more detail below.

Topic guide for site visit interviews with AIO program staff and partners. The study team will conduct virtual semi-structured interviews with key AIO program and partner staff. For each of the AIOs selected, the member of the two-person team primarily responsible for logistics will make initial contact by phone with the individual listed as the primary contact in PIA’s records. The team will then send an e-mail to inform the AIO organization of the study and request its cooperation. The initial telephone contact will provide background about the AIO and seek additional information on its partners in order to identify key respondents. Based on this information, the team will contact respondents and determine the best timing for the interview in order to accommodate the schedule of local respondents. The interview topic guides are provided in Attachment B.

Focus group guide for apprentices enrolled in inclusive apprenticeship programs. The study team will conduct virtual semi-structured focus groups with apprentices enrolled in inclusive apprenticeship programs supported by PIA AIOs. For each of the apprenticeship programs selected, the member of the two-person team primarily responsible for logistics will make initial contact by phone with the individual listed as the primary contact in PIA or the AIO’s records. The team will then send an e-mail to inform the apprenticeship program of the study and request its cooperation. The initial telephone contact will provide background about the program and seek additional information on its apprentices in order to identify potential focus group participants. Based on this information, the team will contact apprentices and determine the best timing for the focus group in order to accommodate the schedule of the apprentices. The focus group guides are provided in Attachment C.

Data will be analyzed using descriptive analysis of key variables, including presentation of the data in tables and graphs. When the differences in a key variable between two groups are discussed, the study team will use two-sample t-tests to report whether those differences are statistically significant. The study team will not conduct multivariate analysis of the results. Information collected during site visit interviews and focus groups will be analyzed thematically and tabulated, when appropriate. The study team will use pairwise deletion in the case of missing data.

B.3. Methods to maximize response rates and minimize nonresponse

The study team will make use of best practices to encourage high response rates while minimizing burden and non-response. For the surveys, these methods include:

Web administration. It is anticipated that most respondents will prefer to complete the surveys online. This is based on our experience administering the AAI Grantee Survey (which is analogous to AIO respondents) and the AAI Employer Survey (which is most analogous to partners). All 45 AAI Grantee Survey respondents elected to respond online, 66 of the 68 AAI Employer Survey respondents elected to respond online.³ This choice allows the respondent to complete on their own schedule and pace, as well as complete the survey over multiple sessions. The web survey system used by the data collection team also supports mobile browsers, such as tablets or cellular phones.

Multiple modes of administration. To comply with Section 508 of the Rehabilitation Act, participants who may have difficulty completing a web survey will be offered the option of completing the surveys by telephone.

Technology to reduce burden. To reduce burden, the surveys will employ drop-down response categories so respondents can quickly select from a list, dynamic questions and automated skip patterns so respondents only see those questions that apply to them (including those based on answers provided previously in the survey), and logical rules for responses so respondents’ answers are restricted to those intended by the question. These features should minimize data entry burden by participants and facilitate high quality responses.

Tested questionnaire. The study team will test the survey for clarity and length. Following programming the survey, the study team will conduct formal testing using 2-3 PIA staff and experts at Urban. Fewer than 10 testers will be involved in testing the instrument.

For the interviews, we expect that all PIA program staff identified will agree to participate. We will work with the lead AIO staff person to identify AIO staff and partners who would be willing to participate. We will then work with the local lead organization contact to identify organization staff and partner staff. For the apprentice focus group, we will work with the primary contact person for the apprenticeship program to identify appropriate apprentices and to schedule focus groups. We will use PIA and AIO program staff to provide introduction to partners and discuss importance of the study to increase partner participation in interviews.

B.4. Individuals consulted on statistical aspects of design and on collecting and/or analyzing data

Staff responsible for overseeing the collection and analysis of data are listed in Table B.1 and individuals consulting on the efforts are listed in Table B.2.

Table B.1. Individuals overseeing the collection and analysis of data for the Research to Support the Partnership on Inclusive Apprenticeship (PIA)

The Urban Institute

Lauren Eyster Principal Investigator Diana Elliott Project Director Daniel Kuehn Task Director

Table B.2. Individuals consulting on the collection and analysis of data for the Research to Support the Partnership on Inclusive Apprenticeship (PIA)

The Urban Institute	Lauren Eyster Principal Investigator Diana Elliott Project Director Daniel Kuehn Bhavani Arabandi Task Director
Office of Disability Employment Policy	Michael Reardon Supervisory Policy Advisor, Employment-Related Support Policy Division of Policy Development Scott Robertson Senior Policy Advisor

1 Gardiner, Karen, Daniel Kuehn, Elizabeth Copson, and Andrew Clarkwest. 2021. Expanding Registered Apprenticeship in the United States: Description of American Apprenticeship Initiative Grantees and Their Programs. Report prepared for the U.S. Department of Labor, Employment and Training Administration. Abt Associates, Rockville, MD; and Urban Institute, Washington, DC.

2 Ibid.

3 Gardiner, Karen, Daniel Kuehn, Elizabeth Copson, and Andrew Clarkwest. 2021. Expanding Registered Apprenticeship in the United States: Description of American Apprenticeship Initiative Grantees and Their Programs. Report prepared for the U.S. Department of Labor, Employment and Training Administration. Abt Associates, Rockville, MD; and Urban Institute, Washington, DC.

1