Supplemental Statistical Methods Supporting Statement for Add Us In Evaluation

The Office of Disability Employment Policy (ODEP) of the U.S. Department of Labor (DOL) requests clearance to conduct a web-based survey of Add Us In (AUI) grantees’ workforce system customers. AUI grantees seek to increase the number of people with disabilities employed by small businesses by collaborating and coordinating with businesses owned and operated by women, minorities, people with disabilities, veterans, and lesbian, gay, bisexual, and transgender individuals. AUI grantees also seek to implement strategies to increase the capacities of these targeted businesses to provide integrated employment opportunities for people with disabilities.

AUI grantees face many challenges to fostering systems change, including the frequent lack of coordination in current efforts by workforce, disability services, and other organizations involved in promoting employment of people with disabilities (PWD), and difficulties in collaborating effectively across multiple organizational entities in public and private sectors. DOL/ODEP has contracted with IMPAQ International, LLC, to assess systems changes related to AUI grantee efforts. As part of its evaluation, IMPAQ will use a web-based customer satisfaction survey. The AUI evaluation will reveal the specific practices of grantees that have shown promise in overcoming these challenges and that have resulted in positive participant outcomes and workforce systems change.

B1. Respondent Universe and Sampling Methods

1.2 Sampling Design / Respondent Selection

This survey will be conducted with a purposive sample. Grantees will recommend respondents from any of the groups below, with which they have a noteworthy partnership:

Chambers of Commerce
Business Associations
Vocational Rehabilitation
American Job Centers
Local Workforce Investment Boards

Grantees will also provide contact individuals at any additional partner entities they believe should be surveyed.

1.3 Potential Respondent Universe

The potential respondent universe is any management-level individual at any business entity partner of any Add Us In grantee.

1.4 Sampling Unit

From the universe described above, grantees are being asked to select those partnerships that they believe are meaningful in terms of employing individuals with disabilities. We will request a contact individual for each partnership entity (Chamber of Commerce, LWIB, etc.).

1.5 Population Frame and

1.6 Estimated Sample Size

AUI grantees submit quarterly reports to their Federal Program Officers. These reports include information on grantee efforts to seek and forge new partnerships. A list of the recipients of AUI outreach and partnership efforts suggests a universe of less than 100 individuals. From this universe, we expect grantees will recommend approximately 80 individuals total.

1.7 Expected Response Rates

We expect an 80 percent response rate for the survey, based on planned pre-survey and follow-up activities coordinated with each grantee director. This will yield a total sample of 64 respondents. Examples of similar efforts that yielded a response rate of 80 percent or higher include the Job Corps National Survey Data Collection Project and Project GATE, both of which were conducted for the Employment and Training Administration at the U.S. Department of Labor.

B2. Statistical Methods for Sample Selection and Degree of Accuracy Needed

2.1 Statistical Methodology for Stratification

N/A

2.2 Sample Selection Methodology

The universe of grantee consortia will be part of this survey effort. From each grantee consortium, we will receive a purposive sample of partners that we will survey.

2.3 Estimates of Variance

N/A

2.4 Analysis Plans

The results of the web survey will be analyzed using basic descriptive statistics. Cross-tabs will allow comparisons of means, standard deviations, etc. Key variables will include:

Organization type
Respondent type
Frequency of grantee contact
Self-reported change in attitude (Y/N)
Self reported change in behavior (Y/N)

2.5 Minimal Substantively Significant Effect

N/A

2.6 Unusual Problems

N/A

2.7 Periodic / Cyclical Data Collection

N/A

B3. Maximizing Response Rates and Addressing Nonresponse

3.1 Methods to Maximize Response Rate / Issues of Non-Response

The survey will be preceded with a notification letter generated by ODEP. When the survey itself is delivered, it will be accompanied by an explanatory e-mail detailing the need for the data collection. A direct, unique web link will be provided for every potential respondent, making the survey very easy to access (i.e. no password or multiple log-ins necessary). The survey is brief (respondents can complete the survey in around 15 minutes). Finally, all non-respondents will receive a follow-up reminder to complete the survey, I week after survey delivery. The survey timeframe will be 6 weeks.

3.1a Nonresponse Bias Analyses

The survey is not intended to be representative. No attempt will be made to draw inferences to any population other than the set of units that responded to the data collection effort.

3.1b Nonresponse Weights

N/A

3.1c Other Procedures to Address Missing Data

N/A

3.2 Accuracy and Reliability of Information Collected

The results from the web survey will be used to explore exemplary cases, according to grantees and their partners. The survey is not expected to be representative, and researchers will not make generalizations based on it.

3.3 Justification for non-systematic data-collection

This survey is intended to collect information from engaged partners. AUI grantees are the best judge of who their most successful or impactful partners are.

B4. Test Procedures

4.1 Test of Procedures and Methods to Minimize Burden and Improve Utility

This survey was tested by 2 experienced employment and training researchers who are not working on this particular project. Each of these pilots was followed by a cognitive interview. As a result of these interviews, minor edits were made to the survey, for clarification purposes.

4.2 Approval for Pilot Tests with 10 or More Respondents

N/A

B5. Contact Information and Confidentiality

5.1 Consultant Contact Information

No uncompensated individuals were consulted on any aspect of this design.

5.2 Analyst Organization Information

All project staff from the IMPAQ evaluation team contributed to the development of the survey, to ensure that the best decisions were made regarding its design. All data collection and analysis will be conducted by the following individuals:

Name	Organization	Phone Number	E-mail Address
Anne Chamberlain	IMPAQ	443.718.4343	[email protected]
Kay Magill		808.225.2944	[email protected]
Linda Toms Barker		808.934.9297	[email protected]

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-29