EF_Eval_PRA_Package_Part_A_ 2014-07-02

SUPPORTING STATEMENT

Employment First State Leadership Mentoring Program Community of Practice Survey

General Instructions

A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(i)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below, and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When the question “Does this ICR contain surveys, censuses or employ statistical methods” is checked "Yes,” Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.

Specific Instructions

A. Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

This ICR is for a new data collection (Web survey) associated with the Evaluating and Documenting the Initial Results of the Office of Disability Employment Policy’s Employment First State Leadership Mentoring Program (EFSLMP) study. The EFSLMP Evaluation includes a new data collection that is a survey of 30 State government teams involved in the EFSLMP Community of Practice (CoP), the activities of which are being funded by the U.S. Department of Labor (DOL), Office of Disability Employment Policy (ODEP). The CoP provides monthly technical assistance sessions, webinars, and discussion forums designed to facilitate collaboration and sharing of information, approaches, and practices in promoting employment for individuals with significant disabilities.

In the last decade there has been a movement among advocates and State- and local-level agencies to adopt “Employment First” (EF) as a guiding, integrated employment principle for individuals with disabilities. EF refers to policies and practices that support integrated employment as the first option for youth and adults with significant disabilities (Niemiec, Lavin, & Owens, 2009). Integrated employment refers to “jobs held by people with disabilities in typical workplaces, where: the preponderance of persons employed do not have disabilities; they earn the greater of minimum or prevailing wages; they are paid directly by the employer and qualify for employer-related benefits; and there is an opportunity for career advancement” (Lowe, 2012, p. 4). Payment of prevailing wages may be required by law under the Davis-Bacon and Related Acts (Federally financed or assisted construction contracts), McNamara-O’Hara Service Contract Act (Federal service contracts) or State or local law. EF is a response to the widespread preferential use of Community Rehabilitation Programs (CRPs) that pay sub-minimum wages and include non-work day activities. This type of employment still prevails across the country despite the commitment of several states to workplace integration and the compelling empirical evidence that individuals with significant disabilities prefer integrated employment when provided with adequate supports (Migliore, Mank, Grossi, & Rogan, 2007) and achieve better outcomes in terms of wages, benefits, and diversity of industry placement (Boeltzig, Scott Gilmore, & Butterworth, 2006). In fact, despite the evidence in favor of integrated, community-based employment, in recent years the percentage of individuals receiving integrated employment services across the country fell (Butterworth et al., 2011), and public funds still overwhelmingly go to CRPs and other types of nonintegrated employment (DiLeo & Rogan, 1999). Promoting and disseminating EF practices and strategies is a critical step toward addressing the persistent problem of low participation rates of individuals with disabilities in the workforce and fulfilling the promise of full integration.

ODEP has been involved in promoting integrated employment since the agency’s inception in 2001 and has been a leader in the development and dissemination of promising practices such as self-employment and customized employment for individuals with disabilities. In 2007, ODEP Assistant Secretary Neil Romano prepared a memorandum emphasizing the importance of EF as the basis for a national movement that supports integrated employment at or above minimum wage for people with disabilities (Romano (ODEP), 2009). Under the EF approach, integrated employment is the first option for youth and adults with significant disabilities, and refers to jobs held by people with the most significant disabilities in typical workplace settings where the majority of persons employed are not persons with disabilities. In these jobs, the individuals with disabilities earn wages consistent with wages paid to workers without disabilities in the community performing the same or similar work; the individuals earn at least minimum wage, and they are paid directly by the employer. In 2011, ODEP developed an Integrated Employment Toolkit to provide valuable information to individuals, community employment agencies, policymakers, and others that pursue integrated employment as the desired employment goal for youth and adults with disabilities. This Toolkit includes practical and, in some cases, adaptable information and documents to facilitate the movement of states, organizations, and, most importantly, youth and adults to integrated employment as their primary option for employment (http://www.dol.gov/odep/ietoolkit/). ODEP also has promulgated the use of customized employment, universal design for employers, and self-employment as proven evidence-based practices that lead to employment for people with disabilities.

To advance EF, ODEP created the EFSLMP. This program helps states align policies, regulations, and funding priorities to encourage integrated employment as the primary outcome for individuals with significant disabilities. Through this initiative, ODEP is providing support and informational resources for selected states that desire systems change reflecting the EF approach, but have struggled to fully implement it as the primary service delivery system for people with disabilities. ODEP is also funding a Web-based capacity building and information sharing forum in the form of the EFSLMP CoP.

As previously noted, this ICR is being submitted to obtain clearance under the Paperwork Reduction Act for a new data collection survey of the 30 states associated with the EFSLMP’s CoP. This data collection is essential to the measurement of program implementation, stakeholder engagement and satisfaction with technical assistance, and potential for replication, and will provide the DOL with important information for strategic planning, program replication, and the development of integrated employment policies and practices. This data collection is also designed to gauge the effectiveness of ODEP’s efforts to promote the implementation of EF policies and practices and determine how well remote training and online forums facilitate the implementation of EF activities in each participating state.

As part of the CoP, since the fall of 2012 ODEP has been conducting monthly technical assistance webinars and other information dissemination activities via its electronic portal known as ePolicyWorks. CoP memberships were offered to states that had applied to receive the EFSLMP grant but were not selected, states that received the Administration on Intellectual and Developmental Disabilities (AIDD) grant, and those that applied but did not receive the AIDD grant. However, many other states have shown interest in participating in the CoP due to its strong reputation among State agencies involved the EF Initiative; the number of participants has increased month-to-month to the current 30 states. To participate in the CoP, states must include representatives from six State systems that are connected to employment policy and funding for individuals with disabilities. These systems include:

1. Department of Rehabilitation Services;

2. Department of Mental Health;

3. Department Intellectual and Developmental Disabilities;

4. Workforce Development system;

5. Department of Education (Special Education Division); and

6. Medicaid agencies.¹

The goal of the technical assistance provided through the EFSLMP CoP is to increase the capacity of participating states to implement policies, practices, and approaches that place integrated employment as the priority outcome for individuals with disabilities.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

ODEP is conducting a survey of participating CoP states to document and analyze the development and implementation of EF policies and practices. The goals of this effort are as follows:

• Identify the kinds of EF policies and practices that exist in participating states.

• Document the factors that promote or impede knowledge transfer and the adoption and implementation of EF practices and approaches among participating states.

• Assess the effectiveness of virtual collaboration tools in promoting knowledge transfer and increasing State capacity to promote and implement EF practices and approaches.

• Assess the effectiveness of the CoP to engage AIDD’s Employment Systems Change grantees, as well as other State EF leaders from the six State systems.

To achieve these goals, the study will rely on a survey of respondents from each of the State systems from the CoP states. The research questions to be addressed in the CoP survey are as follows:

• What are the factors that contribute to or impede knowledge transfer related to the implementation of information, policies, and approaches covered in CoP webinars?

• How effective are virtual and in-person collaborations in increasing preparedness and capacity of states interested in implementing EF policy practices and approaches?

• What is the impact of technical assistance on State capacity to pursue and implement EF practices and approaches?

In order to join in the CoP, states must nominate individual representatives from each of the previously specified six State systems. The nominated individuals will serve as the respondents for the Survey of CoP Participating States. Findings from the survey will allow DOL to understand:

• the EF policy landscape across states;

• levels of interest, adoption, and implementation of EF practices;

• the effectiveness of the technical assistance provided by ODEP via the Web and other online portals and discussion forums; and

• stakeholder recommendations and State needs in terms of EF policy and technical assistance.

The EF efforts vary tremendously across states, but also across State systems. CoP activities have also focused at times on topics that are more germane to a specific State system. It is, therefore, critical that we understand how the landscape presents itself across systems (i.e., agencies) in order to better tailor ODEP’s systems change efforts on behalf of Employment First.

Finally, the CoP survey will provide ODEP with information regarding the effectiveness of the technology used to facilitate knowledge transfer that results in changes in State policies, approaches, and practices related to EF. As a policy agency, this information will help ODEP measure its Government Performance and Results Act (GPRA) outcome of producing policy-relevant and actionable documents, events, and collaborative opportunities that result in knowledge transfer and the utilization of its technical assistance efforts. The effectiveness of the agency’s efforts rests on being able to develop policy materials that meet precise needs and measure the impact of its work on State agencies.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The EFSLMP survey will use a Web-based delivery system that will allow respondents to take the survey at any time within the window of the data collection period. Participation is voluntary and the Web system will show respondents the privacy requirements. Respondents will receive invitations and reminders to complete the survey via e-mail. Completion rates will be tracked in real time. The Evaluation Team will follow up with respondents who do not complete the questionnaire via telephone in an attempt to increase response rates. This will occur on a weekly basis. The Web system used for this data collection will be compliant with Section 508 requirements of the amended Rehabilitation Act of 1973. All survey data will be stored in secure data servers.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The CoP survey includes one new data collection activity and does not cover information that has been collected previously. The information collected seeks to assess the impact and effectiveness of the technical assistance provided online through the EFSLMP, as well as the extent to which the EFSLMP influences the policies and practices of participating states. ODEP has not found this information elsewhere and believes it has never been collected before. The EFSLMP Web survey will gauge the overall influence of the EFSLMP on State policies and practices, as well as the satisfaction of EFSLMP activities among participants.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small entities will be involved in this study.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The collection of this data is critical to ODEP’s efforts to continue to influence policy to improve employment outcomes for individuals with disabilities. ODEP believes the data for this research has not previously been collected and the results will offer guidance to DOL in how to assist states in the implementation of EF policies. Currently many states are developing EF initiatives, but do not have access to information about other states’ efforts to help inform their process or to evaluate their efforts and how they compare to other states. Not collecting this information means that the effectiveness of the EFSLMP cannot be determined. In addition, it will mean not being able to identify which EFSLMP activities are most effective and which activities should be revised or discontinued. Not conducting the EFSLMP Web survey may lead to inefficiencies that potentially waste tax dollars as well as negatively impact the lives of people with disabilities. The CoP was created to promote knowledge sharing among states and the dissemination of best practices through technical assistance webinars led by experts in integrated employment for individuals with disabilities. The evaluation of the EFSLMP will contribute to more effective knowledge dissemination practices and more positive outcomes in the employment of individuals with disabilities.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

* requiring respondents to report information to the agency more often than quarterly;

* requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

* requiring respondents to submit more than an original and two copies of any document;

* requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

* in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

* requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

* that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

* requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no special circumstances associated with this information collection.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported. Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every three years—even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

On April 30, 2014, ODEP published a notice in the Federal Register to provide the public with an opportunity to submit comments over a 60-day period. See 79 FR 24453. ODEP received no comments in response to the notice.

In developing the survey instrument for this data collection effort, ODEP consulted with disability experts from each of the 6 State systems to be included in this data collection, namely the Department of Rehabilitation Services; the Department of Mental Health; the Department Intellectual and Developmental Disabilities; the Workforce Development system; the Department of Education (Special Education Division); and Medicaid agencies. ODEP convened conference calls with these experts to go over and refine the data collection instrument. The names and area of expertise for each person consulted is listed below.

Persons Consulted on Data Collection Instrument:

• David Mank, University of Indiana, expertise in community employment and transition.

• Michael Morris, Burton Blatt Institute and National Disability Institute, expertise in the Workforce Development system, employment of people with disabilities, asset development.

• Rie Kennedy-Lizotte, National Association of State Directors of Developmental Disabilities Services, expertise in EF funding sources, rate and rates reimbursement, policy orientation.

• Lisa Mills, Moving to a Different Drum: Disability Policy and Services Consulting, expertise in Medicaid funding, Wisconsin Medicaid Infrastructure Grant.

• Melodie Pazolt, Director of Employment Services in Mental Health, Washington State, expertise in mental health. Stephen Wooderson, Council of State Administrators of Vocational Rehabilitation, former VR Commissioner of Iowa, expertise in vocational rehabilitation.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No incentives will be offered for this voluntary survey.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The Web survey will include the following privacy statement:

Responses to this data collection are voluntary and will be used only for statistical purposes. Reports prepared for this study will summarize findings across the entire CoP and will not associate responses with a specific state or individual. We will not provide information that identifies you or your state to anyone outside the study team, except as required by law. Responses are intended to be anonymous; therefore, please do not include any person identifiers (e.g., your name, address) in your response.

Responses to this data collection are voluntary. Reports prepared for this study will summarize findings across the sample and will not associate responses with specific individuals. We will not provide information that identifies participants to anyone outside the study team, unless required by law.

Personal information will not be collected, and respondents will be asked not to provide it. Respondents will be fully informed about the purpose of this study and the names of respondents will not be included in any reports from the study. Completed surveys will be maintained by the contractor in a password-protected database. Comments made through the surveys and interviews will not be attributable to specific individuals.

Direct identifying information will be removed and indirect identifying information will be top-coded and recoded and combined so that they are no longer identifiable. In all published reports, participants will be combined so that no single participant is identifiable.

Privacy is an important part of the study design. For this reason, Social Dynamics will ensure the privacy of all individuals who provide data. A pledge of privacy is a major positive incentive for potential respondents to participate in the study. Its absence would be a significant deterrent and could create complications in implementing the study. Social Dynamics will take the following precautions to ensure the privacy of all data collected:

• All Social Dynamics staff and subcontractors, including analysts, coders, editors, and keypunchers, will be instructed in the privacy requirements of the study and will sign statements affirming their obligation to maintain privacy;

• Information will be reviewed and data will be cleaned only by Social Dynamics and its subcontractor staff;

• Data files will contain no personal identifiers for program participants; and

• Analysis and publication of study findings for the participant survey will be in terms of aggregated statistics only.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument includes no sensitive questions.

12. Provide estimates of the hour burden of the collection of information. The statement should:

* Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

* If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.

* Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included under “Annual Cost to Federal Government.”

The PRA requires the agency to account for the amount of burden that it is placing on the public when seeking information on behalf of the Federal government. This burden is measured in terms of hours (see Table A-1) and includes the following activities:

• Reviewing instructions;

• Using technology to collect, process, and disclose information;

• Adjusting existing practices to comply with requirements;

• Searching data sources;

• Completing and reviewing the response; and

• Transmitting or disclosing information.

Respondents to this collection of information are representatives of the six State systems who have participated in the CoP sessions (6 per state, for all 30 states in the CoP).

Table A-1. Estimated Hours of Burden Due to CoP Survey

State	State Name	State Agency Lead			Total Hours/ State	Total Time Value/State
		# of Resp.	Hours/ Response*	Time Value/ Response
1	Alaska	6	0.26	$10.28	1.56	$61.68
2	Alabama	6	0.26	$10.28	1.56	$61.68
3	Arkansas	6	0.26	$10.28	1.56	$61.68
4	California	6	0.26	$10.28	1.56	$61.68
5	Colorado	6	0.26	$10.28	1.56	$61.68
6	Washington, DC	6	0.26	$10.28	1.56	$61.68
7	Delaware	6	0.26	$10.28	1.56	$61.68
8	Florida	6	0.26	$10.28	1.56	$61.68
9	Georgia	6	0.26	$10.28	1.56	$61.68
10	Iowa	6	0.26	$10.28	1.56	$61.68
11	Idaho	6	0.26	$10.28	1.56	$61.68
12	Illinois	6	0.26	$10.28	1.56	$61.68
13	Indiana	6	0.26	$10.28	1.56	$61.68
14	Kentucky	6	0.26	$10.28	1.56	$61.68
15	Maine	6	0.26	$10.28	1.56	$61.68
16	Maryland	6	0.26	$10.28	1.56	$61.68
17	Michigan	6	0.26	$10.28	1.56	$61.68
18	Mississippi	6	0.26	$10.28	1.56	$61.68
19	Missouri	6	0.26	$10.28	1.56	$61.68
20	Montana	6	0.26	$10.28	1.56	$61.68
21	North Carolina	6	0.26	$10.28	1.56	$61.68
22	North Dakota	6	0.26	$10.28	1.56	$61.68
23	New Jersey	6	0.26	$10.28	1.56	$61.68
24	New York	6	0.26	$10.28	1.56	$61.68
25	Oregon	6	0.26	$10.28	1.56	$61.68
26	Rhode Island	6	0.26	$10.28	1.56	$61.68
27	Tennessee	6	0.26	$10.28	1.56	$61.68
28	Virginia	6	0.26	$10.28	1.56	$61.68
29	Washington	6	0.26	$10.28	1.56	$61.68
30	Wisconsin	6	0.26	$10.28	1.56	$61.68
Total		180.00	0.26	$10.28	46.8	$1850.4

*State agency staff will participate in a CoP Survey that will be 15–20 minutes in duration based on an estimate of burden from a pilot test. For purposes of estimating burden, ODEP assumes the average time to be 0.26 hours per response.

State Employees in Management Occupations Based on Bureau of Labor Statistics Overall Average Mean Hourly Rate: $39.54 retrieved from http://www.bls.gov/oes/current/naics4_999200.htm#11-0000

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

* The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

* If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collections services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

* Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

No other costs to respondents or record keepers are anticipated.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

Agencies may also aggregate cost estimates from Items 12, 13, and 14 in a single table.

The sum of costs to administer the CoP survey are estimated to be $156,350 (see Table A-2). These figures were estimated as the sum of the anticipated direct labor; fringe and burden on direct labor; other direct costs associated with staffing the site visits (e.g., computers, software, telephone, reproduction, shipping, mail, travel and per diem, general and administrative costs, and contractor fee); and staffing the design and implementation, including all equipment and supplies (e.g., computers, software, telephone, reproduction, shipping, mail, travel and per diem, general and administrative costs, and contractor fee). Estimated time to complete the CoP survey is six months. After the OMB approval, information collection will take less than one year, therefore there is no concern for annualized costs.

Table A-2. Breakdown of Costs for the CoP Survey

CoP Survey Tasks		COST
Pilot Testing		$12,934
Training and TA		$4,230
Programming		$15,100
Monitoring Response		$16,239
Follow-Up with Respondents		$22,817
Analysis		$69,618
Federal Staff Cost to Administer		$15,469
TOTAL		$156,407

Federal Staff Cost to Administer
Federal Salary	Hourly Rate* DC Based GS-13, Step 5	Hours for CoP Survey	Total Federal Salary	Daily Expenses (M&I) Per Diem **	Total Expenses (M&I) 40 days	Total Federal Cost to Administer
GS-13 Step 5	$48.83	180	$8,789	$167	$6,680	$15,469
* http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2014/salhrl.pdf at 32
** http://www.gsa.gov/portal/category/100120

15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.

The CoP survey is a new data collection (Web survey).

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The data collected from the ODEP CoP questionnaire will be used to prepare descriptive statistics tables and graphs on the issues related to EFSLMP CoP efforts in the 30 states participating in the initiative. The descriptive statistics will detail whether efforts are being undertaken at the State level to promote integrated employment, the public agencies that are in charge of those efforts, the extent to which there is interagency collaboration, perceived benefits of this collaboration, and barriers to the adoption and implementation of EF policies and practices. A comparison of agencies’ barriers and efforts to implement EF strategies will be highlighted. Finally, the CoP questionnaire also includes questions designed to assess the technical assistance received by each State agency through the EFSLMP and recommendations for the EFSLMP CoP, as well as ratings of development/implementation of EF strategies.

The analysis will provide descriptive information with bivariate statistical presentations in tabular and graphical form. Initially, the analysis will focus on respondent characteristics (i.e., education level, position title, years employed in position), EFSLMP participation, training and technical assistance (T&TA) needs, and development/implementation of EF activities at the State level. Descriptive information will highlight differences among groups of respondents and the factors associated with variability in development/implementation of EF activities.

The analysis will begin with a description of the respondent sample, including the following information: agency affiliation, length of time at agency, job title, and length of time in current position:

Question 1: At what agency are you employed?

Question 2: How long have you worked at this agency?

Question 3: What is your job title?

Question 4: How long have you held this position?

Next, the analysis will provide graphical displays and bivariate cross-tabulation tables using Questions 1–4 (as independent variables) and Questions 5 and 6, which include information on external agencies that provide T&TA, as dependent variables.

Question 5: Excluding ODEP’s EFSLMP CoP, what additional agencies or organizations have you and your colleagues working on your state’s Employment First efforts received technical assistance from in the last two years?

Question 6: Please indicate all the topics that you and your colleagues working on your state’s Employment First efforts received T&TA from in the last two years from the sources you selected in question 5.

The cross-tabulation analyses will highlight differences in the distribution among groups on the variables of interest (e.g., types of training received and training sources).

Interval-level response categories are used in Question 7 to rate respondents’ level of satisfaction with each type of T&TA practice on a 5-point interval-level scale. This question will be presented using descriptive analyses, including frequency distributions and cross-tabulations to examine group differences using respondent characteristics from Questions 1–6.

Question 7: Listed below are 12 features of the ODEP’s CoP T&TA provided to CoP states. Please rate your level of satisfaction for each of the T&TA characteristics.

a.) Information presented during webinars

b.) Quality of speakers during webinars

c.) Level of interaction among participants

d.) Opportunities to ask questions and get feedback

e.) Frequency and duration of webinars

f.) Relevance of topics covered in monthly webinars

g.) Real-world examples from CoP states

h.) Quality of contacts made with other states through participation in the CoP

i.) ePolicyWorks systems

j.) Monthly CoP Electronic Bulletins

k.) Annual In-Person National EFSLMP CoP meeting (held in November 2012)

l.) Opportunities to provide feedback to Federal policymakers via participation

Questions 8 and 9 also use a 5-point rating scale. A graphical display will be used to identify group differences on mean satisfaction ratings using respondent characteristics from Questions 1–6 as independent variables.

Question 8: Below please indicate whether you have used what you have learned from participating in the CoP to influence your state’s Employment First planning and implementation efforts.

Yes

No

No Opportunity

Question 9 of the EFSLMP CoP questionnaire collects information on the “state of development/implementation” for 5 types of State EF activities:

1. Strategic Planning

2. Policy Development & Operational Alignment

3. Interagency Collaboration

4. Funding and Reimbursement

5. Outcomes and Measures

Question 9: Listed below are six components of Employment First that have been topics of discussion during various CoP technical assistance events. Please select the "stage of development/implementation" in your state agency for each of these components using the following scale:

1. Not yet present

2. In its infancy (e.g., early conversations are happening)

3. Under development (e.g., beginning to formulate, conceptually planning, drafting language, etc.)

4. In existence (e.g., approved, formalized, beginning to be operationalized)

5. Fully implemented (e.g., fully established, monitored with a focus on improvement, and used to guide policy development)

6. Status unknown

Question 9 will provide DOL with a self-rated estimate of the progress that each state has made to date, which will help to target future T&TA efforts. The development/implementation indicator for each of the components will also be used to create quantitative scores (e.g., average/median scores, and an individual score for each component).

The qualitative data for the EF evaluation derive from questions 10 and 11, which are open-ended questions that ask respondents to describe their most valuable experiences as an EFSLMP CoP participant, as well as recommendations for improving the EFSLMP CoP.

Question 10: Please describe what you thought was most valuable in your experience with the EFSLMP CoP.

Question 11: Please provide recommendations for how you would improve the EFSLMP CoP in the future.

Using this information, a content analysis will be conducted using a rigorous approach to qualitative data analysis using principles of Grounded Theory modified to fit the scope and purpose of the EF Evaluation (Glaser & Strauss, 1967). As is the case with most qualitative analyses, the data require reduction. Key to this effort is the development of a coding scheme that is aligned with the conceptual framework of the overall evaluation. We will create a hierarchy of conceptual categories linked to the evaluation research questions. The categories, or “codes,” at the top of the hierarchy will, at a minimum, align with the three research questions presented in section 2 of Part A of this package. Also included as “top codes” will be the T&TA activities provided to the EFSLMP CoP.

The DOL will utilize information from the CoP questionnaire to inform its strategic planning process, promote best and promising practices, and help states in adopting and implementing EF initiatives. Any published materials will be used to inform policymakers and other stakeholders on EF policy and technical assistance. A final report will be made available at the DOL’s Web site. Additionally, materials may be prepared for journals, conferences, and associates interested in the subject. Findings will be used to improve technical assistance materials and methods sponsored by the DOL. We anticipate that the Web survey will be sent out to respondents on October 1, 2014, after receiving PRA clearance. The data collection will last until the end of November, 2014. ..

Deliverables	Performance period
OMB PRA Clearance for CoP survey	September 1, 2014
CoP Survey Data Collection Implementation	October – November 2014
Data Analysis and Report Development	December 2014
Final Report Draft	January 2015
Final Report	February 2015

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The Web site will display the expiration date.

18. Explain each exception to the topics of the certification statement identified in “Certification for Paperwork Reduction Act Submissions.”

ODEP seeks no exceptions to the certification statement.

References

Boeltzig, H., Scott Gilmore, D., & Butterworth, J. (2006). The national survey of community rehabilitation providers, FY2004-2005 Report 1: Employment outcomes of people with developmental disabilities in integrated employment. Research to Practice 44. Retrieved from http://www.communityinclusion.org/article.php?article_id=177

Butterworth, J., Cohen Hall, A., Smith, F. A., Migliore, A., Winsor, J., Ciulla Timmons, J., & Domin, D. (2011). StateDate: The national report on employment services and outcomes. Retrieved from http://www.communityinclusion.org/pdf/statedata2010_finaldraft.pdf

DiLeo, D., & Rogan, P. (1999). Toward integrated employment for all: APSE's position on segregated services for people with disabilities. Retrieved from http://choices.muskie.usm.maine.edu/conversion/TowardIntegratedEmploymentAll.pdf

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing Company.

Lowe, S. (2012, August 14). The employment first state leadership mentoring program. [PowerPoint presented at the DEI Project Leads Monthly Administrative Call].

Migliore, A., Mank, D., Grossi, T., & Rogan, P. (2007) Integrated employment or sheltered workshops: Preferences of adults with intellectual disabilities, their families, and staff. Journal of Vocational Rehabilitation, 26(1), 5–19.

Niemiec, B., Lavin, D., & Owens, L. A. (2009). Establishing a national employment first agenda. Journal of Vocational Rehabilitation, 31(3), 139–144. 

Romano, N. (ODEP). (2009). “Employment first” creating opportunities to move to integrated employment (Memorandum). Retrieved from http://wayback.archive-it.org/org-334/20090116010812/http://www.dol.gov/odep/events/employmentfirst/employmentfirst.htm

1 States may use different names for each of the six State systems listed above.

- 15 -