OMB_Supporting _Stmt_A_ PVRES_8_08_07 3285

U.S. Department of Education

Post Vocational Rehabilitation Experiences Study

Office of Management and Budget

Clearance Package Supporting Statement

And Data Collection Instruments

Part A

May 16, 2007

Revised August 8, 2007

Contents

Page

Introduction 1

A. Justification 2

A1. Circumstances Making the Collection of Information Necessary 2

A2. Purpose and Use of the Collected Information 7

A3. Use of Technology to Reduce Burden 9

A4. Efforts to Identify Duplication 10

A5. Methods to Minimize Burden on Small Entities 11

A6. Consequences if Collection is not Conducted 11

A7. Special Circumstances 11

A8. Federal Register Comments and Consultations Outside the Agency 12

A9. Decision to Provide Any Payment or Gift to Respondents, Other Than Remuneration of Contractors or Grantees 17

A10. Assurance of Confidentiality 17

A11. Justification for Any Questions of a Sensitive Nature 20

A12. Estimates of the Hour Burden 21

A13. Estimate for the Total Annual Cost Burden to Respondents 23

A14. Estimates of Annualized Costs to the Federal Government 23

A15. Reasons for Any Program Changes or Adjustments 24

A16. Plans for Tabulation and Publication, Analytic Techniques, and Time Schedule 24

A17. Approval to Not Display the Expiration Date 28

A18. Explanation of Exceptions 28

Part A References 29

B. Collections of Information Employing Statistical Methods 32

B1. Respondent Universe and Sampling Methods 32

B2. Procedures for the Collection of Information 36

B3. Methods to Maximize Response Rates 41

B4. Tests of Procedures or Methods 53

B5. Consultations on Statistical Aspects of the Design 54

Part B References 55

List of Tables

Page

Table 1 Estimates of VR Agency Respondent Burden – Baseline Only 21

Table 2 Estimates of Survey Respondent Burden 22

Table 3 Estimated Annualized Costs to the Federal Government 23

Table 4 Estimated Costs by Expense Category and Study Phase 24

Table 5 Types of Data Analyses and Usefulness 27

Table 6 Response Rates for Ticket-to-Work Study and the SSI/Medicaid Surveys and Projected Response Rate for PVRES 35

Table 7 Definition of Sampling Strata and Sample Allocation 37

Table 8 Allocated Sample Sizes and Expected Standard Errors in the Second
Followup by Subgroup 38

List of Exhibits

Exhibit A-1 Panel of Experts 15

Exhibit A-2 Anticipated PVRES Schedule 28

List of Appendixes

Page

Appendix A. Crosswalk between Study Questions, Data Elements and

Interview Questions A-1

Appendix B. Survey Instrument B-1

Appendix C. Federal Register Notices C-1

Appendix D. Advance Materials D-1

Appendix E. Confidentiality and Non-Disclosure Forms E-1

Appendix F. Request to State Agencies F-1

Appendix G. List of Non-Response Bias Study Variables G-1

Appendix H. Refusal Confirmation Script H-1

SUPPORTING STATEMENT FOR
PAPERWORK REDUCTION ACT 1995 SUBMISSION
INFORMATION COLLECTION PLAN FOR THE
POST VOCATIONAL REHABILITATION EXPERIENCES STUDY

INTRODUCTION

Little is known about the post-closure needs of individuals served by the State Vocational Rehabilitation (VR) program in terms of maintaining stable employment and advancing in employment. The purpose of this study is to identify the post-closure needs of four subgroups of individuals served by VR: transitional youth, persons with mental retardation, persons with mental illness, and persons who received Social Security disability benefits. Westat will contact a national sample of these former VR consumers whose service records have been closed to determine if they retain and advance in employment and to identify other services and supports they might receive over the period to maintain employment or advance in employment.

A national sample of individuals who exited the VR program following services will be sampled from the 2006 RSA 911 Case Service Report. Westat will locate and contact these former consumers and invite them to participate in the study. Participation will include a baseline data collection and two annual followup surveys.

The surveys will obtain individual level information that will then be aggregated to obtain national-level estimates of post-program outcomes for the four subgroups. As reports and datasets are produced for each wave of the study, results will be published on the study’s own web site and use of the public-use data base by other researchers will be encouraged.

The study will be conducted by Westat and its subcontractor, InfoUse. Ongoing guidance to the study will be provided by a Panel of Experts composed of distinguished members of national associations, administrators, researchers, job placement officials and staff of Federal agencies.

The study will consider:

• the labor market progression of the four subgroups of VR consumers, whose service records have been closed, in terms of employment status, earnings, and receipt of job-related benefits;

• the extent to which these VR consumers continue to obtain services and resources, including post-employment services, and how such services improve employment stability and advancement;

• noneconomic outcomes, such as community integration;

• how receipt of Social Security disability benefits changes over time;

• how the demographic, disability and other characteristics of individuals affect economic and other outcomes; and

• the policy implications of study findings for the VR Services Program and for the long-term employment of individuals with disabilities served by the program.

A. JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

The Rehabilitation Services Administration (RSA) in the Office of the Assistant Secretary for Special Education and Rehabilitative Services (OSERS), Department of Education oversees formula and discretionary grant programs that help individuals with physical or mental disabilities to obtain employment and live more independently through the provision of such supports as counseling, medical and psychological services, job training, and other individualized services and social supports.

RSA’s major Title I formula grant program provides funds to state VR agencies to provide employment-related services for individuals with disabilities, giving priority to individuals who are significantly disabled. While RSA maintains extensive data on VR service provision and outcomes at case closure, little is known about the post-closure needs of former VR consumers in terms of maintaining stable employment and advancing in employment after leaving the VR program.

This data collection is for a longitudinal study of former consumers of the state VR Services Program that focuses on their post-program experiences, with particular emphasis on long-term employment status, earnings, and change in benefits. The study will focus on the post-VR experiences of four subgroups of former VR consumers: (1) persons with mental illness (MI), (2) persons with mental retardation (MR), (3) transitional youth (TY), and (4) persons who received Social Security disability benefits (SSB). These groups have been identified as having specific unique challenges with respect to the established methods of delivery of VR services, including the time-limited nature of the services. The Post Vocational Rehabilitation Experiences Study (PVRES) is designed to determine the degree to which former VR consumers make satisfactory progress in employment, identify post-closure services that may assist them to do so, and discover variables that may impede their long-term success.

RSA conducts evaluations of the Title I program with authorization under section 14(a) of the Rehabilitation Act (P.L. 93-112; P.L. 105-220), which states: “For purposes of improving program management and effectiveness, the Secretary, in consultation with the Commissioner, shall evaluate all the programs authorized by this Act, their general effectiveness in relation to their cost, their impact on related programs, and their structure and mechanisms for delivery of services, using appropriate methodology and evaluation research designs.”

PVRES will be the second national longitudinal study of VR services. The Longitudinal Study of the Vocational Rehabilitation Services Program (LSVRSP) started data collection over 10 years ago in 1995, and completed it in January 2000, with a focus on identifying the relationship between delivery of VR services and economic and noneconomic outcomes. While the study did include one cohort of individuals who had exited VR, the focus was on the outcomes of VR service delivery. The study, while providing a rich resource for identifying consumer, service, and VR office characteristics that contributed to outcomes, does not provide adequate detail on post-closure experience. It was conducted prior to the implementation of significant new employment policies for people with disabilities, including the 1999 Ticket to Work and Work Incentive Improvement Act (TWWIIA) and the Workforce Investment Act of 1998 (WIA), which became effective July 1, 2000. The need for more understanding of post-program experiences is demonstrated in many research studies related to the identified subgroups.

Subgroups of Former VR Consumers

Persons with Mental Illness. People with mental illness represent a substantial percentage of consumers of VR services; about one in five successfully employed clients of VR has a primary disability of mental disorder (Jans, Stoddard, & Kraus, 2004). In addition, mental illness is one of the leading reasons that people receive both Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI); about one in three recipients of these benefits have a mental disability. Mental impairments also are associated with the longest entitlement periods for Social Security benefits (Jans, Stoddard, & Kraus, 2004).

People with severe mental illness often require ongoing services, treatments, and social supports to maintain employment (Shankar & Collyer, 2003). A large study conducted for the Substance Abuse and Mental Health Services Administration (SAMHSA) by PVRES Panel of Experts (POE) member Judith A. Cook found that both clinical and vocational services are necessary for people to find and keep employment and, further, that people who receive well-integrated and coordinated vocational and clinical services have greatly improved outcomes compared to those receiving nonintegrated services (Cook, 2002). On the job, people with psychiatric disabilities often need different kinds of accommodations and supports from those used by people with other kinds of disabilities. These include, for example, job coaches, flexible scheduling, and modified job duties (Akabas, 1994; Center for an Aging Society, 2004; MacDonald-Wilson, Rogers, Massaro, Lyass, & Crean, 2002). Clinical and treatment factors such as psychiatric hospitalizations, use of mental health service, and self-ratings of functioning have also been shown to influence employment outcomes among consumers with psychiatric disabilities (Razzano et al., 2005). In a study of job retention factors that affect sustained employment for homeless people with mental illness, PVRES POE member William McCarriston found that housing, workplace supports, health care, case management and ongoing social supports were critical determinants of job retention for this subgroup (McCarriston, Turner, & Hursh, 2004). While this literature suggests that certain vocational and clinical services as well as workplace and social supports improve employment outcomes for people with mental illness, little is known about how these and other services and supports affect VR consumers with mental illness after closure of VR services.

Persons with Mental Retardation. A comprehensive national review of employment and income status of adults with mental retardation and other developmental disabilities revealed that the vast majority were unemployed and that most had not utilized any employment services (Yamaki & Fujiura, 2002). This finding underlines the importance of understanding more about the factors and services that support stable employment and advancement of people with mental retardation.

Leff and colleagues, conducting a meta-analysis of data from seven sites of the Employment Intervention Demonstration Program, found that job development was a very effective service for acquisition of employment by consumers with severe mental retardation, and that job support was associated with retention of a first job (Leff et al., 2005). They concluded that more research is needed to determine the long-term effectiveness of these services with this population, a goal to which the PVRES study will contribute.

Looking at employed young adults with mental retardation, Butterworth and colleagues found that individuals had higher success rates when they were integrated into the organization through organizational structures and social interaction with coworkers (Butterworth, Hagner, Helm, & Whelley, 2000). Commonly reported accommodations to employees with mental retardation include extra supervision time, providing flexible hours and using the services of a job coach. Especially when these accommodations were in place, employers reported favorable experience employing individuals with mental retardation (Olson, Cioffi, Yovanoff, & Mank, 2001). Recent research has found that the majority of young people with mental retardation are living with their parents and that participation of parents and other social supports affect employment outcomes (Luftig & Muthert, 2006). Research has also shown that consumers with mental retardation who were more self-determined had better employment outcomes (Wehmeyer & Palmer, 2003).

Looking specifically at outcomes for VR consumers with mental retardation, Moore, Feist-Price, and Alston (2002) found that an individual’s ability to obtain competitive employment is influenced by the individual, his or her significant others, and the types of VR services provided. Moore and colleagues also found that those who received job placement services, business/vocational training, and counseling were nearly twice as likely to obtain a competitive job and that those services were generally more available to people with milder mental retardation than those with more severe mental retardation (Moore, Harley, & Gamble, 2004). These studies concern VR consumers with mental retardation at the time of exit from VR. However, little is known about what happens to VR consumers with mental retardation after they complete VR services and what supports and services are most effective in helping them maintain and advance in employment.

Transitional Youth. Youth with physical and mental disabilities face multiple barriers to effective transition to adulthood, especially in the area of employment (Sawin et al., 1999). Research studies indicate high levels of unemployment among youth who have recently transitioned from post-secondary education programs, and more information is needed about strategies and supports for youth to obtain and maintain employment, and advance in careers (Wehman & Revell, 1997).

Findings from two National Longitudinal Transition Studies (NLTS and NLTS-2) indicate that youth with disabilities lag far behind their peers without disabilities in employment outcomes after high school (Blackorby & Wagner, 1996; Cameto & Levine, 2005). Among youth with disabilities in both cohorts there were differences in employment outcomes based on type of disability, gender, ethnicity, and other factors. The majority of youth enrolled in the LSVRSP study participated in special education. Services and outcomes of special education and non-special education youth were compared, along with involvement of family members in the service program and factors associated with employment outcome and earnings levels at exit. Receipt of specific VR services was strongly correlated with achieving an employment outcome and entering competitive employment. The study did not report on post-closure employment retention (Hayward & Schmidt-Davis, 2000).

Luecking and Fabian found that integration of employment experiences with secondary and postsecondary education improved the success rate of transitional youth in finding and sustaining employment (Luecking, 2003; Luecking & Fabian, 2000). Thus, it is important to understand more about what kinds of services are received by VR consumers who are transitional youth, and how post-VR services and supports affect their employment outcomes.

People Who Received SSA Disability Benefits. The 1999 TWWIIA represents a policy change to address longstanding disincentives in the return to work efforts of beneficiaries collecting SSI and SSDI. Nevertheless, recent studies have found that work disincentives still exist and make it hard for Social Security recipients to find and sustain employment. Chan and colleagues found that VR consumers with orthopedic disabilities who received Social Security benefits had significantly lower rates of competitive employment than comparable peers who did not receive benefits (Chan et al., 2006). Drew and his colleagues found that receipt of disability compensation discouraged full participation in VR, resulting in poorer rehabilitation outcomes among more than 22,500 subjects with physical or psychiatric disabilities (Drew et al., 2001). In a qualitative study of Social Security recipients with psychiatric disabilities, O’Day and Killen (2002) found that current Social Security policies placed significant barriers in the employment paths of these consumers.

However, another study found that receipt of Social Security benefits was positively correlated with vocational success among people with schizophrenia (Russinova, Wewiorski, Lyass, Rogers, & Massaro (2002). Looking at the relationship between SSI receipt and employment characteristics of 1,640 young adults with disabilities, Berry (2000) found that those who collected SSI were less successful in obtaining employment and that the subjects who received SSI differed significantly in terms of work history and other factors that affect employment. Other factors significantly affecting employment outcomes included highest grade completed, family income, activity limitations, hospitalization, and health status. A recent study by Tremblay and colleagues found that Social Security recipients with psychiatric disabilities in Vermont who received benefits counseling achieved significantly greater improvements in earnings than matched recipients who did not receive the counseling (Tremblay, Smith, Xie, & Drake, 2006).

The LSVRSP final report on consumer characteristics indicates that persons who received VR services were less likely to achieve a competitive employment outcome if they were older, if their disability was significant or most significant, or if they were receiving SSI or SSDI (Hayward & Schmidt-Davis, 2003). The U.S. General Accountability Office (GAO) is currently completing a study of VR agency practice with respect to SSA beneficiaries. The study will not include information from individual consumers, but will focus on agency-level practices and programs. Consumer-level data have been identified as a gap in understanding the VR services and outcomes for SSA beneficiaries (GAO, 2007).

For these four subgroups, obtaining or retaining employment may depend on the availability and use of post-exit support and services, including post-employment services from VR. The PVRES will provide needed information about post-VR experience and longer-term employment outcome successes and difficulties.

A2. Purpose and Use of the Collected Information

This is a new collection. The primary audiences of the information are RSA and the state VR agencies that administer the service program in the states. The information will be used for program evaluation and ongoing program improvement, including the development of training programs and improvement of the long-term employment of individuals with disabilities served by the VR program. The information might also contribute to legislative oversight and potential improvement of the program for outcomes and benefits, not just knowledge and policy.

The findings are intended to contribute to Federal and state agency knowledge of successful employment strategies and the use and need for other services and resources after VR case closure. The findings will have policy implications for the VR Services Program and for the long-term employment of individuals with disabilities served by the program. The study will address the following specific research questions for the four subgroups defined in the previous section:

1. What is the labor market progression of former VR consumers in terms of employment status, earnings, stability, advancement, and receipt of job-related benefits?

2. To what extent do VR consumers, whose service records have been closed, continue to obtain post-closure and other services and resources from the community following participation in VR?

3. What are the noneconomic outcomes (e.g., community integration) of VR consumers whose service records have been closed?

4. In what ways do post-employment services and resources from the community relate to economic and noneconomic outcomes?

5. How and in what ways do former consumers’ receipt of SSA and other benefits change over time following participation in VR?

6. How are the demographic, disability, and other characteristics of individuals related to economic and other outcomes following participation in VR?

A separate analysis will examine the outcomes of sampled consumers who had supported employment as a goal in their individualized plan for employment.

Answers to these questions will inform RSA and state VR agencies about the post-closure experiences of the former VR consumers, including employment, earnings, receipt of benefits, and receipt of other services and resources. In addition, RSA plans to widely disseminate the study data and results to additional audiences to ensure knowledge transfer and utilization. The response to A16 (Plan for Tabulation and Publication) provides further detail about dissemination.

Appendix A provides a crosswalk between the study research questions and the specific survey questions to be used in the data collection. The instrument for conducting the interviews will collect information in several topic area groupings (Appendix B):

• Employment

• Consumer disability characteristics

• Workplace features (accommodations)

• Earnings and benefits

• Community resources, services, supports and attitudes

• Education and training

• Socio-demographic characteristics

The overall study will also include the use the several extant data sources to supplement the analysis. The RSA 911 Case Service Report (RSA 911) file will provide detailed information about the consumers at the time of application to VR and at the time of closure, as well as the VR services they received. This information will be useful in the analysis as well as in conducting nonresponse bias analysis. SSA administrative data files, some of which have been linked to the RSA 911, are expected to be made available for this study for validation of reported earnings and SSA disability benefits and for analysis along with the survey data. The Current Population Survey (CPS), the American Community Survey (ACS), and Local Area Unemployment Statistics (LAUS) will be used to provide information about the labor market environment for the period of time covered by the study.

A3. Use of Technology to Reduce Burden

The survey will be administered by Westat using computer-assisted telephone interviewing (CATI). CATI interviewing will reduce respondent burden by increasing interviewer efficiency and allowing for the targeting of questions based on prior responses or other information about the respondent. A web-based version of the instrument was considered, but rejected, because the expected number of respondents who would select this method was judged to be too low to justify the cost of developing and administering the web-based version.

For respondents with hearing impairments, state relay services will be accessed. In addition, it is expected that some respondents will use assistive technology devices that have been previously provided to them as part of the VR services they received. A paper version will be available for persons unable to respond by telephone.

A study web site (www.pvres.org) has been established to communicate study progress and disseminate findings. The web site will provide information for study participants, including privacy and confidentiality assurances, data collection clearances, and contact information for Westat and RSA. Details about the use of the web site for dissemination are described in the study’s Dissemination Plan, not attached, but available from the Department’s project officer.

The FY 2006 RSA 911 electronic data file will be used to construct the sampling frame for drawing the sample of former VR consumers. The file includes information on demographic characteristics, various types of status at application and closures, services received, and other information that will be used for analysis and imputation. Social Security Number (SSN) is the only identification of an individual in the file. SSNs will be sent to state VR agencies in a request for the names and contact information of the sampled former consumers. The agencies are expected to provide the information from centralized electronic administrative data files.

A4. Efforts to Identify Duplication

There is no alternative source of information on the post-VR experiences of these subgroups of consumers that would address the study questions. RSA’s existing annual performance data are based on reporting from the state VR agencies, including exit information on individuals whose services were closed in the prior year. While the RSA 911 provides extensive information about former VR consumers, the post-VR experiences are not known, and available data are not adequate to address the study questions.

In the LSVRSP, the sample acquisition and data collection activities occurred during 1995 through 1999. The study focused primarily on the services individuals received prior to service record closure, and the sample design included only a subset of the state VR agencies. A portion of the sample was tracked over a period of five years to examine long-term outcomes. Because of the timing of sample selection, the study does not reflect the effect of more recent disability employment legislation (WIA) and does not represent the service environment that is currently in place and affecting the study groups (e.g., Ticket to Work, One-Stop service centers).

The proposed study will interview former consumers, randomly sampled from all state VR agencies, about their experiences after record closure. No other data collection efforts are available for this population or this time period. The sample design will ensure that there is a representative sample of each of the four subgroups of former VR consumers.

A5. Methods to Minimize Burden on Small Entities

There is no anticipated impact on small businesses or other small entities.

A6. Consequences if Collection is not Conducted

If the data collection is not conducted, then RSA will not obtain needed information about the post-closure experiences of former VR consumers, including information about employment status, advancement in employment, earnings, receipt of public benefits, and services and resources received after closure to maintain or advance in employment. This will restrict RSA’s ability to improve program management and effectiveness, including post-closure services.

This longitudinal study is designed so that former consumers are interviewed at three points in time and that the interviews are conducted annually. This approach is intended to minimize the burden on the respondents without compromising the quality of the data that will be collected. Respondents are asked about their experiences over “the past 12 months,” and about their “current” situation. Recall error would increase substantially if the survey was conducted less than annually, contributing to response bias. If fewer than three rounds of data collection occur, then longitudinal analysis would be seriously compromised.

A7. Special Circumstances

There are no special circumstances that would cause this information collection to be conducted in any manner listed below. This collection of information complies with the requirements of 5 CFR 1320.5.

• Requiring respondents to report information to the agency more often than quarterly;

• Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Requiring respondents to submit more than an original and two copies of any document;

• Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

• In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• Requiring the use of statistical data classification that has not been reviewed and approved by OMB;

• That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Requiring respondents to submit proprietary trade secrets or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

A8. Federal Register Comments and Consultations Outside the Agency

a. Federal Register Notice and Comments 

The public was given an opportunity to review and comment on the proposed data collection (FR Notice Volume No. 72, No 52, dated March 19, 2007, Page 12785) with comments due May 18, 2007 (Appendix C). The following two comments were received:

“ education dept annual survey post vocational rehabilitation experience study…this certainly does not need to be taken every single year and represents substantial cost to take it every year. take it one time and if the results are satisfactory, it does not need to be redone for five years minimum. if the results are unsatisfactory, you had better get to the bottom of what you are doing wrong. the frequency schedule involves far too many tax dollars for no accomplishment…” [name and address excluded]

The commenter expressed that an annual survey is not needed and that it would represent a substantial cost if conducted annually. The commenter suggested doing the survey once, and that it would not need to be done for at least another five years.

Response: The proposed data collection is intended to support a longitudinal analysis of former VR consumers. The study’s focus is on determining the labor market progression and need and use of supportive services by former VR consumers over three consecutive years. It is not proposed as a continuing annual survey. The three data points of a longitudinal data collection will provide critical information to understand the extent of employment and unemployment over time and the change in consumers’ use of services to help the consumers find and/or maintain employment over time. To follow the suggestion of doing one survey, and repeating no sooner than five years later, would lose the benefits of a longitudinal study because of loss of data points describing the consumer experience. Conducting a longitudinal survey with a five-year interval would not be feasible due to the difficulty of locating respondents five years later (some form of annual tracing with the accompanying costs would be needed to maintain a useable response rate for the followup survey).

The second commenter made a suggestion regarding the content of one section of the survey instrument.

“My only comment on the Information Collection for the Post Vocational Rehabilitation Experiences Study has to do with Section E of the survey. One of our hypothesis with respect to this study is that individuals in the study group who continue to receive services and support in the community will have better long term outcomes. Section E of the survey is intended to obtain information about those supports. In particular, question E.2.B. is designed to elicit information about the individuals participation in such programs. However, with the exception of "day treatment program," which often serve individuals with mental illness, I do not think that the programs listed (each of which the subject must provide a Yes or No answer), reflects the type of support programs that these consumers are likely to participate in. I strongly believe that this question needs to be revised to include other community programs that typically serve individuals with mental retardation and mental illness.”

The commenter expressed the view that the response categories to the questions did not adequately include the types of programs that typically serve individuals with mental retardation and mental illness. These proposed changes and RSA’s response are discussed below.

Response: The instrument has been revised to provide response categories that more fully address the types of programs that typically serve individuals with mental retardation and mental illness. This involved focusing more on the types of programs and services available to these populations and less emphasis on the type of organization providing the program or service.

b. Consultations Outside the Agency

RSA contracted with RTI International to develop a Study Design Report for this longitudinal study of the VR Services Program. The Study Design Report (March, 2005) specified research questions, provided a conceptual framework for organizing data collection and analysis, and described recommended approaches to selection of a nationally representative sample of former VR consumers. To develop the study design, RTI reviewed the relevant literature and conducted interviews with Federal and state officials, other researchers, and rehabilitation professionals.

RSA contracted with Westat (and its subcontractor InfoUse) to conduct the PVRES. Westat developed a detailed Study Plan that was submitted for review to the study’s POE. The POE represented subject matter and methods experts of the Department of Education and other Federal agencies, and state VR agencies. The POE was convened in a meeting in Washington, D.C. in February of 2006 to review the Study Plan, including the sampling plan and draft survey questions. Subsequently, the Study Plan was revised based on feedback from the POE and also consideration of budget and respondent burden. The revised Study Plan was distributed to members of the POE and a second meeting was convened in March 2007 to discuss the revised plan.

Exhibit A-1 shows the membership of the POE, which includes both non-Federal and Federal members. Federal members include representatives from the Departments of Education, Labor, Social Security, and HHS.

Exhibit A-1. Panel of Experts

Non-Federal Members
Elmer Bartels	Council of State Administrators of Vocational Rehabilitation (CSAVR) Representative Commissioner, Massachusetts Rehabilitation Commission
Renee Cameto	Senior Social Science Researcher SRI International, Center for Education
Patrick Cannon*	National Council of State Agencies for the Blind (NCASB) Representative Director, Michigan Commission for the Blind
Judith Cook	Director, Mental Health Service Research Program Professor of Sociology in Psychiatry
Sharon Davis**	Director, Professional and Family Services The Arc of the United States
Gerry Hendershot	Consultant on Disability and Health Statistics
John Kregel	Professor of Special Education Chairman, Department of Special Education and Disability Policy Virginia Commonwealth University School of Education
Richard Luecking	President, Transcen
Catriona Macdonald	Policy Advisor National Council of State Agencies for the Blind (NCSAB)
Bill McCarriston	Chair, State Rehabilitation Council (Massachusetts)
Fred Schroeder**	National Council of State Agencies for the Blind (NCSAB)
Andrew Sperling	National Alliance on Mental Illness Director of Federal Legislative Advocacy
Sue Swenson*	Executive Director, The Arc of the United States
Glen White	Professor, Applied Behavioral Science Director, RTC on Independent Living University of Kansas
Federal Members
Barbara Altman***	National Center for Health Statistics (NCHS)
Roseann Ashby	Office of Special Education and Rehabilitative Services U.S. Department of Education
Hugh Berry	Office of Special Education and Rehabilitative Services U.S. Department of Education

Exhibit A-1. Panel of Experts (continued)
Federal Members (cont’d)
Laurie Collins	Office of the Under Secretary U.S. Department of Education
Richard Ensor**	Division of One-Stop Operations U.S. Department of Labor
Jerry Elliott	Office of the Assistant Secretary U.S. Department of Education
Melodie Johnson	Office of Special Education and Rehabilitative Services U.S. Department of Education
David Keer	National Institute on Disability and Rehabilitation Research U. S. Department of Education
Cille Kennedy	Research Policy Analyst Office of the Assistant Secretary for Planning and Evaluation U.S. Department of Health and Human Services
Greg March	Office of Special Education and Rehabilitative Services U.S. Department of Education
Steven Zwillinger	Contracting Officer’s Representative Rehabilitaton Services Administration U.S. Department of Education
Jim Maxwell	Office of the Under Secretary U.S. Department of Education
Paul O’Leary	Project Officer Office of Disability and Income Support Programs U.S. Social Security Administration
Chonita Tillmon*	Disability Policy Specialist Employment and Training Administration U.S. Department of Labor

The survey questions were pre-tested with former VR consumers in five states, including cognitive interviews of consumers in four states. The cognitive interviews included discussion and feedback about questions in specific sections of the survey. The interview was administered in its entirety to four persons with disabilities in one state. Sections of the interview were administered and discussed with participants in the remaining states. This matrix design limited the burden on each participant and resulted in no question being asked of more than nine persons.

A9. Decision to Provide Any Payment or Gift to Respondents, Other Than Remuneration of Contractors or Grantees

We propose giving a payment of $10 to each respondent who completes a baseline interview and, for each followup, an unconditional $10 in advance to all respondents remaining in the sample after the baseline interview. The payments are intended to encourage participation. For the baseline data collection, the incentive will be described in the advance materials and at the start of the interview as a thank you for respondents’ time and participation in the study. It was decided that the contact information available for the baseline advance mailing, which could be over two years old for a large portion of the sample, is not a sufficiently reliable means to provide unconditional incentives. An unconditional advance incentive will be included in the advance mailings for the followups since more current and reliable contact information will be available for those rounds of data collection.

Those who assist respondents or act as proxies will also receive $10 for assisting with an interview. Interpreters who receive hourly pay whom respondents arrange to have present during the interview will be reimbursed at their regular rate.

The use of incentives is warranted given that the sample will include hard-to-find individuals (because of the quality of the contact information and the tendency of some with mental illness to make frequent changes in housing arrangements) and to ensure that participation is high for each of the four targeted subgroups. Incentives are also warranted because of the unique burden participation will impose on segments of the sample with certain disabilities (e.g., those who tire easily, those who must arrange to have accommodations ready, etc.). Because this is a longitudinal study, the payment is expected to motivate the baseline respondents to continue their participation throughout the study. Payments to participants were recommended by several members of the POE for improving response rates.

A10. Assurance of Confidentiality

Advance materials and the introduction to the interview will ensure respondents that confidentiality will be maintained, except as required by law. The draft advance letter, draft informed consent form, and the Frequently Asked Questions enclosure containing these assurances are provided in Appendix D.

Activities to ensure confidentiality of individuals and their attributes will be conducted in accordance with the Privacy Act of 1974, which safeguards individuals against invasion of personal privacy by (1) permitting them to determine what personal records are collected, maintained, used, or disseminated; (2) preventing personal records from being used for purposes other than those to which they agreed; and (3) giving individuals access to their records and allowing them to correct or amend those records. Project staff will adhere to the regulations and laws regarding the confidentiality of individually identifiable information. All contractor staff members and subcontractors working on the study with access to the data are required to sign a confidentiality pledge. A copy of Westat’s Assurance of Confidentiality of Survey Data is provided in Appendix E.

Specific steps to guarantee confidentiality include the following:

• Identifying information about the respondents (e.g., SSN, respondent name, address, and telephone number) will not be entered into the data collection or analysis data file, but will be kept separate from other data and password protected. A unique identification number will be used for building raw data and analysis files.

• A fax machine used to send or receive documents containing confidential information will be kept in a locked field room, accessible only to study team members. When sending faxes, study staff will call ahead to make sure the authorized recipient is waiting for the fax.

• In emails, respondents will be referred to by first and last initial and unique identification number. Files containing more information will be password protected.

• Confidential materials will be printed on a printer located in a limited access field room. If printing documents that contain confidential information from shared network printers, authorized study staff will be present and retrieve the documents as soon as printing is complete.

• Statistical disclosure control procedures will be followed to reduce the risk of disclosing the identity of survey respondents through the release of survey data. This would involve reducing or modifying the data before release of public-use data sets. (See A16 for detailed discussion.)

• In public reports, findings will be presented in aggregate for the subgroups of interest (i.e., mental illness, mental retardation, transitional youth, and Social Security beneficiaries) or by type of respondent (e.g., employed/not employed at closure). (See A16 for detailed discussion.) No reports will identify individual respondents or the VR programs that served them.

• Access to the sample files will be limited to authorized study staff only; no others will be authorized such access.

• All members of the study team will be briefed regarding confidentiality of the data. Each person involved in the study will sign and have notarized an affidavit of nondisclosure attesting to his/her understanding of the significance of the confidentiality requirement (Appendix E).

• A control system will be in place, beginning at sample selection, to monitor the status and whereabouts of all hard-copy data collection materials during transfer, processing, coding, and data entry. These include sign-in/sign-out sheets and the hand-carrying of documents by authorized project staff only.

• All data will be stored in secure areas accessible only to authorized staff members. Computer-generated output containing identifiable information will be maintained under the same conditions.

• When any hard copies that contain confidential information are no longer needed, they will be shredded.

As mentioned in the Introduction, RSA has contracted with Westat to conduct the proposed data collection. As part of its contract with Westat, RSA (and the U.S. Department of Education) established security requirements for the handling of all forms of data associated with the study, particularly regarding data collection and handling, web-based display, archiving, and dissemination. The contractor is complying with the 2002 Federal Information Security Management Act (FISMA) and is using NIST 800-37 and 800-53 guidance documents for certification and accreditation of its data systems for the study.

In accordance with the Privacy Act of 1974, the Department of Education published a notice of a new system of records in the Federal Register for PVRES. The notice discusses the routine use of records maintained in the system of records, noting that any disclosure of individually identifiable information from a record in the system must comply with the requirements of Federal law and regulations. (Included in Appendix C, Federal Register Notices.)

SSNs are sensitive data and will be treated with the utmost security to ensure their safety. All personally identifiable information will be stored separately from survey data. Westat’s project network directory is limited to project staff, and subdirectories containing data are limited to those staff members that need access to the data to perform their job duties.

In addition to published reports, the PVRES design includes release of public-use data sets for each wave of the study and linked data sets that will support longitudinal analysis. Statistical disclosure analyses will be performed and appropriate controls will be used prior to release of reports and public-se data sets. All individual identifiers will be removed from data sets.

Chapter 34, Section 362.38 of the Code of Federal Regulations describes the general provisions for the protection, use and release of personal information. According to Chapter 34, Section 362.38 (d) of the Code of Federal Regulations, personal information may be released to an organization, agency, or individual engaged in audit, evaluation, or research only for purposes directly connected with the administration of the vocational rehabilitation program or for purposes that would significantly improve the quality of life for applicants and eligible individuals and only if the organization, agency, or individual assures that—

1. The information will be used only for the purposes for which it is being provided;

2. The information will be released only to persons officially connected with the audit, evaluation, or research;

3. The information will not be released to the involved individual;

4. The information will be managed in a manner to safeguard confidentiality; and

5. The final product will not reveal any personal identifying information without the informed written consent of the involved individual or the individual’s representative.

A11. Justification for Any Questions of a Sensitive Nature

There are no questions about sexual behavior or attitude or religious beliefs. There are a few questions that are of a sensitive nature: the survey asks respondents about changes in their medical condition, impairment, or disability in the past 12 months, as well as their general health; and asks if the respondent participated in various social and medical programs including 12-step programs. These are asked because they are relevant to the role of the state VR program and post-employment services for persons with disabilities.

All sampled former consumers will receive a packet of materials about the study, including information about the voluntary nature of their participation and the confidentiality of their responses. They will be told that they can refuse to answer a question if they do not want to answer that question. No interview will be undertaken with sampled former consumers before he/she provides his/her consent to participate (or his/her legal guardian provides consent).

A12. Estimates of the Hour Burden

Information Collection (IC) #1—VR Agencies. For the baseline survey only, state VR agencies will be asked to provide contact information for the former consumers selected into the sample from the RSA911 file. Appendix F contains the letter to be sent by RSA authorizing release of the information to the PVRES contractor, the request to be sent by the PVRES contractor, and the list of contact information to be requested. The 75 agencies located in the 50 states and District of Columbia were informed of the study in late 2005 and the plans to ask them to extract such information from centralized individualized files.

• State agencies will be notified of the need for information by RSA and asked to extract data from their electronic files. The specific information requirements and procedures for submitting it will be conveyed by the PVRES contractor. An average of 20 person hours is estimated to be required by each of the 75 agencies, for a total of 1,500 hours.

• Agencies in four states have indicated that consumers must be notified in advance before contact information can be provided to the contractor. The final number of states may be higher. Westat will assist agencies in these states in preparing packages to send to former consumers to describe the study and request their consent. State personnel will need to produce and affix mailing labels, mail the packages, and keep track of responses. This burden will vary by the size of the state sample. We assume eight agencies will each prepare mailings for 138 former consumers on average, and 10 minutes is required per consent, or 184 hours in total.

• Local VR agencies may be contacted to help locate former consumers by providing information not maintained centrally by their state and only available in counselor’s files. We estimate this will be done for 10 percent of the sample and will require 20 minutes of a counselor’s time to retrieve the file and provide requested information, or 267 hours in total.

Estimates of respondent burden for the state VR agencies are given in Table 1. An estimated total of 1,951 hours is shown for the 75 VR agencies.

Table 1. Estimates of VR Agency Respondent Burden – Baseline Only

Activity	Number of respondents	Total hours	Hourly cost to respondent	Cost to agency respondent
Prepare electronic file	75	1,500	$25	$37,500
Mail & monitor request for consent	8	184	$15	$2,760
Research local records	75	267	$25	$6,675
Total burden	75	1,951		$46,935

Information Collection #2—Former VR Consumers. Estimates of the burden for former VR consumers to respond to the survey instrument (Appendix B) and the informed consent form (in Appendix D) are provided in Table 2. The total burden for three rounds of data collection is estimated to be 9,146 hours. The timing estimates are derived from pretest experience and are adjusted based on several factors:

• the number of target respondents for each round of data collection;

• the estimated time of 5 minutes for respondents to review and complete the informed consent form;

• the estimated time of 36 minutes to administer the questions for respondents following the path for having been employed during the prior 12 months or the shorter path of 26 minutes if not employed; and

• an adjustment to the hours for the 37 percent of respondents expected to require extended administration time due to the nature of their disability.

Based on the LSVRSP, we estimated 69 percent of respondents will follow the interview for employed persons during the baseline round; 73 percent will do so for the first followup; and 76 percent will do so for the second followup. Based on pretest experience we adjusted the hours upward by 30 percent to account for the 37 percent of the sample with a primary disability of mental retardation affecting administration time.

Table 2. Estimates of Survey Respondent Burden

Instrument/path	Completions	Mins	Hours	Adjusted hours	Hourly cost to respondents	Cost to respondents
Baseline-employed	3,367	138,047	2,301	2,991	$7.33	$21,922
Baseline-not employed	1,513	46,903	782	1,017	$0.00	$0
Baseline total	4,880	184950	3,083	4,007		$21,922
First followup-employed	3,028	124,148	2,069	2,690	$8.48	$22,812
First followup-not employed	1,120	34,720	578	752	$0.00	$0
First followup total	4,148	158,868	2,648	3,442		$22,812
Second followup-employed	2,979	122,139	2,035	2,646	$9.62	$25,453
Second followup-not employed	941	29,171	486	632	$0.00	$0
Second followup total	3,920	131,710	2,522	3,278		$25,453
Total burden				10,728		$70,187

The wage categories used for cost to survey respondents are based on LSVRSP findings using the reported average hourly wage at closure for the PVRES baseline and the wage three years after closure for the PVRES second followup. The first followup is an average of these two. No hourly cost has been assigned to unemployed respondents. The total cost to respondents, $70,187, is nearly one-half of the expected totals of incentive payments ($129,320) to respondents.

When IC #1 and IC #2 are combined for the baseline only (first year of data collection), the hourly burden is 1951 + 4,007 = 5,958 hours and the number of respondents 75 + 4,880 = 4,955.

A13. Estimate for the Total Annual Cost Burden to Respondents

PVRES does not have a record-keeping component and will not incur cost burdens other than as described above (A12) for respondents.

A14. Estimates of Annualized Costs to the Federal Government

The estimated total cost to the Federal Government associated with the PVRES data collection is $5,905,905. This cost represents the value of contractor services over a period of 60 months to perform all activities associated with the proposed data collection. The average cost per year over five years is $1,181,181.

There are three phases to the study. Phase 1 is for 30 months, Phase 2 is for 12 months, and Phase 3 is for 18 months. Allocating the costs from each Phase across fiscal years (October through September), gives the annualized costs reported in Table 3.

Table 3. Estimated Annualized Costs to the Federal Government

Time Period	Estimated Annualized Costs
October 2005 – September 2006	$1,079,383
October 2006 – September 2007	$1,079,382
October 2007 – September 2008	$1,284,697
October 2008 – September 2009	$1,317,485
October 2009 – September 2010	$1,144,957

Included are costs for all expenses to be incurred because of the proposed data collection. This includes development of the study plan, data collection instrument development and testing, establishing and meeting with a panel of experts, establishing and maintaining data security, establishing and maintaining the study’s web site, all aspects of data collection (including incentive payments); data cleaning, coding and processing; descriptive, explanatory, and longitudinal analyses; preparation of various project reports and data files; dissemination of reports and data; and general project management and coordination with the government project officer. These costs are derived from the contractor’s budget for all three phases and include hourly costs for all contractor and subcontractor staff; other direct costs such as printing, local travel, and interviewer telephone usage; and contractor’s indirect costs. Table 4 provides estimated costs by expense category and study phase.

Table 4. Estimated Costs by Expense Category and Study Phase

Study Phase	Expense Category
	Labor	Subcontract	Other Direct and Respondent Incentives	Overhead, G&A, and Fees	Total Costs
1	$857,405	$388,568	$228,455	$1,224,026	$2,698,454
2	$403,777	$337,662	$143,362	$605,213	$1,490,014
3	$481,239	$330,317	$153,614	$752,266	$1,717,436
All	$1,742,421	$1,056,547	$525,431	$2,581,505	$5,905,904

A15. Reasons for Any Program Changes or Adjustments

This is a new collection. Therefore, the entire burden is new.

A16. Plans for Tabulation and Publication, Analytic Techniques, and Time Schedule

Several reports will be produced based on the post VR experiences survey of former VR consumers. A Baseline Data Collection Report will discuss issues or problems related to sampling, establishing contact with former consumers, and collecting baseline data. It will also provide (a) a profile of the sampled population in terms of their characteristics, employment status and earnings; (b) analysis of former VR consumers with prior closures; and (c) a chapter describing the validity of the baseline survey data collected on earnings and SSI/SSDI benefits.

An Interim Report and a Final Report will track the changes in post-VR experiences from year to year. There will be separate report sections for (a) the four main subgroups, and among them, individuals who did not report employment at case record closure but later found employment; and (b) individuals who had a previous case closure (in the 36 months prior to their 2006 closure).

The sections of the reports focused on the four subgroups will be formatted based on the research objectives of the study. Particular emphasis will be given to former consumers’ long-term employment status, earnings, and receipt of Federal benefits. In addition, the reports will discuss services subsequently provided to former consumers to assist them in maintaining employment and advancing in their careers. ACS, LAUS, and other data sources will be used to control for general unemployment and other contextual factors.

Analytical Techniques

The study team will analyze data from the PVRES survey and extant data sources using various approaches, including basic descriptive analysis and more complex explanatory analyses. The team will perform cross-sectional analysis in each wave but emphasis will be given to the longitudinal analysis to address the research questions. All analyses will be performed using appropriate software packages that can handle complex survey data. Detailed types of analysis to be performed are as follows:

Descriptive Analysis. This will include estimates of central tendency such as mean and median, estimates of proportions for categorical variables in the form of cross-tabulation by subgroups of special interests, demographic variables, geographic variables, and other independent variables such as disability category, estimates of totals, estimates of correlation coefficients, contingency table analysis, and bivariate analysis. This type of analysis will be used to suggest what to examine and how to conduct higher levels of data analysis.

Transitional Probability Analysis. Transitional probability is the probabilistic measure of an individual’s transition from one state to another. This approach will be used to analyze the change of employment status, benefit recipient status, disability status, etc. from one data collection period to the next. This approach will be useful to study the transition at the aggregate level. This analysis will indicate the dynamics of study subjects’ movement from one state to another between waves of data collection.

Event History Analysis or Survival Analysis. Survival analysis is a special case of a broader class of event history analysis. The transition probability of an outcome variable (e.g., employment) is analyzed over time, and one can examine association of the probability with explanatory variables. The estimated model also enables one to predict the probability of an individual’s status (e.g., employment) at a given time (e.g., one year after exiting VR service) given specific values of explanatory variables used in the analysis.

Analysis Using Multiple Regression and General Linear Model (GLM). Regression analysis is used for both continuous and categorical dependent variables. In the case of categorical variables, logistic regression is frequently employed. Poisson regression is useful when the dependent variable is the count of a rare event during a given period of time. It will be useful to study VR consumers with employment at case closure in terms of the number of times they become unemployed during the study period. GLM is a broad class of linear models that includes regression models mentioned above. It assumes that a function of the mean of a dependent variable is linear in explanatory variables. Under this framework, one can perform various traditional analyses such as analysis of variance (ANOVA).

Hierarchical Linear Modeling (HLM). Longitudinal data contain repeated measurements for the same individuals. The repeated measurements for a given individual are nested within the individual. This longitudinal data structure can be modeled meaningfully and naturally by HLM (or multi-level models). For example, analyzing the growth of earnings of employed consumers over time would indicate the relation between earnings growth and covariates on the consumers used in the analysis.

Interpretation of the analysis results should be done with caution when establishing causal or treatment effects on outcome variables because the study is observational; that is, assignment of individuals to treatment is not random. However, there are ways of establishing causal effects by employing techniques such as instrumental variables and propensity scores. These possibilities will be explored. If successful, they will add much value to the analysis.

Table 5 shows types of data analyses and their usefulness to answer the study research questions stated in the response to question A2.

Table 5. Types of Data Analyses and Usefulness

Analysis type	Use of extant data	Research questions
		1	2	3	4	5	6
Descriptive	RSA, SSA,LAUS						
Transitional Probability	RSA					
Event History	RSA, SSA, ACS, LAUS					
Regression/GLM	RSA, SSA, ACS, LAUS						
HLM	RSA, SSA, ACS, LAUS						

Producing Public Use Datasets and Supporting Documentation

Study data will be prepared for use as public-use datasets. In addition to providing public-use datasets, descriptive statistics will be developed from the data in the study’s written reports and products. Other researchers wishing to use the PVRES data in their own work will have access to public-use data files and supporting materials. These public-use data files will be constructed to permit maximum access to the survey data while protecting the confidentiality and privacy of respondents. The following data products will be made available on the project website: questionnaire; public-use datasets; user’s manual; codebook; means and frequency distributions for the baseline and first followup surveys only. Because there are no plans to support the study web site beyond September 2010, the contractor will forward to the Department of Education all files so that the Department, at its option, can make them available on its website.

Before making statistical reports, means and frequency distributions, and data sets of the survey data available to the public, OSERS will submit them to the NCES disclosure review board (DRB) for review and approval.

Time Schedule for the Project

Exhibit A-2 provides a summary of the anticipated schedule for the PVRES.

Exhibit A-2. Anticipated PVRES Schedule

Task	Baseline Wave 1 (2007)	First Followup Wave 2 (2008)	Second Followup Wave 3 (2009)
Submit full study to IRB	5/22/07
IRB approval of study	6/19/07
Receive OMB clearance	7/31/07
Request contact information	8/10/07
Mail recruitment packets	10/5/07
Begin calling sampled consumers	10/15/07
Conduct data collection interviews	10/07 – 3/08	8/08 – 12/08	8/09 – 12/09
Prepare Sample Acquisition Report	12/21/07
Conduct analysis of nonresponse bias	4/08 - 5/08	1/09 - 2/09	1/10 - 2/10
Prepare weights, imputations, models	4/08 – 6/08	1/09 – 3/09	1/10 – 3/10
Prepare data analysis tables	6/08 – 8/08	3/09 – 5/09	3/10 – 5/10
Conduct validation study	6/08 – 8/08
Completion of baseline report	9/20/08
Completion of first interim report		6/19/09
Completion of final report			8/20/10
Post data and report to study website	12/08	9/09

A17. Approval to Not Display the Expiration Date

RSA will display the OMB control number and expiration date for the survey under this clearance.

A18. Explanation of Exceptions

There are no exceptions to the certification statement.

References

Akabas, S. (1994). Workplace responsiveness: Key employer characteristics in support of job maintenance for people with mental illness. Psychosocial Rehabilitation Journal, 17(3), 91-101.

Blackorby, J., & Wagner, M. (1996). Longitudinal postschool outcomes of youth with disabilities: Findings from the National Longitudinal Transition Study. Exceptional Children, 62(5), 399-413.

Butterworth, J., Hagner, D., Helm, D., & Whelley, T. (2000). Workplace culture, social interactions, and supports for transition-age young adults. Mental Retardation, 38, 342-353.

Cameto, R., & Levine, P. (2005). Changes in the employment status and job characteristics of out-of-school youth with disabilities (Chapter 5). In: NLTS-2, Changes over time in the early post-school outcomes of youth with disabilities. Retrieved January 11, 2007, from http://www.nlts2.org/reports/2005_06/index.html

Center on an Aging Society. (2004). Workers affected by chronic conditions: How can workplace policies and programs help? Issue Brief Number 7, Georgetown University. Retrieved March 30, 2006 from http://ihcrp.georgetown.edu/agingsociety/pdfs/
workplace.pdf

Chan, F., Cheing, G., Chan, J.Y.C., Rosenthal, D.A., & Chronister, J. (2006). Predicting employment outcomes of rehabilitation clients with orthopedic disabilities: A CHAID analysis. Disability and Rehabilitation: An International Multidisciplinary Journal, 28(5), 257-270.

Cook, J.A. (2002). Employment, a workable option despite mental illness. SAMSA News, 10, 1. Retrieved January 4, 2007, from http://www.samhsa.gov/SAMHSA_News/VolumeX_1/article3.htm

Drew, D., Drebing, C.E., Van Ormer, A., Losardo, M., Krebs, C., Penk, W., & Rosenheck, R.A. (2001). Effects of disability compensation on participation in and outcomes of vocational rehabilitation. Psychiatric Services, 52(11), 1479-1484.

Hayward, B.J., & Schmidt-Davis, H. (2000) A longitudinal study of the Vocational Rehabilitation Service Program, fourth interim report: Characteristics and outcomes of transitional youth in VR. Research Triangle Park, NC: Research Triangle Institute.

Hayward, B.J., & Schmidt-Davis, H. (2003), A longitudinal study of the Vocational Rehabilitation Service Program, final report 1: How consumer characteristics affect access to, receipt of, and outcomes of VR services. Research Triangle Park, NC: Research Triangle Institute.

Jans, L., Stoddard, S., & Kraus, L. (2004). Chartbook on mental health and disability in the United States. Washington, D.C.: U.S. Department of Education, National Institute on Disability and Rehabilitation Research.

Leff, H.S., Cook, J.A., Gold, P.B., Toprac, M., Blyler, C., Goldberg, R.W, McFarlane, W., et al. (2005). Effects of job development and job support on competitive employment of persons with severe mental illness. Psychiatric Services, 56(10), 1237-1244.

Luecking, R. (2003). Employer perspectives on employing and accommodating youth in transition. Journal of Special Education Technology, 18, 4.

Luecking, R., & Fabian, E. (2000). Paid internships and employment success for youth in transition. Career Development for Exceptional Individuals, 23, 205-222.

Luftig, R.L., & Muthert, D. (2005). Patterns of employment and independent living of adult graduates with learning disabilities and mental retardation of an inclusionary high school vocational program. Research in Developmental Disabilities, 26(4), 317-325.

McCarriston, W.T., Turner, L., & Hursh, N. (2004). Housing, employment, health care, and legal policies and regulations: Impact on homeless individuals’ ability to sustain employment. Retrieved February 13, 2006, from www.cwsbos.com/news/research/
research_brief_three.pdf

MacDonald-Wilson, K.L., Rogers, E.S., Massaro, J. M., Lyass, A., & Crean, T. (2002). An investigation of reasonable workplace accommodations for people with psychiatric disabilities: Quantitative findings from a multi-site study. Community Mental Health Journal, 38, 35-50.

Moore, C.L., Feist-Price, S., & Alston, R.J. (2002). Competitive employment and mental retardation: Interplay among gender, race, secondary psychiatric disability, and rehabilitation services. Journal of Rehabilitation, 68(1), 14-19.

Moore, C.L., Harley, D.A., & Gamble, D. (2004). Ex-post-facto analysis of competitive employment outcomes for individuals with mental retardation: National perspective. Mental Retardation, 42(4), 253-262.

O’Day, B., & Killen, M. (2002). Does U.S. Federal policy support employment and recovery for people with psychiatric disabilities? Behavioral Sciences & the Law, 20(6), 559-583.

Olson, D., Cioffi, A., Yovanoff, P., & Mank, D. (2001). Employers’ perceptions of employees with mental retardation. Journal of Vocational Rehabilitation, 16(2), 125-133.

Razzano, L.A.,Cook, J.A., Burke-Miller, J.K., Mueser, K.T., Pickett-Schenk, S.A., Grey, D.D., Goldberg, R.W., et al. (2005). Clinical factors associated with employment among people with severe mental illness: Findings from the employment intervention demonstration program. Journal of Nervous and Mental Disease, 193(11), 705-713.

Russinova, Z., Wewiorski, N.J., Lyass, A., Rogers, E.S., & Massaro, J.M. (2002). Correlates of vocational recovery for persons with schizophrenia. International Review of Psychiatry, 14(4), 303-311.

Sawin, K.J., Cox, A.W., Metzger, S.G., Horsley, J.W., Harrigan, M.P., Deaton, A., & Thompson, E.C. (1999). Transition planning for youth with chronic conditions: An interdisciplinary process. National Academies of Practice Forum: Issues in Interdisciplinary Care, 1(3), 183-196.

Shankar, J., & Collyer, F. (2003). Vocational rehabilitation of people with mental illness: The need for a broader approach. Australian e-Journal for the Advancement of Mental Health, 2, 2.

Tremblay, T., Smith, J.; Xie, H., & Drake, R.E. (2006). Effect of benefits counseling services on employment outcomes for people with psychiatric disabilities. Psychiatric Services, 57(6), 816-821.

U.S. General Accountability Office. (2007). Report to the CSAVR Research Committee on the current VR agency study (verbal report).

Wehmeyer, M.L., & Palmer, S. (2003). Adult outcomes for students with cognitive disabilities three years after high school: The impact of self-determination. Education and Training in Developmental Disabilities, 38(2), 131-144.

Wehman, P., & Revell, W.G. (1997). Transition into supported employment for young adults with severe disabilities: Current practices and future directions. Journal of Vocational Rehabilitation, 8(1), 65-74.

Yamaki, K., & Fujiura, G.T. (2002). Employment and income status of adults with developmental disabilities living in the community. Mental Retardation, 40(2), 132-141.

* March 2007 meeting only

*^* February 2006 meeting only

*^** Transferred to non-Federal status after February 2006 meeting