1820-New (04796) Part B Supporting Statement Evaluation of State RA SEP

U.S. Department of Education

Rehabilitation Services Administration (RSA)

EVALUATION OF STATE VOCATIONAL REHABILITATION AGENCY ADMINISTRATION OF SUPPORTED EMPLOYMENT SERVICES PROGRAMS

Office of Management and Budget

Clearance Package Supporting Statement for Paperwork Reduction Act Submissions

Part B

Collections of Information Using Statistical Methods

Table of Contents (Part B)

B. COLLECTIONS OF INFORMATION USING STATISTICAL METHODS 25

B1. Respondent Universe and Sampling Methods. 25

B2. Describe the procedures for the collection of information including: 26

B3. Methods to Maximize Response Rates 27

B4. Tests of Procedures or Methods . 28

B5. Consultations on Statistical Aspects of the Design. 28

APPENDICES 29
APPENDIX A Questionnaire 30

APPENDIX B Draft Advance E-mail to VR Agency Director 45

APPENDIX C Draft Advance E-mail from the Contractor 48

APPENDIX D Draft Username E-mail to the VR Agency Contacts 50

APPENDIX E Draft Password E-mail to the VR Agency Contacts 52

APPENDIX F endorsements 54

U.S. Department of Education

Rehabilitation Services Administration

EVALUATION OF STATE VOCATIONAL REHABILITATION AGENCY ADMINISTRATION OF SUPPORTED EMPLOYMENT PROGRAMS

B. COLLECTIONS OF INFORMATION USING STATISTICAL METHODS

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. The following documentation should be provided with the Supporting Statement Part A to the extent that it applies to the methods proposed. For further information, please consult OMB’s Standards and Guidelines for Statistical Surveys.

B1. Describe the potential respondent universe (including a numerical estimate) and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, state and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The response universe consists of 80 State VR combined, general and blind agencies that receive Federal grant funds to serve individuals with disabilities to prepare for and engage in gainful employment. This includes agencies in the District of Columbia and Territories (Puerto Rico, the U.S. Virgin Islands, Guam, Northern Marianas, and American Samoa). The survey of all 80 State VR agencies will be conducted as a web-based survey. Because this is a census, no statistical sampling methods will be applied. The survey period will last six weeks.

RSA will send out the first electronic notice to all State VR agencies to inform them of the forthcoming data collection immediately after receiving OMB clearance. To facilitate the smooth operation of the survey and cooperation of the agencies, we have obtained the endorsement of the Council of State Administrators of Vocational Rehabilitation (CSAVR) and the National Council of State Agencies for the Blind (NCSAB) (see Appendix F).

The contractor anticipates that at a minimum 90 percent of VR agencies will respond to this survey. If the response rate is below 80 percent by the end of four weeks, the contractor will contact the non-responding VR agencies to encourage them to participate. This will include follow-up phone calls and e-mails. During the survey process, the contractor will examine the responses for completeness and consistency and will communicate with respondents as needed to obtain complete data and address inconsistencies in the responses.

B2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This data collection will not use any statistical methodology for stratification and sample selection because the whole universe of VR agencies will be surveyed.

After the survey data are collected, the contractor will identify three VR agencies from among the respondents to explore more deeply the activities related to SE policies, practices, and funding. Selection of the three sites will be based on survey results that indicate strong or promising SE practices. The contractor will contact these three sites to conduct in-depth discussions with State VR agency staff. During the discussions, the contractor may request access to relevant documents and data that might provide further information about program administration.

There are no unusual problems requiring specialized sampling procedures.

Questionnaire Design

The contractor developed a questionnaire in collaboration with RSA staff that had direct experience working in state VR agencies on supported employment. The RSA staff participated in early pretests and cognitive interviewing to assist the contractor in the development process. The contractor also shared the document with CSAVR and revised the instrument based on CSAVR feedback.

Data Collection Methods and Procedures

Below are the steps that will be followed for the data collection:

• The contractor will develop the web application of the survey instrument.

• RSA will send an electronic notice to all VR agencies to inform them of the forthcoming data collection.

• The contractor will follow up with an introductory e-mail to the VR contact person. This e-mail will include a description of the purpose of the evaluation and how information will be used and request the agency to identify a staff member who will be responsible for coordinating between the agency and the contractor.

• The contractor will send a follow-up e-mail to the VR agency directors with information on how to enter the survey webpage and establish a username. A second e-mail will provide the password to access the survey. Each agency will receive a unique user name and password.

• VR agencies will access the secure web survey and complete the survey.

• The contractor will monitor the website and survey responses.

• The contractor will download the survey responses into a database to analyze the data.

• The contractor will contact non-responding VR agencies (after four weeks of survey administration) to encourage them to respond to the survey or clarify any questions they may have.

Followup of non respondents

Contractor staff members will follow up by telephone and/or e-mail with those VR agencies that do not respond to the web survey within four weeks after they were sent the web link and instructions on how to access the survey. These staff members are experienced in nonresponse avoidance and conversion of non respondents.

There is no use of periodic data collection cycles to reduce burden.

B3. Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

The following describes procedures to be followed to maximize the number of respondents who complete the survey:

The contractor has obtained endorsements from the CSAVR and NCSAB for the purpose of securing the cooperation of all VR agencies. The contractor will contact VR agency directors using the advance notification e-mails and ask the directors to select individuals who are knowledgeable of the SE program and/or SE expenditures to complete the VR agency survey. The instrument allows for multiple respondents from the same agency to complete a particular section of the survey. An advance e-mail will be sent to the identified VR staff member to explain the purpose of the study and how to access the survey. The contractor will follow up with identified staff to clarify questions and ensure timely submission by the established deadline.

The contractor will create a dedicated e-mail account for this study and provide a phone number that VR agencies can call if they have any questions.

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The survey was developed by the contractor in collaboration with experienced RSA staff. The survey instrument was reviewed by a CSAVR staff person and two state VR agency directors in the process of seeking CSAVR endorsement. The contractor conducted pretests of the instrument with three state VR agencies in May/June 2011. Based on the pretesting results, minor revisions were made to the instrument.

B5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other persons who will actually collect and/or analyze the information for the agency.

Name	Affiliation	Telephone Number	Responsibility
Frank Bennici	Westat	301-738-3608	Consult and analyze data
Robert Harmon	Westat	240- 314-2348	Consult, collect, and analyze data