VISTA Survey SSB

SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSIONS

B. Statistical Methods

1. Respondent Universe and Sampling Methods

Describe sample design, including plans for stratification or other specialized sampling procedures. If plan to survey the entire universe, explain why this is essential/preferable to using a sample.

The proposed data collection is an online survey will be an online survey of 800 VISTA sponsors. The sample will comprise a complete census of VISTA project leads. CNCS will identify the most appropriate project leads at each sponsor organization. The participating VISTA sponsors are expected to reflect these project types:

• Standard Projects, which represent the majority of projects and receive no direct federal funding, with CNCS providing the member living allowance and other limited benefits directly to the VISTA member;

• Program Grants, which receive federal funds from CNCS and are expected to use grant dollars to cover project related expenses such as the VISTA member living allowance, VISTA member travel, and personnel expenses;

• Support Grants, which receive federal funds from CNCS to cover all or a portion of personnel expenses and/or member travel, and for which CNCS provides the member living allowance and other limited benefits directly to the VISTA member; and

• National and Multi-State Intermediary Projects, which include sponsoring organizations that have VISTA members serving in two or more states, and may be standard, program, or support projects.

2. Information Collection Procedures

Describe the methods to be used for developing the sampling frame and selecting the sample. Describe fully the information collection procedures, including advance letters or phone calls, and plans for follow-up (to be described more fully in B.3. – the next section below).

CNCS will send each VISTA sponsor a pre-notification email to inform them about the survey and to request their participation. JBS will then send a personalized survey invitation to project lead or site supervisor at each sponsoring organization. The invitation email will contain information assuring respondents of the confidentiality of their responses. Following the survey invite, reminders will be sent to non-respondents and those with partial completion of surveys. Finally, non-respondents will be contacted by phone up to three times to remind them about the survey and provide the options to complete the survey online or by phone.

3. Methods to Maximize Response Rates

Describe methods to maximize response rates, and to deal with issues of nonresponse. For most data collections, OMB requires that response rates be at least 75-80%, preferably 80%. You must provide convincing evidence to support the estimated response rate/s. If a similar study had been done in the past few years, it is appropriate to use that experience as the basis for the expected response rate to the proposed study.

Following the survey invitation, reminders will be sent to non-respondents and those with partial completion of surveys. Finally, non-respondents will be contacted by phone up to three times to remind them about the survey and provide the options to complete the survey online or by phone. JBS researchers will enhance intrinsic motivation to complete the survey sponsors by highlighting the ways that survey data will be used to help all sponsors and by emphasizing the importance of their individual response.

4. Tests of Procedures

Describe any test of procedures of methods to be undertaken. All data forms and data collection procedures submitted for clearance should be pilot tested. Tests involving 9 or fewer respondents are within OMB exemptions for clearance. However, larger pilot tests must first have OMB approval. If instruments have previously been tested and have known psychometric properties they should be explicitly discussed, including literature citations. Similarly, if scales that have been previously tested and validated are included be certain to identify them here.

JBS conducted a pilot test of the survey with eight VISTA sponsor site supervisors, project directors, project managers and intermediaries. This pilot test included content review focusing on completeness of instructions and questions for respondents; clarity of instructions and questions for respondents; relevance of questions for target audience; use of appropriate and relevant terms; and appropriateness of format. JBS also conducted cognitive interviews during pre-test to determine the extent to which respondents’ understandings of the questions match their intended meaning. The eight respondents for the pilot of the survey took the survey and a JBS researcher conducted the phone interview within five days after the respondent completed the survey. The phone interviews occurred between February 28 – March 9, 2020.

Respondents’ feedback from the pilot test revealed that the survey instructions, navigation, questions, and response options were clear and were interpreted as intended, and that survey questions were relevant to VISTA sponsors. Respondents’ feedback indicates that the survey easy to complete, though some found it difficult to respond to the question about their role on the VISTA project. For questions that ask participants to select up to three responses they thought it would be helpful to have respondent rank their responses. Some pilot testers suggested adding questions on the usefulness of conference convenings. For recruitment tools and practices (question 8) some pilot testers suggested adding “word of mouth” as a response option. JBS staff reviewed the results from the pilot and revised the survey to incorporate the pilot testers’ feedback.

5. Statistical Consultants

Provide the contact information (names, titles, addresses, phone numbers, e-mail address) of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and /or analyze the information for the agency. If the persons collecting or analyzing the data are contractors, list their contact information and the names of the personnel responsible for receiving and approving contract deliverables. If analysis is to be done by someone other than the contractor or project officer, provide their contact information.

Consultant: JBS International, Inc.

The following individuals were consulted regarding the statistical methodology: Annie Georges, Ph.D.; Gina Cardazone, Ph.D.; Robert Shannon, M.A.; Victoria Wheat, M.A; and Donald Pratt, Ph.D.