SuppStatement_2013_OnlineTr_2013-09-05_Revised Materials Form 1 12 15

Supporting Statements for the

Paperwork Reduction Act Submission

Agency: Office of Justice Programs, Office for Victims of Crime

Title: OVC TTAC Online Trainings Package:

Online Training Form

Victim Law Survey

NVAA Instructor Feedback

NVAA Supervisor Feedback

Materials Feedback

Focus Group Protocol

A. JUSTIFICATION

1. Necessity of Information Collection

The Office for Victims of Crime (OVC) is a Federal agency within the Office of Justice Programs, U.S. Department of Justice. This is a request for a new collection to serve the information needs of the OVC Training and Technical Assistance Center (TTAC) under the Office of Justice Program contract.

Congress formally established OVC in 1988 through an amendment to the 1984 Victims of Crime Act (VOCA) to provide leadership and funding on behalf of crime victims. The mission of OVC is to enhance the nation’s capacity to assist crime victims and to provide leadership in changing attitudes, policies, and practices to promote justice and healing for victims of crime. As one part of its mission, OVC is committed to providing victims of crime with access to comprehensive, quality services. One of the ways it does this is through its Technical Assistance, Publications and Information Resources Unit (TAPIR). The TAPIR develops and disseminates training, technical assistance (TA), and information resources that support victim service providers and allied professionals. To further the dissemination of these resources, TAPIR manages the OVC Resource Center (OVCRC), the OVC Training and Technical Assistance Center (OVC TTAC), education and outreach initiatives, and the publication and dissemination of OVC materials and grant products.

This training and technical assistance center works to further the agency’s mission through increasing access to resources in the field. OVC TTAC was created in 1998 to serve as a central access point for OVC's training and technical assistance (TTA) resources and to funnel resources to local, state, tribal, and Federal agencies to strengthen their capacity to serve victims. OVC sponsors training on victim issues for service providers, law enforcement personnel, prosecutors, the judiciary, clergy, and medical and mental health professionals. OVC TTAC provides user-friendly, efficient, and cost-effective resources by training agencies and organizations on victim-related topics; providing technical assistance in areas such as policy and program development, management, and evaluation; and maintaining a Consultant Network of experts to support OVC's initiatives, customized TTA, and operate a speaker’s bureau for conferences, focus groups, and other meetings. OVC TTAC does this through a process that includes needs assessment, analysis, service coordination, and follow-up. Secondly, OVC TTAC works collaboratively with the OVC training and TA provider consortium to help develop or enhance their service delivery capabilities. OVC TTAC assistance in this area includes materials development, arranging topical training sessions, the delivery of specialized TA to support program development, and maintenance of an interactive Web site designed to cultivate communication across providers and to promote shared learning between the providers and the field at-large.

OVC and OVC TTAC are interested in assessing client satisfaction with assistance provided, and obtaining client feedback on how assistance can be improved. OVC TTAC’s evaluation team conducts these assessments for OVC by collecting data from participants, requesters, and consultants/instructors, analyzing this data, and creating reports for OVC TTAC and OVC for internal improvement purposes. The current package includes six instruments that collectively establish the OVC TTAC Online Trainings Package, including three surveys related to online trainings, two surveys about online resources, and one focus group protocol for virtual and in-person focus groups. With the exception of the Focus Group protocol, the evaluation team does not share identifiable survey data with anyone outside of the evaluation team. The Focus Group protocol will also report all information as de-identified; however, this form may also be administered and resulting information synthesized by the programmatic team of OVC TTAC (as opposed to the evaluation team). In addition, surveys do not require identifiable information, although participants may provide contact information for a future followup survey if they desire. An exception to this is for forms where identification is required for CEU purposes (e.g., to let the CEU team know that the evaluation form was completed) or for reimbursement purposes (e.g., the NVAA instructor form). The procedures for sharing and protecting this personally-identifiable information are described in item 2 below. All information collected on the feedback forms is protected in accordance with the Privacy Act of 1974.

2. Needs and Uses

This is a new collection related to online trainings, resources, and focus groups (may also be conducted in-person). This information collection request is designed specifically to collect information on the needs of the field and to monitor the effectiveness of OVC TTAC’s programming, ensuring accountability, and quality customer service. The Package is comprised of the following forms: Online Training Form, Victim Law Survey, NVAA Instructor Feedback, NVAA Supervisor Feedback, Materials Feedback, Focus Group Protocol. Each instrument will be used to assess client satisfaction with of OVC TTAC’s training and technical assistance activities, explore the needs of the field, and to assess client perspectives on how such services can be improved.

Personally identifying information will not be released by the evaluation team, with the exception of (1) providing names of those who completed forms that are required for reimbursement purposes, (2) providing names of those who completed forms that are required for CEU accreditation, and (3) the focus group protocol which may be used by the programmatic team of OVC TTAC rather than the evaluation team. In all cases, the instructions clearly and explicitly explain these restrictions on confidentiality.

OVC TTAC employs on-site procedures to further secure personally identifiable information. Evaluation data cannot be viewed by anyone outside of the evaluation team members, who have signed certificates of confidentiality. These certificates indicate that the identity of persons interviewed and related data are to remain confidential; that the removal of names or disclosure of identities and related information is strictly forbidden; and that the contents of interviews are not to be discussed with anyone except Needs Assessment and Evaluation team members. Online survey data is stored on a secure Web server until it is extracted and imported into the evaluation team’s databases in a secure-access folder. All data is aggregated for analyses and reporting (except for the focus group protocol where data will be de-identified but not aggregated).

• The Online Training Form will be administered to every person accessing OVC TTAC’s applicable online trainings, either by email collected through registration or by a pop-up at the end of the training. The forms are completed through an on-line survey, and hard-copy versions are available upon request (these would be emailed or faxed to the respondent, as needed). The form is designed to gather information about the participant’s satisfaction with the service received. The form is also customizable to a number of online training needs. For instance, the introduction can either say that it’s completely voluntary or that it is required for CEU accreditation, depending on the training. Some trainings have modules and necessitate the module-level questions, while these questions would be removed for simpler online trainings. Self-paced trainings will not use presenter questions, while trainings with “virtual classrooms” with an active instructor (like the NVAA described below) will. Highlighted questions are also known questions that are relevant for only certain online trainings. This form does not request that the respondent provide personally-identifiable information other than their name if completion is required for CEU accreditation or if they want to participate in a 3-month followup survey (survey previously approved by OMB control #1121-0341.

• The Victim Law Survey will be administered by a pop-up to every person (or every nth person depending on Web traffic) accessing OVC TTAC’s VictimLaw Web site. The form is completed through an on-line survey, and hard-copy versions are available upon request (these would be emailed or faxed to the respondent, as needed). The form is designed to gather information about the participant’s satisfaction with the service received. This form does not request that the respondent provide personally-identifiable information other than their zip code for purposes of mapping respondents to understand usage trends.

• The NVAA Instructor Feedback form will be administered to instructors of the National Victim Assistance Academy, a blended format academy with three online trainings and one blended training that has an online portion and an in-person portion. The instructors will be sent a link to an online survey by email, but they can request to complete a hard-copy version if they prefer. The survey is designed to gather information about the instructor’s satisfaction with the assistance received from OVC TTAC in order to prepare and deliver the training, feedback on the curriculum and materials, and suggestions for improvement for the NVAA. This form does not request that the respondent provide personally-identifiable information, other than the training they instructed. This will be used to identify who has fulfilled the reimbursement requirements.

• The NVAA Supervisor Feedback form will be administered to supervisors of National Victim Assistance Academy participants. Participants give their supervisor’s contact information when they register for the training. The Evaluation Team uses this contact information to email supervisors a link to an online survey (hard-copy versions can be sent by request) three months after completion of the Academy. The survey is designed to gather information about how the participant’s involvement of the NVAA impacted the participant, their work, and the organization for which they work. This form does not request that the respondent provide personally-identifiable information.

• The Materials Feedback form will be administered to every person accessing OVC TTAC’s applicable online curriculum resources through a pop-up request to participate in a survey (either at that time or at a later date). The form is completed through an on-line survey, and hard-copy versions are available upon request (these would be emailed or faxed to the respondent, as needed). The form is designed to gather information about the participant’s satisfaction with the downloaded materials. This form does not request that the respondent provide personally-identifiable information other than their zip code for purposes of mapping respondents to understand usage trends.

• The Focus Group protocol will be used for leading focus groups, either through on-line forums, other virtual meetings, or in-person meetings. The protocol includes a wide variety of questions that might be asked in a typical focus group. No single focus group would use all of the listed questions; instead, this would serve to be a list of approved questions for focus groups in the future. The typical focus group would cover approximately 12 questions during a 90-120 minute period. The questions are designed to gather information about diverse topics, including providing feedback on resources or tools, exploring the needs of a field, and gathering information about a certain project or response, such as planning activities, challenges, and lessons learned. The protocol does not include questions of a sensitive nature, and all responses will be de-identified for reporting.

3. Efforts to Minimize Burden

The evaluation team is committed to reducing the burden on survey respondents to the extent possible. Flexible modes of completion, use of online surveys to minimize data entry, and customizable instruments that allow for use of most relevant items should help minimize the burden on respondents. For instance, respondents are invited to complete online surveys for these activities (except the focus group protocol), but they may request a hard-copy form if they prefer to complete it through this mode. During the development process, survey forms were streamlined to focus on including the most relevant questions. Furthermore, instruments such as the online training form and focus group protocol also allow for customization to select only the necessary items from the listed questions, depending on the resource or activity being evaluated. These efforts were made to reduce the time burden on respondents. General data entry and analysis will be conducted using SPSS and other comparable statistical software.

3. Efforts to Identify Duplication

The information to be collected is only for the purposes of OVC TTAC and is not available elsewhere.

3. Methods to Minimize Burden on Small Businesses

Small businesses or other small entities are not a specific target population for OVC TTAC services. However, should members of this target population request services, their level of satisfaction with the services rendered may be requested. Furthermore, the amount of potential burden placed on respondents was considered when the OVC TTAC Online Training Package was developed, and every attempt was made to reduce the time and effort needed to complete the forms.

3. Consequences of Less Frequent Collection

The OVC TTAC Online Training Package is designed specifically to monitor the effectiveness of OVC TTAC‘s programming, ensuring accountability, and quality customer service. The data will then be used to advise OVC TTAC on ways to improve the support provided to its users and the victim service fields at-large. Without this information, OVC TTAC will be at a disadvantage with regard to knowledge about the quality and effectiveness of services being rendered and user satisfaction.

3. Special Circumstances Influencing Collection

• Participants of online trainings are requested to complete the form immediately following the completion of the training (via pop-up or email). This is important to capture immediate feedback on the training and the content covered during the training before it is forgotten. Instructor respondents have up to one week to respond.

• Respondents are only required to submit surveys for those needing reimbursement (instructors) or if an individual wants to receive CEU credits.

• Respondents are not required to maintain records for this data collection effort.

• A statistically-based survey method (i.e., with probability-sampling, missing response adjustment/analysis, or statistical estimation techniques) is not being used.

• The statement of confidentiality on the survey forms conforms to the Privacy Act of 1974. A statement of confidentiality is provided with explanations of the limitations of confidentiality and voluntary nature of surveys. Respondents are not asked to disclose sensitive or protected information for any survey forms.

3. Public Comment and Consultation

A 60- and 30-day notice will be published in the Federal Register to solicit public comments in accordance with the Paperwork Reduction Act requirements. If we receive comments, those comments will be summarized and actions taken by OVC TTAC described herein. Moreover, experts in the field of training and technical assistance have been consulted in the creation of the OVC TTAC Online Training Package. These experts have helped to refine the forms to ensure that they are comprehensive yet not overly burdensome for respondents.

3. Payment or Gift to Respondents

No payments or gifts will be provided to respondents for completing any of the forms. However, completing the forms is one of the multiple requirements for instructors receiving reimbursement. Participation is voluntary except in situations when respondents would like reimbursement or CEU credits.

3. Assurance of Confidentiality

All information on the feedback forms will be protected in accordance with the Privacy Act of 1974. Any release of information will conform to the stipulations of the Privacy Act and the guidelines of the Institutional Review Board (IRB) as determined by Title 45 Part 46 of the Code of Federal Regulations (see Section 11 for information on IRB). Only members of the evaluation team will have access to completed forms for the purposes of entry and analysis (except the focus group form which will be used by the programmatic team of OVC TTAC). Once the information from each form has been entered into an electronic database, only those persons with a valid identification, password, and permissions will have access to the information. No personally-identifiable information will be contained within the electronic database, other than contact information for followup surveys or names for reimbursement or CEU crediting purposes. The physical forms will be maintained in a locked filing cabinet with limited access.

3. Justification for Sensitive Questions

There are no questions deemed to be sensitive in nature. OVC TTAC’s evaluation activities have undergone review and been approved by ICF’s Institutional Review Board (IRB) (ID#112136), which was established to ensure that research is conducted in compliance with Federal regulations, particularly Title 45 Code of Federal Regulations, Part 46, which is the general IRB rule applicable to Federally sponsored research. The primary purpose of the IRB is to protect the welfare of human research subjects and to ensure that physical, psychological and social risks to them are minimized.

3. Estimates of Hour Burden

The OVC TTAC Online Training Package contains forms that will require varying levels of burden hours to complete. For this reason, we have outlined the estimated annual burden hours for each form as well as aggregated estimated burden hours for the entire Package. These estimates are based on the expected number of events within each activity type and average number of respondents per event from past years. Pilot testing of the forms with staff was used to derive average completion times. Actual calculated times were rounded up to the next multiple of 5 for conservative estimates.

The Online Training Form will be completed by participants of a variety of online trainings. The survey is customizable and all listed items are not expected to be used for any single training. However, as a conservative estimate of burden, we will estimate with the full number of listed items (47). These items include mostly closed-ended and rating scale items with a small number of open-ended questions. This form is expected to take approximately 10 minutes to complete. Approximately 11,605 individuals are expected to complete the forms on an annual basis for an estimated total annual burden of 1,856.8 hours.

Number of Respondents (annually): 11,605

Frequency of Response: Once

Average Burden Hours Per Response: 0.16 hrs. (10 minutes)

Estimated Total Annual Burden Hours: 1,856.8

The VictimLaw Survey will be completed by individuals accessing the VictimLaw Web site. The survey has 22 items, including mostly closed-ended and rating scale items with a small number of open-ended questions. This form is expected to take approximately 10 minutes to complete. Approximately 100 individuals are expected to complete the form on an annual basis for an estimated total annual burden of 16 hours.

Number of Respondents (annually): 100

Frequency of Response: Once

Average Burden Hours Per Response: 0.16 hrs. (10 minutes)

Estimated Total Annual Burden Hours: 16

The NVAA Instructor Feedback form will be completed by individuals instructing National Victim Assistance Academy online and blended trainings. The survey has 15 items, including mostly closed-ended and rating scale items with a small number of open-ended questions. This form is expected to take approximately 5 minutes to complete. Approximately 8 individuals are expected to complete the form on an annual basis for an estimated total annual burden of 0.66 hours.

Number of Respondents (annually): 8

Frequency of Response: Once

Average Burden Hours Per Response: 0.08 hrs. (5 minutes)

Estimated Total Annual Burden Hours: 0.66

The NVAA Supervisor Feedback form will be completed by individuals supervising NVAA participants. The survey has 21 items, including mostly closed-ended and rating scale items with a small number of open-ended questions. This form is expected to take approximately 10 minutes to complete. Approximately 360 individuals are expected to complete the form on an annual basis for an estimated total annual burden of 57.6 hours.

Number of Respondents (annually): 360

Frequency of Response: Once

Average Burden Hours Per Response: 0.16 hrs. (10 minutes)

Estimated Total Annual Burden Hours: 57.6

The Materials Feedback form will be completed by individuals using materials obtained from OVC TTAC (typically by downloading from the OVC TTAC Web site). The survey has 28 items, including mostly closed-ended and rating scale items with a small number of open-ended questions. This form is expected to take approximately 5 to 10 minutes to complete. Approximately 500 individuals are expected to complete the form on an annual basis for an estimated total annual burden of 65 hours.

Number of Respondents (annually): 500

Frequency of Response: Once

Average Burden Hours Per Response: 0.13 hrs. (7.5 minutes)

Estimated Total Annual Burden Hours: 65 (500 people x .13 hours)

The Focus Group Protocol will be used to facilitate focus groups. Similar to the Online Training Form, this protocol includes a wide range of questions, not all of which would be used during any one session. Typically, these focus groups will have discussion on approximately 12 open-ended discussion questions across a 90-120 minute period (we estimate 105 minutes below). Up to 225 individuals are expected to participate in focus groups on an annual basis for an estimated total annual burden of 393.75 hours.

Number of Respondents (annually): 225

Frequency of Response: Once

Average Burden Hours Per Response: 1.75 hrs. (105 minutes)

Estimated Total Annual Burden Hours: 393.75

Aggregated total number of respondents/responses for this entire package: 12,398

Percent of Responses expected to complete electronically: 99%

Aggregated annual hour burdens for entire Package: 2,340.81

3. Estimate of Cost Burden

The estimated total annual cost burden to respondents resulting from the collection of information as part of the OVC TTAC Online Training Package is $43,068.34. The Package contains forms that will require varying levels of burden hours to complete, which will affect the estimated cost burden. The estimates of annualized cost to respondents are based on appropriate wage rate categories and annual salaries for position types in which respondents serve.

Training Participants/Online Resource Users: 12,390 responses x $3.47 per response = $42,993.30.

Instructors: 8 responses x $9.38 per response = $75.04.

Total annual cost: $43,068.34.

3. Estimated Annualized Cost to Federal Government

We estimate the annualized cost to the Federal government to be $89,443. This cost estimate is based on the work projected for completion under the contract for this training and technical assistance effort. As outlined below, the estimated annual Federal costs associated with the OVC TTAC Online Training Package include the capital/startup and operating and maintenance costs necessary for this information collection to include: the quantification of hours for managerial and support staff to administer the Online Training Package process; the acquisition or development of automated, electronic, mechanical, or other technological collection techniques; and operational expenses (e.g., equipment, overhead, printing, etc.) for the three years for which this approval is sought.

• Capital/Startup costs: $6,530. This amount includes labor for instrument design and development, as well as labor for incorporating surveys into existing online training platforms.

• Operating and Maintenance costs: $82,913. This amount reflects the total annual costs for operating and maintaining evaluation activities, including the necessary software and labor necessary to implement, analyze, and report on this effort and printing costs for paper surveys.

3. Reasons for Program Changes

This information collection request is a request for a new OMB control number. OVC TTAC has additional evaluation feedback forms currently approved under OMB Control Number 1121-0341. However, these are new forms to supplement the other package in order to obtain feedback from other types of assistance, including online trainings, online resources, and focus groups (both virtual and in-person).

3. Plans for Publication

There are no current plans for external publication. OVC intends to review the results for internal program management purposes.

3. Expiration Date Approval

OVC TTAC will display the OMB control number and expiration date.

3. Exceptions to Certification Statement

There are no exceptions to Item 19 of OMB form 83-I.

OMB Submission – OVC TTAC Online Training Package Page 11