Revised_1121-0277_Supporting_Statement_in_minutes_1-2-2014[1]

Supporting Statements for the

Paperwork Reduction Act Submission

Agency: Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention

Title: OJJDP’s NTTAC Feedback Form Package

OMB No: 1121-0277

A. JUSTIFICATION

1. Necessity of Information Collection

The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is a federal agency within the Office of Justice Programs, U.S. Department of Justice. Authorized through the Juvenile Justice and Delinquency Prevention Act (1974), OJJDP supports programs and policies that serve children, families, and their communities, through grants to state and local governments and youth-serving organizations. OJJDP’s mission is to provide national leadership, coordination, and resources to prevent and respond to juvenile delinquency and victimization. OJJDP supports states and communities in their efforts to develop and implement effective and coordinated prevention and intervention programs and to improve the juvenile justice system so that it protects public safety, holds offenders accountable, and provides treatment and rehabilitative services tailored to the needs of juveniles and their families

Established in 1995, OJJDP’s NTTAC is designed to assist practitioners and communities identify and implement evidence-based prevention and intervention programming with customized training and technical assistance that aims to significantly impact delinquency, substance abuse and the other co-occurring problems that confront youth in the juvenile justice system everyday. To do this, the OJJDP's NTTAC supports information collection, cataloguing and dissemination efforts, provides training and technical assistance, and coordinates OJJDP’s Network of TA providers in order to increase collaboration and efficient use of resources.

OJJDP’s NTTAC is interested in assessing the ongoing needs of OJJDP TTA Providers and juvenile justice professionals as they relate to Training and Technical Assistance and informing continuous improvement of services while also developing an exploratory understanding of how TTA is helping professional do their work. There are 13 different feedback forms included in this package intended to assess the variety of OJJDP’s NTTAC activities while ensuring minimum burden to stakeholders. Additionally, due to the goals of the feedback effort, participants do not provide identifying information on any of the instruments.

2. Needs and Uses

This is the revision and extension of the previously approved OJJDP’s NTTAC Feedback form package. The revision includes separating out the OVC TTAC forms. There are also significant revisions to the items throughout all the forms. Primarily, we shortened each form to reduce burden on respondents while also asking more targeted questions that better address our information needs for each type of TA in the current iteration of the OJJDP NTTAC contract (e.g., webinar, meeting). We also added a number of questions that would be optional for each TA event. This was intended to provide flexibility for ensuring the best questions are included for each individual TA event while questions that are not necessary for a particular audience are eliminated to avoid undue burden to those respondents.   Overall, the forms are shorter, more targeted, and more reflective of OJJDP’s current information needs.

The Feedback Form Package is designed specifically to monitor the effectiveness of OJJDP’s NTTAC products and services to drive continuous improvement, quality customer service, and preliminary understanding of how services are helping the field. Many of the forms contain ‘optional’ questions due to the varied nature of each event. Given the three year length of the OMB clearance, we have included optional questions in anticipation of possible information needs in the coming years. All feedback will be collected online if possible and the use of skip patterns/deletion of optional questions will be used to the tailor the form to each event and reduce overall burden on respondents. The expected maximum burden for each form is provided in this application to showcase the upper limit of response time. It is expected that each form will be shorter and response burden limited for each individual event. The Package is comprised of the following forms:

• Webinar Feedback Form – This will be completed by individuals who attend OJJDP’s NTTAC sponsored webinars. This form is intended to capture important feedback from participants about TTA events, including the performance of the presenter, the applicability of the TTA information, and how the participant intends to use the information in their job. Given the variety of webinars, some questions may be more applicable than others for any particular event. In order to reduce burden, the form will be tailored to each event. The form will be completed online using skip patterns and optional questions when possible to minimize burden. The burden for this form will not exceed 5 minutes per form.

• Requester Feedback Form– This form will be completed by each requester of technical assistance services through OJJDP’s NTTAC. The form is designed to gather information about the requester’s satisfaction with the service received. The burden for this form will not exceed 5 minutes.

• Consultant Feedback Form – This form will be completed by any consultant providing training or technical assistance through OJJDP’s NTTAC. It is designed to gather information about the consultant’s satisfaction with the assistance/support received and with his/her assessment of the effectiveness of the service s/he delivered (what worked and what did not work). Additionally, the consultant is asked to provide information on additional assistance necessary for participants and comments related to improving the consultant process. The burden for this form will not exceed 5 minutes.

• Online Training Feedback Form – This will be completed by individuals who complete online trainings through OJJDP’s Online University. This form is intended to capture important feedback from participants about the technology utilized, quality of the training content, applicability of the information, and how the participant intends to use the information in their job. Given the variety of online trainings and various training developers, some questions may be more applicable than others for any particular training. In order to reduce burden, the form will be tailored to each training event. The form will be completed online using skip patterns and optional questions when possible to minimize burden. The burden for this form will not exceed 5 minutes per form.

• Website Feedback Form – This form will be completed on an as needed basis for users of the OJJDP’s NTTAC website. The form will be available on a continuous basis online through the website or delivered by an e-mail link to assess a specific resource/tool/revision on the site. The form is designed to gather information on the usability, appropriateness, and effectiveness of the Web site. The burden for this form will not exceed 5 minutes.

• Participant Follow-up Interview Guide – For events of high interest or priority, follow-up interviews with randomly selected participants will provide extended information into the utility of the event to the field. The Participant Follow-Up Interview Form measures the extent to which participants used the information and tools provided during a specific event as well as their perspective on the quality and relevance of the event. This form will be administered online, over the phone, or by e-mail as needed to reach participants. The burden for this form will not exceed 10 minutes.

• Overall Conference Feedback Form and Conference Session Feedback Form – These forms are intended to provide feedback at any OJJDP-sponsored conference event facilitated by OJJDP’s NTTAC. The Overall form is intended to assess logistics and conference quality as a whole. The Session form is intended to assess the utility and quality of individual sessions, if appropriate. These forms will be completed online at conferences or on the conference website. The burden for the session form will not exceed 3 minutes and the burden for the overall form will not exceed 5 minutes.

• Focus Group Guide for Needs Sensing or the OJJDP’s NTTAC Focus Group Guide for Resources/Tools – We may conduct focus groups with 8-10 juvenile justice professionals periodically throughout the year as needed. Focus groups will be conducted in a webinar/phone or in-person format to discuss and clarify specific information and assistance needs of juvenile justice professionals or gather feedback to inform planning around specific tools and resources for the field. The burden for these forms will not exceed 90 minutes per participant.

• General Meeting Feedback Form or FACJJ Meeting Feedback form – OJJDP’s NTTAC facilitates meetings for OJJDP as needed including facilitation of the Federal Advisory Committee for Juvenile Justice meetings. These feedback forms will assist in understand participants perspectives around the agenda, meeting utility, meeting format (e.g. webinar), and utility to their work in order to inform meeting planning. The burden for these forms will not exceed 5 minutes.

• TA Provider Survey – As the coordinator of OJJDP’s Network of TTA providers, it is imperative to understand existing resources and the types of services and tools that can increase TA Provider capacity. The TA Provider Survey will give us a mechanism to assist in prioritizing services and tools through a better understanding of existing capabilities, challenges, and strengths. The burden for this form will not exceed 10 minutes.

3. Efforts to Minimize Burden

To anticipate changing information needs, we have included all possible questions on each form however; we do not expect to use all the questions for every TA event. We will tailor each form to each event with no form exceeding a five minute burden aside from the focus group guides, TA Provider survey and follow-up interviews. Additionally, all feedback forms will be online aside from the focus group guides. We will also provide other avenues for completing the forms as requested including hard copy, fax, and e-mail. Finally, depending upon participant availability, we will conduct focus groups through a webinar format or over the phone to reduce travel and availability burdens.

3. Efforts to Identify Duplication

The information to be collected is only for the purposes of OJJDP’s NTTAC and is not available elsewhere.

3. Methods to Minimize Burden on Small Businesses

Small businesses or other small entities are not a specific target population for OJJDP’s NTTAC services. However, should members of this target population request services, their level of satisfaction with the services rendered will be requested. Furthermore, the amount of potential burden placed on respondents was considered when the OJJDP’s NTTAC Feedback Form Package was developed and every attempt was made to reduce the time and effort needed to complete the forms.

3. Consequences of Less Frequent Collection

The Feedback Form Package is designed specifically to inform planning and delivery OJJDP’s NTTAC products and services, ensuring quality and usability. Without this information, OJJDP’s NTTAC will be at a disadvantage with regard to knowledge about the quality and effectiveness of services being rendered and user satisfaction.

3. Special Circumstances Influencing Collection

• Respondents of the Webinar Feedback Form and Online Training Feedback Form, Overall Conference Feedback Form and Conference Session Feedback Form are requested to complete the form immediately following the event or within one week of the event. This is important to capture immediate feedback on the event and the content covered during the event before it is forgotten.

• Respondents are not required to submit any documents.

• Respondents are not required to maintain records for this data collection effort.

• A statistically based survey method is not being used.

• Respondents are not asked to provide personally identifiable information.

3. Public Comment and Consultation

A 60- and 30-day notice has been published in the Federal Register to solicit public comments in accordance with the Paperwork Reduction Act requirements. One comment was received requesting a copy of each form upon finalization. Upon OMB approval of final forms, we will provide those forms to the requester.

3. Payment or Gift to Respondents

The purpose of this OJJDP’s NTTAC Feedback Form Package is to assess the quality and relevance of the training and technical assistance activities. No payments or gifts will be provided to respondents for completing any of the forms. Participation is voluntary.

3. Assurance of Confidentiality

All information on the feedback forms is collected in accordance with the Privacy Act of 1974. Any release of information will conform to the stipulations of the Privacy Act and the guidelines of the Institutional Review Board (IRB) as determined by Title 45 Part 46 of the Code of Federal Regulations. Only those who are required to review and process the forms will have access to the forms. Once the information from each form has been entered into an electronic database, only those persons with a valid identification, password, and permissions will have access to the information. Any physical forms will be maintained in a safe location with limited access.

3. Justification for Sensitive Questions

There are no questions of a sensitive nature. A “Determination of IRB Status” was completed for the OJJDP’s NTTAC Feedback Form Package and it was determined that the Package is exempt from review. Policies requiring IRB review are applicable to research activities as stated in 45 CFR §46.101(a). However, all of the protocols contained in the OJJDP’s NTTAC Feedback Form Package do not constitute “research” as it is defined in 45 CFR §46.102(d). The OJJDP’s NTTAC Feedback Form Package is not a “means of systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” (45 CFR §46.102(d)). On the contrary, the OJJDP’s NTTAC Feedback Form Package was determined to be non-research activities for the purpose of obtaining customer feedback regarding technical assistance and training services provided by and through OJJDP’s NTTAC.

3. Estimates of Hour Burden

The OJJDP’s NTTAC Feedback Form Package contains forms that will require varying levels of burden hours to complete. For this reason, we have outlined the estimated annual burden hours for the forms as well as aggregated estimated burden hours for the entire Package.

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden hours per Response	Total Burden Hours
Webinar Participant Feedback Form	3500	1	5 minutes	291.67
Online Training Participant Feedback Form	500	1	5 minutes	41.67
General Meeting Feedback Form	50	1	5 minutes	4.17
FACJJ Meeting Feedback Form	20	1	5 minutes	1.67
Requester Feedback Form	40	1	5 minutes	3.33
Consultant Feedback Form	20	1	5 minutes	1.67
Overall Conference Feedback Form	200	1	5 minutes	16.67
Conference Session Feedback Form	600	1	3 minutes	30
Website Feedback Form	100	1	5 minutes	8.33
Follow-up Interview Guide	20	1	10 minutes	3.33
Focus Group Guide for Needs Sensing.	20	1	90 minutes	30
Focus Group Guide for Resources/Tools	20	1	90 minutes	30
TA Provider Survey	50	1	10 minutes	8.33
Total Annual Burden	5140			470.83

Aggregated total number of respondent/responses for this entire package: 5140

Percent of Responses completed electronically: 99%

Aggregated annual hour burdens for entire Package: 470.83

3. Estimate of Cost Burden

The estimated total annual cost burden to respondents resulting from the collection of information as part of the Feedback Form Package is $19,260.90. The Package contains forms that will require varying levels of burden hours to complete, which will affect the estimated cost burden. The estimates of annualized cost to respondents are based on appropriate wage rate categories and annual salaries for position types in which respondents serve.

Instrument	Number of Respondents	Number of Responses per Respondent	Cost per Response	Total Cost
Webinar Participant Feedback Form	3500	1	$3.47	$12145.00
Online Training Participant Feedback Form	500	1	$3.47	$1735.00
General Meeting Feedback Form	50	1	$3.47	$174.50
FACJJ Meeting Feedback Form	20	1	$3.47	$69.40
Requester Feedback Form	20	1	$3.47	$138.80
Consultant Feedback Form	20	1	$3.47	$69.40
Overall Conference Feedback Form	200	1	$3.47	$694.00
Conference Session Feedback Form	600	1	$1.30	$780.00
Website Feedback Form	100	1	$3.47	$347.00
Follow-up Interview Guide	20	1	$6.94	$138.80
Focus Group Guide for Needs Sensing.	20	1	$65.55	$1311.00
Focus Group Guide for Resources/Tools	20	1	$65.55	$1311.00
TA Provider Survey	50	1	$6.94	$347.00
Total Annual Cost Burden				$19,260.90

3. Estimated Annualized Cost to Federal Government

We estimate the annualized cost to the Federal government to be $65,468.31. This cost estimate is based on the task order of work projected for completion under the contract for this training and technical assistance effort. As outlined below, the estimated annual Federal costs associated with the OJJDP’s NTTAC Feedback Form Package include the capital/startup and operating and maintenance costs necessary for this information collection to include: the quantification of hours for managerial and support staff to administer the Feedback Form Package process; the acquisition or development of automated, electronic, mechanical, or other technological collection techniques; and operational expenses (e.g., equipment, overhead, printing, etc.) for the three years for which this approval is sought.

• Capital/Startup costs: $10,072.48. This amount includes instrument design and development. This amount also includes the use and maintenance of information technology to store, generate, and assist in distributing and collecting the data necessary for carrying out this effort.

• Operating and Maintenance costs: $55,395.83. This amount reflects the total annual costs for operating and maintaining any automated, electronic, mechanical or technological collection techniques, as well as, the labor necessary to implement, analyze and report on this effort.

3. Reasons for Program Changes

Due to agency action, this information collection is the result of OJJDP’s NTTAC new role as coordinator of the OJJDP TA Network and primary webinar delivery mechanism. We have asked for revisions in the way that NTTAC collects information to better reflect the changing information needed to improve products and services without undue burden on respondents with tailored, more relevant forms for each event.

3. Plans for Publication

There are no plans for publication. OJJDP intends to review the results for internal program management purposes.

3. Expiration Date Approval

OJJDP’s NTTAC surveys will display the OMB control number and expiration date.

3. Exceptions to Certification Statement

There are no exceptions to Item 19 of OMB form 83-I.

OMB Submission –OJJDP’s NTTAC Feedback Form Package Page 9