OJJDP_NTTAC_1121-0277_Supporting_Statement_Jul2021

Supporting Statements for the

Paperwork Reduction Act Submission

Agency: Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention

Title: OJJDP’s NTTAC Feedback Form Package

OMB No: 1121-0277

A. JUSTIFICATION

1. Necessity of Information Collection

The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is a federal agency within the Office of Justice Programs, U.S. Department of Justice. Authorized through the Juvenile Justice and Delinquency Prevention Act of 1974 (as amended in 2018 through the Juvenile Justice Reform Act), OJJDP supports programs and policies that serve children, families, and their communities, through grants to state and local governments and organizations that serve youth. OJJDP provides national leadership, coordination, and resources to prevent and respond to juvenile delinquency and victimization. OJJDP supports the efforts of states, tribes, and communities to develop and implement effective and equitable juvenile justice systems that enhance public safety, ensure youth are held appropriately accountable to both crime victims and communities, and empower youth to live productive, law-abiding lives.

OJJDP created the National Training and Technical Assistance Center (NTTAC) in 1995 to develop and disseminate juvenile justice and child victimization prevention resources, promote and coordinate collaboration among TTA providers, increase TTA responsiveness to consumer needs, and promote the use of best practices and standards in TTA assessment, planning, and delivery. OJJDP’s NTTAC supports juvenile justice and child victimization prevention professionals on the following priorities identified by the agency that relate to various juvenile justice and child victimization prevention topics:

• Supporting Law Enforcement and Prosecutors

• Child Safety

• Youth Violence Prevention

• Cyberbullying Prevention

• Restorative Justice

• Youth and Family Engagement

• Court Services

• Mentoring

• Reducing Drug Use

• Combating Gangs

• Evidence-Based Practices

• Diversion and Re-entry

The information collected by OJJDP’s NTTAC is for the purpose of assessing the ongoing needs of juvenile justice and child victimization professionals, and OJJDP TTA Providers. This information is to support the continuous improvement of TTA services. Included in this Package are 7 different feedback forms intended to assess the variety of OJJDP’s NTTAC activities while ensuring minimum burden to stakeholders. Additionally, due to the goals of the feedback effort, participants do not provide identifying information on any of the instruments.

2. Needs and Uses

This is the reinstatement with change of the previously approved OJJDP’s NTTAC Feedback Form Package. The Feedback Form Package is designed specifically to monitor the effectiveness of OJJDP’s NTTAC products and services to drive continuous improvement, quality customer service, and preliminary understanding of how services are helping the field. Many of the forms contain optional questions due to the varied nature of each event. Given the 3-year length of the OMB clearance, we have included optional questions in anticipation of possible information needs in the coming years. All feedback will be collected online if possible and the use of skip patterns/deletion of optional questions will be used to the tailor the form to each event and reduce overall burden on respondents. The expected maximum burden for each form is provided in this application to showcase the upper limit of response time. It is expected that each form will be shorter and response burden limited for each individual event. The Package is comprised of the following forms:

• Training and Meeting Participant Feedback Form – This will be completed by individuals who attend OJJDP’s NTTAC sponsored trainings and meetings, which can be held virtually (via webinar) or in person. This form is intended to capture feedback such as overall participant satisfaction, what the participant learned from the event, whether participants plan to implement evidence-based practices following the event, how participants intend to use the information in their jobs, overall feedback, and participant demographics. Given the variety of virtual and in-person events, some questions may be more applicable than others for any particular event. To reduce burden, the form will be tailored to each event. The form will be completed primarily online using skip patterns and optional questions when possible to minimize burden. Hard copy forms may also be available when requested for in-person events. The burden for this form will not exceed 5 minutes per form.

• Field Evaluation Form – This survey will gather feedback from professionals in the field on OJJDP’s NTTAC services. The survey includes questions about the OJJDP Help Desk, OJJDP resources, TTA360, and outreach and dissemination. The form also collects information on participant training needs and general demographic questions. This form will be administered online. The burden for this form will not exceed 15 minutes per form.

• TTA Services Feedback Form – This form will be completed by requesters of TTA services provided by OJJDP’s NTTAC. The form is designed to gather information about requester satisfaction with the TTA service received. The survey includes questions on general satisfaction with services, recipient behaviors following the TTA services, whether recipients plan to implement evidence-based practices following the services, how recipients intend to use the information in their jobs, overall feedback, and recipient demographics. This form includes a followup portion to administer post-TTA services with questions about reflection on satisfaction with TTA, open-ended feedback, and what behaviors the recipients actually implemented following the services. This form will be completed online or in-person. The burden for the TTA Services form will not exceed 5 minutes, and the post-TTA Services form will not exceed 5 minutes.

• Online Training Participant Feedback Form – This will be completed by individuals who complete OJJDP online trainings (should they be developed in the future). This form is intended to capture feedback from participants about general satisfaction, what participants learned from the training, applicability of the information, how the participants intend to use the information in their jobs, whether the participants plan to implement evidence-based practices following the training, overall feedback, and participant demographics. The form will be completed online to minimize burden. The burden for this form will not exceed 5 minutes per form.

• Conference Participant Feedback Form – This form is intended to provide feedback at any OJJDP-sponsored conference event supported by OJJDP’s NTTAC. This form includes tailored questions that can be asked about an overall conference or individual conference sessions. The Overall form is intended to assess logistics and conference quality as a whole. The Session form is intended to assess the utility and quality of individual sessions, if appropriate. These forms will be completed online at conferences or on the conference website. The burden for the Session form will not exceed 5 minutes and the burden for the Overall form will not exceed 5 minutes.

• Needs Sensing Form – This form includes questions that might be selected for a typical focus group or interview. It is not intended that a focus group or interview would include all of these questions. Instead, this serves as a “bank” of focus group or interview questions from which to choose. Focus groups will not last longer than 90 minutes. We may conduct focus groups with 5-10 juvenile justice and child victimization prevention professionals periodically throughout the year as needed. Focus groups and interviews will be conducted virtually or in person to discuss and clarify specific information and assistance needs of juvenile justice professionals or to gather feedback to inform planning around specific tools and resources for the field. The burden for these forms will not exceed 90 minutes per participant.

• TTA Provider Evaluation Form – This survey gathers feedback from OJJDP TTA Providers on OJJDP’s NTTAC services. The survey includes questions about webinars, the OJJDP TTA Help Desk, OJJDP resources, TTA360, and outreach and dissemination. As the coordinator of OJJDP’s Network of TTA providers, it is imperative for OJJDP’s NTTAC to understand existing resources and the types of services and tools that can increase TTA Provider capacity. The TTA Provider Evaluation form provides a mechanism to assist in prioritizing services and tools through a better understanding of existing capabilities, challenges, and strengths. The burden for this form will not exceed 10 minutes.

No sophisticated statistical techniques (e.g., sampling plans, imputation or other missing data adjustments, statistical estimation techniques) will be used. Procedures of administration are described above for each instrument. Data to be provided will be sufficiently accurate for the purpose of continuous program improvement.

1. Efforts to Minimize Burden

To anticipate changing information needs, we have included all possible questions on each form; however, we do not expect to use all the questions for every TTA event. We will tailor some forms to the TTA event to ask only relevant questions using skip-logic in survey software, with no form exceeding a 10-minute burden aside from the Needs Sensing Form and Field Evaluation. Additionally, all feedback forms will be online aside from requested TTA events and the focus group/interview guides. We also will provide other avenues for completing the forms as requested including hard copy, fax, and email. Finally, depending upon participant availability, we will conduct focus groups virtually through a webinar format or over the phone to reduce travel and availability burdens.

3. Efforts to Identify Duplication

The information to be collected is only for the purposes of OJJDP’s NTTAC and is not available elsewhere.

3. Methods to Minimize Burden on Small Businesses

Small businesses or other small entities are not a specific target population for OJJDP’s NTTAC services. However, should members of this target population request services, their level of satisfaction with the services rendered will be requested. Furthermore, the amount of potential burden placed on respondents was considered when the OJJDP’s NTTAC Feedback Form Package was developed, and every attempt was made to reduce the time and effort needed to complete the forms.

3. Consequences of Less Frequent Collection

OJJDP’s NTTAC Feedback Form Package is designed specifically to inform planning and delivery of OJJDP’s NTTAC products and services, ensuring quality and usability. Without this information, OJJDP’s NTTAC will be at a disadvantage with regard to knowledge about the quality and effectiveness of services rendered and user satisfaction.

3. Special Circumstances Influencing Collection

• Respondents to the Training and Meeting Participant Feedback Form, Online Training Feedback Form, TTA Services Form (including 3-month post-event followup form), and Conference Feedback Form are requested to complete the form immediately following the event or within one week of the event. This is important to capture immediate feedback on the event and the content covered during the event before it is forgotten.

• Respondents are not required to submit any documents.

• Respondents are not required to maintain records for this data collection effort.

• A statistically based survey method is not being used.

• Respondents are not asked to provide personally identifiable information.

3. Public Comment and Consultation

A 60- and 30-day notice will be published in the Federal Register to solicit public comment in accordance with the Paperwork Reduction Act requirements. If we receive comments, those comments will be summarized, and OJJDP’s NTTAC will take actions described herein.

3. Payment or Gift to Respondents

The purpose of this OJJDP’s NTTAC Feedback Form Package is to assess the quality and relevance of the training and technical assistance activities. No payments or gifts will be provided to respondents for completing any of the forms. Participation is voluntary.

3. Assurance of Confidentiality

All information on the feedback forms is collected in accordance with the Privacy Act of 1974. Any release of information will conform to the stipulations of the Privacy Act and the guidelines of the Institutional Review Board (IRB) as determined by Title 45 Part 46 of the Code of Federal Regulations. Only those who are required to review and process the forms will have access to the forms. Once the information from each form has been entered into an electronic database, only those persons with a valid identification, password, and permissions will have access to the information. Any physical forms will be maintained in a safe location with limited access.

3. Justification for Sensitive Questions

No questions of a sensitive nature are included in the OJJDP’s NTTAC Feedback Form Package. A “Determination of IRB Status” was completed for the OJJDP’s NTTAC Feedback Form Package and it was determined that the Package is exempt from review. However, all the protocols contained in the OJJDP’s NTTAC Feedback Evaluation Form Package do not constitute “research.” The OJJDP’s NTTAC Feedback Form Package is not a means of systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge. The OJJDP’s NTTAC Feedback Form Package was determined to be non-research activities for the purpose of obtaining customer feedback regarding technical assistance and training services provided by and through OJJDP’s NTTAC.

3. Estimates of Hour Burden

The OJJDP’s NTTAC Feedback Form Package contains forms that will require varying levels of burden hours to complete. For this reason, we have outlined the estimated annual burden hours for the forms as well as aggregated estimated burden hours for the entire Package.

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Training and Meeting Participant Feedback Form	4,000	1	5 minutes	333.3 hours
Online Training Participant Feedback Form	100	1	5 minutes	8.3 hours
Field Evaluation Form	400	1	15 minutes	100 hours
TTA Services Feedback Form	6	1	5 minutes	0.5 hours
Conference Participant Feedback Form	500	1	5 minutes	41.7 hours
Needs Sensing Form	20	1	90 minutes	30 hours
TTA Provider Evaluation Form	40	1	10 minutes	6.7 hours
Total Annual Burden	5,066		135 minutes	520.5 hours

Aggregated total number of respondent/responses for this entire package: 5,066

Percent of responses completed electronically: 86%

Aggregated annual hour burdens for entire Package: 520.5

3. Estimate of Cost Burden

The estimated total annual cost burden to respondents resulting from the information collection as part of the OJJDP’s NTTAC Feedback Form Package is $18,820.62. The Package contains forms that will require varying levels of burden hours to complete, which will affect the estimated cost burden. The estimates of annualized cost to respondents are based on appropriate wage rate categories and annual salaries for position types in which respondents serve.

Instrument	Number of Respondents	Number of Responses per Respondent	Cost per Response	Total Cost
Training and Meeting Participant Feedback Form	4,000	1	$3.47	13,880.00
Online Training Participant Feedback Form	100	1	$3.47	347.00
Field Evaluation Form	400	1	$3.47	1,388.00
TTA Services Feedback Form	6	1	$3.47	20.82
Conference Participant Feedback Form	500	1	$3.47	1,735.00
Needs Sensing Form	20	1	$65.55	1,311.00
TTA Provider Evaluation Form	40	1	$3.47	138.80
Total Annual Cost Burden				$18,820.62

3. Estimated Annualized Cost to Federal Government

We estimate the annualized cost to the Federal government to be $17,962.04. This cost estimate is based on the task order of work projected for completion under the contract for this training and technical assistance effort. As outlined below, the estimated annual Federal costs associated with the OJJDP’s NTTAC Feedback Form Package include the capital/startup and operating and maintenance costs necessary for this information collection to include: the quantification of hours for managerial and support staff to administer the OJJDP’s NTTAC Feedback Form Package process; the acquisition or development of automated, electronic, mechanical, or other technological collection techniques; and operational expenses (e.g., equipment, overhead, printing, etc.) for the three years for which this approval is sought.

• Capital/Startup costs: $1,303.02. This amount includes labor for updating and programming surveys and maintenance of information technology/survey platform to store, generate, and assist in distributing and collecting the data necessary for carrying out this effort.

• Operating and Maintenance costs: $16,659.02. This amount reflects the total annual costs for operating and maintaining any automated, electronic, mechanical, or technological collection techniques, as well as labor necessary to implement, analyze, and report on this effort.

3. Reasons for Program Changes

This information collection request does not include program changes and is an application for reinstating prior clearance to allow for feedback to be generated during the contract period and the transition to a new OJJDP’s NTTAC contract. Several changes have been made to the previously approved forms to condense the number of instruments from 13 forms to 7 forms. This feedback continues to support OJJDP’s NTTAC role as coordinator of the OJJDP TTA Network and primary webinar delivery mechanism.

3. Plans for Publication

There are no plans for publication. OJJDP’s NTTAC intends to review the results for internal program management purposes.

3. Expiration Date Approval

OJJDP’s NTTAC surveys will display the OMB control number and expiration date.

3. Exceptions to Certification Statement

There are no exceptions to Item 19 of OMB form 83-I.

B. STATISTICAL METHODS

The potential respondent universe is recipients of all OJJDP TTA services and OJJDP TTA Providers. This is an unknown universe, but the expected number of respondents based on previous years and expected activities for the coming year is 5,066. No sampling plans will be used. For eligible events, all participants will be asked to respond to the surveys. The estimated response rate for the previous calendar year (using the previously approved set of forms and different modes of collection) remained between 30% and 40% depending on the event type.

Attempts to increase response rates include providing surveys in online formats and/or hard copy format as best suited to the event, encouraging response at the time of the assistance provided, reducing the length and number of questions per survey, and tailoring each survey only to the event in which the participant attended using specific goals and objectives for each event.

We will be employing basic statistical methods in this information collection, including simple descriptive statistics (e.g., frequencies, mean) and aggregating simple descriptive statistics across multiple events. No inferential statistical tests or sophisticated statistical techniques will be conducted (e.g., sampling plans, imputation or other missing data adjustments, statistical estimation techniques). Procedures of administration are described above for each instrument. Data to be provided will be sufficiently accurate for the purpose of continuous program improvement. Therefore, items one through five (1-5) of section B are not addressed.

The organization responsible for collecting and analyzing the data is: ICF, 9300 Lee Highway, Fairfax, VA 22031. For questions related to statistical methods, please contact Janine Crossman at ICF by email, [email protected], or phone (703) 272–9581.

OMB Submission – OJJDP’s NTTAC Feedback Form Package Page 8