Supporting Statement A - Formative Generic Program Support: Tribal CQIC

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Tribal Continuous Quality Improvement Collaboratives

Formative Data Collections for Program Support

0970 - 0531

Supporting Statement

Part A

February 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Nicole Denmark, Office of Planning, Research, and Evaluation

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for Program Support (0970-0531)

• Description of Request:

This information request is to support the implementation of the Tribal Continuous Quality Improvement Collaboratives (Tribal CQICs). The Early Language and Literacy and Family Engagement Tribal CQICs bring together 19 grantees for a 10-month long structured, and facilitated process for testing evidence-based strategies to make practice improvements towards each collaborative’s aim. Grantee teams in the two collaboratives will complete rapid Plan-Do-Study-Act (PDSAs) cycles of change in practice, assess the results, and scale changes within their own programs. Information collection will include grantees’ planning documents for the rapid cycle tests and a standard set of measures. The information will be used to provide technical assistance to participating grantees and to inform grantees in each collaborative of the progress towards the collaborative’s aim.

We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivity: Tribal CQIC is scheduled to launch March 5^th and ideally data collection will begin immediately.

A1. Necessity for Collection

The Administration for Children and Families’ (ACF) Office of Planning, Research, and Evaluation (OPRE) at the U.S. Department of Health and Human Services (HHS) seeks approval under the umbrella generic: Formative Data Collections for Program Support (OMB #0970-0531) to collect implementation information from 19 Tribal Maternal, Infant, and Early Childhood Home Visiting grantees (Tribal MIECHV) participating in the Early Language and Literacy and Family Engagement Tribal Continuous Quality Improvement Collaboratives (Tribal CQICs).

Based on decades of research, early childhood home visiting programs aim to support families in promoting their children’s development in multiple domains, including early language and literacy. But as evidence-based home visiting programs are brought to scale, impacts on parenting and child development have been more modest. Research suggests that family engagement, including participating and developing trusting relationships with home visitors, is key to home visiting impacts. Grantees participating in the Early Language and Literacy and Family Engagement Tribal CQICs will test strategies to enhance early language and literacy and family engagement, respectively.

There are no legal or administrative requirements that necessitate or authorize this information collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• Delivery of targeted assistance and workflows related to program implementation or the development or refinement of program and grantee processes

• Planning for provision of programmatic or evaluation-related training or technical assistance (T/TA).

The Tribal CQICs, run by ACF’s Tribal MIECHV program utilize the Breakthrough Series Collaborative (BSC) methodology, which is a quality improvement process utilized extensively in health care and human service sectors to create practice change. The Early Language and Literacy and Family Engagement Tribal CQICs bring together 19 grantees for a 10-month long structured, and facilitated process for testing evidence-based as well as innovative strategies to make improvement toward collaborative-level aims related to Early Language and Literary of Family Engagement. The collaborative aim is a quantifiable desired outcome of the collaborative (e.g the collaborative aim for Early Language and Literacy is to increase the percentage of families who read, sign, or tell stories to their children every day). Throughout the collaboratives, grantee teams select strategies to implement and then complete rapid Plan-Do-Study-Act (PDSAs) cycles to test, assess results, and scale change within their own programs. Grantees meet virtually to share learnings from these cycles and review collaborative-level progress on a set of standard measures related to the topic. ACF’s goals for the collaboratives include: 1) improve home visiting practice; and 2) build grantee capacity for CQI.

The purpose of this generic information collection (GenIC) is to inform the program support and technical assistance (TA) for grantees participating in the Tribal CQIC. This information will be used by contracted TA providers and federal staff to provide technical assistance and training to participating grantees on improvement methodology. TA providers and federal staff will also provide ongoing and summative information to participating grantees on rapid cycle PDSAs and progress on measures related to Family Engagement or Early Language and Literacy. This summative, group level information facilitates shared learning and provides a yardstick for progress made towards each collaborative’s aim.

Study Design

Data collection related to grantees’ rapid PDSA tests and data collection on progress on the two topics is necessary to support individual grantees experience and the overall collaborative experience of planning, testing, and assessing strategies to promote family engagement or language and literacy.

Grantees will complete the following activities:

1. Attend two in-person learning sessions and one virtual learning session with content experts and quality improvement specialists during the year.

2. Utilize a PDSA improvement cycle approach between each learning session to test small changes in practice as they strive to implement improvement in their home visiting programs.

3. Collect and report data monthly on a set of 7 measures related to the collaborative topic.

4. Share what they learned and learn about collaborative-level progress on monthly virtual collaborative meetings.

To develop an aim for each collaborative, ACF engaged and convened expert groups to develop Early Language and Literacy and Family Engagement Topic Toolkits which contain, for each topic: 1) a theory of change about the key drivers that affect the collaborative’s aim; 2) set of suggested change strategies relating to the theory of change; and 3) set of 7 standard measures to track progress over time towards the collaborative aim.

To complete small tests of change, it is essential for grantees to plan and record their changes and to track specific measures to monitor progress and improvements at multiple points throughout the collaborative. Grantees will complete Instrument 1: PDSA Planning Tool and Instrument 2: Monthly Data Reporting Template and share these with contracted TA providers. This information will be used for two main purposes: 1) to provide technical assistance to grantees on the improvement methodology; and 2) to aggregate and facilitate grantee learning about rapid PDSA tests completed and collaborative-level progress on 7 standard measures to share during monthly virtual collaborative meetings and learning sessions.

Contracted TA providers from Tribal Home Visiting Evaluation Institute (TEI), an award to James Bell Associates, will receive the PDSA Planning Tool and Monthly Data Reporting Templates from participating grantees. The TEI team will provide individual feedback to grantees related to improvement methodology on their PDSA cycles and note trends. TEI will share trends in the PDSA cycles and will aggregate grantees’ monthly data submissions in a collaborative report. This report will be reviewed by federal staff and other TA providers providing support for the Tribal CQIC to inform planning and will be shared with grantees on the next virtual collaborative meeting. A summary level report will be provided to participating grantees and to ACF at the end of the collaborative documenting the changes that were tested and grantee progress towards the collaborative aim.

The information is meant to contribute to the body of knowledge on ACF programs. It may be relevant to other ACF programs that fund early childhood services in tribal communities, but the results are not generalizable outside of the Tribal MIECHV grantees participating in the collaboratives. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Table 1

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
PDSA Planning	Instrument 1: Plan-Do-Study-Act (PDSA) Planning Tool	Respondents: Grantees Content: The PDSA Planning Tool provides a space for grantee teams to plan and summarize each PDSA cycle run during the Tribal CQIC. The tool provides prompts for teams to articulate their improvement theory, and document all steps in their PDSA cycle. General Purpose: This tool provides a stepwise process for planning, testing, studying, and acting on improvement efforts. It supports teams in keeping their PDSA cycles organized and provides a space to summarize overall process. Grantees are also asked if they developed any resources that they would like to share with other programs in their collaborative. Purpose for program support: Technical assistance providers will review grantees progress, will offer individual grantees’ feedback on the improvement methods individually, and note common challenges or successes. ACF will summarize the types of changes grantees in the collaborative are testing. This information on group process and needs will be integrated into monthly virtual collaborative meetings or in resources for grantees.	Mode: Word template Duration: 3 hours
Monthly Data Reporting	Instrument 2: Monthly Data Reporting Template	Respondents: Grantees Content: The Monthly Data Reporting Template provides a mechanism for grantee teams to report monthly data on a standard set of measures (7 per collaborative) that are hypothesized to lead to improvements in the collaborative aim. Grantees will also be asked a set of reflection questions around successes, challenges, and to identify any TA needs. General Purpose: This template contains grantee-level data on progress on a set of standard measures related to the collaborative aim. Purpose for program support: Data for all grantees participating in the collaborative will be combined to show group progress towards the collaborative aim. Results will inform TA provided on monthly virtual collaborative meetings and will be shared for everyone to see group level progress to facilitate peer sharing. TA providers will complete data quality checks and provide TA as needed to grantees on data processes/reporting.	Mode: Excel template Duration: 2 hours

Other Data Sources and Uses of Information

Not applicable.

A3. Use of Information Technology to Reduce Burden

Information will be collected in Microsoft Word or Excel files sent by grantees to their TA providers. Grantees are already in regular contact with TA providers and can send these files easily as they are completed. We anticipate this format will provide the lowest burden to the respondent.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The topics for the collaboratives are aligned with the goals of ACF’s Tribal MIECHV Grant program. ACF attempted to identify Monthly Data Reporting Templates measures relevant to each collaborative that grantees already collect for quarterly or annual performance reporting to ACF, or as part of their participation in the Multi-Site Implementation Evaluation of Tribal Home Visiting.

• Tribal MIECHV Form 2: Grantee Performance Measures. OMB Control Number 0970-0500, expiration date 08/31/2020

• Tribal MIECHV Form 4: Quarterly Performance Form. OMB Control Number 0970-0525, expiration date 03/31/2022

• Multi-Site Implementation Evaluation of Tribal Home Visiting. OMB Control Number 0970-0521, expiration date 12/31/2021

ACF sought to reduce new data collection requests to minimize burden and increase the utility of data currently being collected. Information that ACF collects through these other information collections on a quarterly, semi-annual, or annual basis is part of grantees’ ongoing data collection. The data will be readily available and easily transferable by the grantee into Instrument 2: Monthly Data Reporting Tool. The information for the collaboratives will be reported monthly.

Table 2 summarizes data requested in the Monthly Data Reporting Template for the two Collaboratives that is collected through the previously referenced information collections.

Table 2

Collaborative	Data requested in Monthly Data Reporting Template that is Already Part of an OMB Collection
Early Language & Literacy	Measure 1: Collected on Tribal MIECHV Form 2: Measure 7 Measure 2: Collected as part of MUSE Implementation Log: Training category Measure 3: Collected on MUSE Implementation Log: Supervision category Measure 5: Collected on Tribal MIECHV Form 2: Denominator for Measure 7 Measure 7: Collected on Tribal MIECHV Form 2: Measure 1 Total: Five measures out of 7 requested are already collected part of an OMB-approved collection
Family Engagement	Measure 3: Collected on Tribal MIECHV Form 2: Measure 3 Measure 4 Collected on Tribal MIECHV Form 4: Table A1 Measure 7: Collected on Tribal MIECHV Form 2: Measure 1 Total: Three measures out of the 7 requested are already collected part of an OMB-approved collection

A5. Impact on Small Businesses

The project will include tribal human service agencies. We will only request information required for the intended use.

A6. Consequences of Less Frequent Collection

Without collecting grantees’ PDSA Planning Tools and Monthly Data Reporting Templates, ACF risks being unable to identify ongoing individual and group-level technical assistance needs or training opportunities related to implementing the collaboratives. The monthly review of collaborative-level progress is also fundamental to the collaborative model and an indicator of the overall success. Without the proposed information, grantees and TA providers might detect trends more slowly and have less information to inform ongoing tests of change, and TA respectively.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. A subsequent notice, updated with more specific information, was published on June 18, 2019, Volume 84, Number 117, page 28307, and provided a thirty-day period for public comment. During the notice and comment periods, no substantive comments were received.

Consultation with Experts Outside of the Study

We consulted with a panel of content experts, quality improvement specialists and grantee representatives in planning the Tribal CQIC. These consultations focused on the data availability for the standard sets of measures, clarity of instructions and ways to minimize burden.

The BSC process involves the inclusion of content experts and quality improvement specialists who will engage with and provide support to the sites on monthly virtual collaborative meetings. The Tribal CQIC team selected six subject matter experts* for the Tribal CQIC in June 2019.

Table 3 Consultations.

Name	Affiliation
Rebecca Riley	Native American Professional Parent Resources
Kyle Noble	Lake County Tribal Health Consortium
Daphne Colacion	Lake County Tribal Health Consortium
Rhea Pierre	Confederated Salish and Kootenai Tribes
Paula Seanez	Navajo Nation
Danielle Garcia	Wellington Group Consulting
Lisa Abramson	Inter-Tribal Council of Michigan
Mary Mackrain	Education Development Center
Tricia Finnerty	Education Development Center
Robin VanDerMoere	Michigan Public Health Institute
Angela Precht	Michigan Public Health Institute
Cathy Ayoub*	Brazelton Touchpoints Center
David Bard*	University of Oklahoma
Heidi Roibal*	Child and Family Support Bureau and Support Services with State of New Mexico
Jessica Barnes*	Michigan State University
Patti Manz*	Lehigh University
Lori Roggman*	Utah State University
Jon Korfmacher	Erickson Institute
Anne Duggan	Johns Hopkins
Nancy Margie	Office of Planning, Research, and Evaluation
Susan Landry	University of Texas Health Science Center at Houston
Deborah Mazzeo	Zero to Three
Lorraine McKelvey	University of Arkansas

A9. Tokens of Appreciation

This information collection will not include tokens of appreciation for participation.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

This effort does not include the collection of personally identifiable information. Information collected will be at the site-level. Grantee teams will complete their site PDSA Planning Tools on behalf of their teams and Monthly Data Reporting Templates on behalf of their program.

Assurances of Privacy

We will inform respondents of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As the contract specifies, the contractor will comply with all federal and departmental regulations for private information. The contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements.

Data Security and Monitoring

JBA, as specified in the Tribal Evaluation Institute contract, have an established Data Security and Monitoring plan that assesses all data security measures and monitoring procedures to ensure secure storage and transmittal of information. This plan will be updated at least annually.

Only site level information will be collected for this project. This information will be stored on a secure SharePoint site. The information collected is highly specific to the activities of the Tribal CQIC and there is no plan to disseminate it.

JBA is a subscriber to the FedRAMP ATO-holding Microsoft Online 365 Service with both Business and Enterprise licenses. JBA operates multiple SharePoint Online sites to separate data between projects and access requirements within those projects. In addition to operating with Microsoft best practices for security, SharePoint Online sites that will be used to store sensitive data will use the following additional controls that fall within JBA responsibilities for management. Additional documentation pertaining to the security of SharePoint Online can be found within their approved FedRamp package.

Access Controls

• JBA SharePoint Online and OneDrive requires users to authenticate using multi-factor authentication for all users who participate in groups with access to sensitive information.

• JBA SharePoint and OneDrive Online uses role-based access permissions to limit access to sensitive data and separate access based on assigned roles.

• Non-Administrators do not have access to modify the security policies, sharing permissions or role-based access permissions.

• Permissions granted to a user account are based on the principal of least privilege so that users are not afforded access to the system greater than their minimum requirements.

• Passwords used by user and administrative accounts require a minimum of 16 characters and must be complex, meaning that they must contain at least one number, one capital letter, and one symbol.

• Passwords are required to be updated every 90 days.
  
  

Remote Access

• JBA only permits users with a valid account access to the JBA operated SharePoint sites. Anonymous or guest access is prohibited.

• JBA SharePoint Online is hosted on the FedRamp approved Microsoft Office 365 SharePoint online service. Microsoft controls remote access to the SharePoint Platform. JBA controls user access into the JBA owned and operated sites. Security related to the transmission to and from SharePoint online is documented in the Microsoft FedRamp package available to the government at https://www.fedramp.gov.

A11. Sensitive Information ¹

The information collection does not include sensitive information.

A12. Burden

Explanation of Burden Estimates

Table 4: Estimated Annualized Burden and Costs to Respondents

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1: Plan-Do-Study-Act Planning Tool	19	10	3	570	$34.44	$19,630.80
Instrument 2: Monthly Data Reporting Template	19	10	2	380	$34.44	13,087.20
Total				950		$32,718

Respondents include one individual, usually the program coordinator, per grantee site to complete and submit the PDSA Planning Tool and the Monthly Data Reporting Template. The information will be requested monthly during the 10 month collaborative. The PDSA Planning Tool is expected to take 3 hours to complete per response and the Monthly Data Reporting Template is expected to take 2 hours to complete and submit.

Costs were estimated based on the job code is 21-1093, Social and Human Service Assistant. Wage data from 2018 is $17.22 per hours. To account for fringe benefits and overhead the rate is multiplied by two which is $34.44. The estimate of annualized cost for hour burden is $34.44.

https://www.bls.gov/oes/current/oes211093.htm

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

Table 5. Estimated Annualized Costs to the Federal Government

Cost Category	Estimated Costs
Instrument Development	$2400
Administration and Analysis of Instruments	$20,684
Monthly and Final Dissemination to ACF and Grantees	$1600
Total costs over the request period	$24,684

A15. Reasons for changes in burden

This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).

A16. Timeline

Data collection will take place following OMB approval for 10 months. Grantees will submit their templates after OMB approval on a monthly basis. During this time, TA and ACF staff team will review the templates for TA needs and summarize progress for the monthly virtual collaborative calls. Summary level information across the 10 months will be provided to grantees at the conclusion of the collaborative.

Table 6: Timeline

	Begins	Complete
Grantee Submission of Instrument 1: Plan-Do-Study-Act Planning Tool and Instrument 2: Monthly Data Reporting during CQIC Implementation	3/5/20	11/2020
Analysis and Use of Instrument 1: Plan-Do-Study-Act Planning Tool and Instrument 2: Monthly Data Reporting to Inform Program Support and Technical Assistance	3/5/20	11/2020
Dissemination to ACF and Grantees	3/5/20	1/2021

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instrument 1: Plan-Do-Study-Act Planning Tool Instructions

Instrument 1: Plan-Do-Study-Act Planning Tool

Instrument 2a: Early Language and Literacy Collaborative Monthly Data Reporting to Inform Program Support and Technical Assistance

Instrument 2b: Family Engagement Collaborative Monthly Data Reporting to Inform Program Support and Technical Assistance

1

13