Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Feedback from Head Start (HS) American Indian and Alaska Native (AIAN) Self-Review
Formative Data Collections for Program Support
0970 – 0531
Supporting Statement
Part A
March 2022
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer: Wendy DeCourcey
Part A
Executive Summary
Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for Program Support (0970-0531).
Description of Request: This request is to gather feedback and insights from 16 Head Start (HS) American Indian and Alaskan Native (AIAN) programs who implemented the HS AIAN Classroom Assessment Scoring System (CLASS) self-review process. The team plans to conduct group discussions or individual interviews with the 48 program leaders and 30 initial observers who implemented the AIAN self-review process in Spring 2022. Questions of interest include a) additional professional development and systems support needed prior to implementing, b) challenges and lessons learned with implementation, and c) recommendations for future implementation. The information is intended to inform the Office of Head Start’s guidance for possible future self-reviews in AIAN HS programs and will not be generalized to a broader population. We do not intend for this information to be used as the principal basis for public policy decisions.
Time Sensitivity: OHS wants to finalize these conversations by August 2022, to inform the OHS guidance over the 2022-2023 program year.
A1. Necessity for Collection
The Classroom Assessment Scoring System (CLASS) is an observational measure of early childhood classroom and teaching quality that is usually implemented by Office of Head Start (OHS) observers who are independent from the observed programs. As part of a self-review pilot, 16 Head Start (HS) American Indian and Alaskan Native (AIAN) programs trained and certified staff from within their programs to complete the classroom observations, to potentially increase the cultural and community validity of the observations. From March 2022 to approximately July of 2022, the group of 16 HS AIAN programs will implement the CLASS using the observers trained and certified staff from their respective AIAN programs (i.e. staff from the program will be observing classrooms within their own program; a.k.a. program ‘self-review’).
The proposed discussions/interviews with the self-review implementers will provide information that is necessary to inform OHS the development of appropriate guidance and professional development, if OHS pursues any additional self-reviews in the 2022-2023 program year. The information will not address validity, benefits or accuracy of the observations; it will address implementation experiences and recommendations of the programs.
A2. Purpose
Purpose and Use
This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):
Delivery of targeted assistance related to program implementation or the development or refinement of program processes.
Planning for provision of programmatic training or technical assistance (T/TA).
Obtaining grantee or other stakeholder input on the development of program performance measures.
The purpose of the proposed interviews and group discussions is to inform OHS guidance and targeted assistance for self-review processes. OHS does not have sufficient information about this model of self-review for the use of CLASS. If OHS decides to continue with the self-review approach, the feedback from the program directors and staff on the self-review approach and the provided training and technical assistance will inform OHS guidance, training, and technical assistance for the upcoming program year (2022-2023).
Themes from the interviews and discussions will be combined in one report for OHS, and a brief summarizing the themes will be provided to the participating programs. Information may inform future resources related to the self-review process that may be made public.
The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker and is not expected to meet the threshold of influential or highly influential scientific information.
Study Design/Information Collection Procedures and Processes
On a rolling basis from March 2022 to approximately July 2022, the sixteen programs plan to implement the CLASS self-review. As soon as possible after this implementation, the team will schedule interviews or group discussions.
We will interview the program directors and relevant leadership from each of the sixteen programs. Relevant leadership will be identified by the program directors and may include Tribal and administrative leadership for their program.
We will schedule small group discussions (2 to 8 people each) for the program CLASS observers. If an individual program observer cannot be scheduled for a group discussion, they would be offered the opportunity to participate in a one-on-one interview.
This qualitative approach is appropriate for gathering community input and feedback on the self-review process, giving each participant an opportunity to provide insights from their experience.
The results are not intended to be generalizable or representative of other groups.
Data Collection Activity |
Instruments |
Respondent, Content, Purpose of Collection |
Mode and Duration |
AIAN HS Director and Leadership Interviews |
Instrument 1: Directors and Leadership Interview |
Respondents: 16 directors with up to two additional leaders per program; Total: 48
Content: Identifying professional development needs, feedback on professional development provided; challenges and lessons learned from implementing; concerns, recommendations and insights for future implementers.
Purpose: Gather program-level and leadership impressions of the training, technical assistance, and quality and features of implementation of the self-review process; Gather recommendations and insights for possible future efforts
|
Mode: Interview
Duration: 45 minutes |
AIAN Staff CLASS Observers Group Discussion Questions |
Instrument 2: HS AIAN Observer Group Discussion Questions |
Respondents: 30 Observers
Content: Identifying professional development needs, feedback on professional development provided; challenges and lessons learned from implementing; concerns, recommendations and insights for future implementers.
Purpose: Gather observer-level impressions of the training, technical assistance, and quality and features of implementation of the self-review process; Gather recommendations and insights for possible future efforts. |
Mode: Group discussion/Interview
Duration: 75 minutes |
A3. Use of Information Technology to Reduce Burden
Interviews and group discussions will occur by phone and by phone conferencing tools, when respondents have access to internet.
A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency
As this information collection is gathering feedback about a novel approach to HS AIAN program observation, there is not existing data that could inform OHS guidance for 2022-2023.
A5. Impact on Small Businesses
Respondents will be employees of community organizations/Tribal government organizations who administer HS AIAN programs. Scheduling will occur at the convenience of the respondents; if a respondent is not available for a group discussion time, they will be offered a one-on-one interview option.
A6. Consequences of Less Frequent Collection
This is a one-time data collection.
A7. Now subsumed under 2(b) above and 10 (below)
A8. Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published two notices in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. The first notice was published on October 13, 2020, Volume 85, Number 198, page 64480, and provided a sixty-day period for public comment. The second notice published on December 28, 2020, Volume 85, Number 248, page 84343, and provided a thirty-day period for public comment. ACF did not receive any substantive comments.
Feedback on the proposed instruments was gathered from staff providing professional development to the HS AIAN CLASS observers.
A9. Tokens of Appreciation
No tokens of appreciation will be provided to participants.
A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing
Personally Identifiable Information
No personally identifiable information will be collected.
Assurances of Privacy
Interviews and group discussions will be recorded, and respondents will be informed of this prior to starting recording. If participants decline to be recorded, note taking will be used to gather information.
Participants will be informed that their names and program names will not be included in the information provided to OHS and that their funding will not be impacted based on their responses. Participants will be informed that quotes from the conversations may be used in reports to OHS to represent themes represented in the interviews and group discussions. Quotes will not be used that would allow for identification of programs or staff.
Data Security and Monitoring
The research teamwill be the only individuals with access to the data, using password protection. All members of the research team have completed IRB training, and they will receive instruction on be instructed by the lead researcher regarding the data security plan, access methods, points of risk for disclosure, and required plan for reporting any potential breaches. The leader of the research team shall securely generate and manage passwords to prevent unauthorized access to information.
Participant, program, and community names will not be used during the sessions, and video recordings of the sessions will only be stored on a password protected external drive in a locked security box until the project is completed. This file will only be used if clarification is needed while finalizing the transcriptions, and it will only be used on a password protected and encrypted computer that is disconnected from the Internet. The video files will be destroyed upon completion of the project.
Audio files of recordings will not contain any participant, program, or community identifiers, and these files will be used to develop transcriptions while working on a password protected and encrypted computer. Once transcription is complete, the audio files will be removed from the computer and stored on the password protected external hard drive in a locked security box for the duration of the project. The audio files will be destroyed upon completion of the project.
Transcriptions will not contain any identifiers, and these files will be stored on a password protected and encrypted computer. Transcriptions will be stored for 3 years after the project is completed.
Data (e.g., transcript, coding) will be kept in electronic format only. All reports from the data will be presenting qualitative findings (e.g. summary themes) and will be reviewed to ensure that individuals and programs cannot be identified.
A11. Sensitive Information 1
No sensitive information will be collected.
A12. Burden
The Director and Leadership Interview (Instrument 1) is expected to take an average of 45 minutes per interview. We will encourage the program directors to also invite a Tribal representative and/or the administrative program staff that is most knowledgeable about the self-review implementation. This means each of the 16 program interviews may include up to 3 people (total possible 48).
The AIAN Program Observer Group Discussions (Instrument 2) are expected to take an average of 1 hour and 15 min. There are a total of 30 observers across the 16 programs.
Estimated Annualized Cost to Respondents
The wage rates are from the May 2020 National Occupational Employment and Wage Estimates from the U.S. Bureau of Labor Statistics. For Program Directors we utilized 11-9030 Education and Childcare Administrators rates and for AIAN Observers we utilized 11-9031 Education and Childcare Administrators, Preschool and Daycare.
Instrument |
No. of Respondents (total over request period) |
No. of Responses per Respondent (total over request period) |
Avg. Burden per Response (in hours) |
Total Burden (in hours) |
Average Hourly Wage Rate |
Total Annual Respondent Cost |
Instrument 1: Director Interview |
48 |
1 |
.75 |
36 |
$47.25 |
$1,701.00 |
Instrument 2: AIAN Observers Discussion Questions |
30 |
1 |
1.25 |
38 |
$25.81 |
$980.78 |
Totals |
78 |
1 |
|
74 |
|
$2,681.78 |
A13. Costs
There are no additional costs to respondents.
A14. Estimated Annualized Costs to the Federal Government
Field work and development of internal reporting is estimated to use .15 FTE for the research team leader. It is estimated it will take the assistants .50 FTE total. With fringe and benefits, and administrative costs, the total costs for information collection is $31,171.48.
A15. Reasons for changes in burden
This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).
A16. Timeline
The HS AIAN CLASS self-review process will be implemented by participating programs from March 2022 to July 2022. Interviews with program directors and group discussions with observers will occur on a rolling basis over the course of this period. The internal report will be completed August 2022. It is expected that a short brief summarizing themes will be provided to participating programs in August 2022.
A17. Exceptions
No exceptions are necessary for this information collection.
Attachments
Instrument 1: Director and Leadership Interview
Instrument 2: HS AIAN Observers Group Discussion Questions
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | DeCourcey, Wendy (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2022-05-04 |