TA Recipient Survey

Appendix C: Primary Data Collection Instruments

Exhibit C-1 TA Recipient Survey

Notes for reviewers: This survey protocol will be administered to Technical Assistance recipients identified as the point of contact by Comprehensive Center Directors. Surveys are expected to take up to 20 minutes per project and will focus on details of projects the Centers conducted, services provided, and capacity developed.

• Purple font indicates programming notes.

• Green font indicates internal notes.

• Blue boxes are instructions or transitions that the interviewer will read to respondents.

• Radio buttons indicate only one answer should be selected.

• Checkboxes indicate the interviewer should check all that apply.

Comprehensive Center Technical Assistance (TA) Recipient Survey

The U.S. Department of Education’s Comprehensive Center Network (CC Network) is intended to help State Education Agencies (SEAs) and Local Education Agencies (LEAs) improve educational outcomes for all students, close achievement gaps, and improve the quality of instruction. The U.S. Department of Education is sponsoring a study to examine whether the Comprehensive Centers helped to meet these goals and what challenges remain to capacity building. Abt Associates is carrying out the study on behalf of the Department. You can help the U.S. Department of Education improve supports and services offered by the Comprehensive Centers by participating in this survey.

We are asking you to complete a short survey about your overall experience working with the Comprehensive Centers. The survey should take about 20 minutes to complete for each Comprehensive Center project you lead within your agency.

We know your time is valuable, and we thank you in advance for the time spent completing this survey! Your responses are vital to the success of this important U.S. Department of Education study.

If you have any questions about this survey, contact us at [email protected]

Paperwork Reduction Act of 1995

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. Public reporting burden for this collection of information is estimated to average a total of 20 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. If you have comments or concerns regarding the status of your individual submission of this survey, the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, please contact the National Center for Education Evaluation/Institute of Education Sciences U.S. Department of Education, 400 Maryland Ave., SW, Washington, DC 20210-4537 or email [email protected] and reference the OMB Control Number XXXX-XXXX. (Note: Please do not return the completed survey to this address.)

Notice of Confidentiality

Information collected for this study comes under the confidentiality and data protection requirements of the Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Per the policies and procedures required by the Education Sciences Reform Act of 2002, Title I, Part E, Section 183, responses to this data collection will be used only for statistical purposes. The reports prepared for the study will summarize findings across the sample and will not associate responses with a specific individual. Any willful disclosure of such information for non-statistical purposes, except as required by law, is a class E felony.

Welcome to the survey about the Comprehensive Center’s Technical Assistance! Since fall 2019, your project team may have received training, technical assistance, coaching, tools, or other resources from staff at one or more Comprehensive Centers, such as the Regional Comprehensive Center or the National Center. For example, staff from [Home Regional Center name], such as [use Home Center project contact name], may have recently worked with you on one or more projects. This survey asks you questions about your experiences with the Comprehensive Centers.

Throughout this survey:

• We will refer to the [Home Regional Center Name] as the “Center.”

• We will refer to [SEA/REA/LEA name] as “your agency.”

• We will refer to you and your colleagues from your agency who worked on the same Center project as a “project team.”

Please feel free to consult with a member of your project team if you need assistance responding to a particular survey question. In addition, please feel free to reference relevant project documents or communications as needed when responding to survey questions.

Introduction

1. Did your project team receive training, technical assistance, coaching, tools, or other resources from the Center on the following project(s)? [Internal note: project specific]

[Programming note: “Project Name” will be prefilled from the name listed in the ASP]

Project names	1. Did you/your project team receive training, technical assistance, coaching, tools, other resources from the Center for this project? Select one response in each row
	Yes	No
Project name 1
Project name 2
Project name 3

[Programming note: If Q1=no for all projects on which respondent is the primary point of contact, direct respondent to Thank you/Exit page. Else, proceed to next section]

Agency Needs and Priorities

Before we ask about the project you worked on, we have some general questions about your work with the Comprehensive Centers.

2. Which of the following topic areas are most related to your agency’s top current educational priorities? Select up to three topic areas.

[Note: list of problems derived from the Annual Service Plans (ASPS)]

[Programming note: R may select up to three options. Separate these examples from the core question by putting them in a table and make the examples fit into the right-hand column]

• Accountability & Assessment (Achievement Gaps, Assessment, Data Use, Formative Assessment, Student Outcome Measures)

• College & Career Readiness (Career & Technical Education, Dropout Prevention, Dual Enrollment/Dual Credit, Graduation, Pre-college Preparation, Work-based Learning)

• Curriculum & Instruction (Curriculum & Development, Resource Use, Standards Alignment, Literacy, STEM + Computer Science)

• Diverse Learners (Students of Color (American Indian/Alaskan Native, Asian, Black, Hispanic), LGBTQ+ Students, English Learners, Low Income Students, Students with Disabilities, Students Experiencing Homelessness, Students in Migrant Families, or Students in Foster Care)

• Early Learning (Kindergarten Readiness, Head Start)

• Educational Equity (Inequities in School Funding, Inequities in Access to High-Quality Instruction, Inequities in Access to Support Services, Catch-up strategies)

• Planning, Evaluation & Management (Internal Communication, Compliance Monitoring, Family & Community Engagement, Financial Management, Needs Assessment, Program Development & Implementation, Program Evaluation, Stakeholder Engagement, Strategic Planning)

• Policy & Legislation (Local or State Education Agency compliance, Every Student Succeeds Act, Policy Development, State Education Agencies, Other Federal Policy)

• Return to School (Strategies to Support Physical Distancing, Addressing Learning Loss During COVID-19, Supporting Students’ Emotional Needs During COVID-19, Communicating Reopening Policies to Families)

• Schools & Classrooms (Instructional Technology, School Choice, School Climate, School Improvement)

• Social-Emotional & Behavioral Learning (Social-Emotional Learning,

Holistic Student Support, Wrap-around Services, Growth Mindset)

• Teachers & Leaders (Certification & Licensure, Educator Evaluation, Educator Preparation, Leadership, Professional Development for Teachers or Leaders, Recruitment & Retention)

• Teaching & Learning (Culturally Responsive Practices, Evidence-Based Practices, Individualized Learning, Multi-Tiered Systems of Support, Online/Distance Learning, Rural Education, Special Education, Student Engagement)

• Other, specify _____________________________

3. In the previous question, you indicated the topic areas related to your agency’s top educational priorities.

[Programming note: List up to three topic areas selected in Q2.]

Topic Area	3a. Have you or someone else from your agency requested services from the Center in this area? Select one response in each row.			3b. Has your agency received services from the Center in this area? Select one response in each row.
	Yes (1)	No (0)	Don’t Know (99)	Yes (1)	No (0)
3a_1. Area X
3a_2. Area Y
3a_3. Area Z

Your Agency’s Work with the Center

We would now like to ask about the project(s) you and your project team may have worked on with the Center. If you worked on multiple projects with the Center, we will ask about each project separately.

[Programming note: “Project Name” will be prefilled from the name listed in the ASP. If Q1=yes for the project, show Q4 – Q8 for each project]

4. The following questions ask about [Project Name] (you or your agency might have a slightly different name for this project). Approximately when did [project name] begin? [Internal note: project specific]

MM / YYYY

• Don’t Know

[Programming note: allow dropdown menu for months and years. Restrict years to 2019-2022]

5. Would you describe [project name] as:

Select one response. [Internal note: project specific]

• Completed: project was completed and is no longer active.

• Ahead of schedule: major milestones have been completed sooner than anticipated with limited modifications

• On target: major milestones have been successfully completed on the anticipated timeline with limited modifications.

• Approaching: major milestones have been completed with moderate modifications.

• Delayed/deferred: major milestones have been delayed or needed major modifications.

• Discontinued: the project has been discontinued due to changes or significant constraints.

[Programming note: If Q5=Completed or Discontinued, go to 5a. Else, proceed to Q6]

5a. Approximately when did [project name] end?

MM / YYYY

[Programming note: allow dropdown menu for months and years. Restrict years to 2019-2022]

• Don’t Know

6. On [project name], did the Center work directly with your project team on the following services? [Internal note: project specific] Select all that apply.

• Conduct a needs assessment

• Develop a logic model (also called a “theory of action” or “theory of change”)

• Select practices, interventions, or efforts

• Plan the implementation of practices, interventions, or efforts

• Support the implementation of practices, interventions, or efforts

• Evaluate the results of a practice, intervention, or effort

• Other, specify _________________________

7. On average, about how many full-time equivalent (FTE) project team members from your agency, including yourself, were required to work on [project name] at any given point in time? [Internal note: project specific]

__ # FTE project team members (round to the nearest tenth place) [Programming note: allow numeric values including decimals]

8. On average, about how often did you or members of your team attend events, trainings, or meetings that were organized by the Comprehensive Center Network and were related to [project name]?

(The Comprehensive Center Network includes the Center you worked with most directly on this [project name], and the other Regional Centers and the National Center.)

[Internal note: project specific]

• More than once per week

• Once per week

• 2-3 times per month

• Once per month

• Once every 2-3 months

• Once every 6 months

• Once per year

Reflecting on Your Work with the Center

Now please think back about all your project team’s work with the Center since fall 2019. These next questions ask how the Center may have helped you and your project team, and whether you found Center activities to be helpful.

9a. To what extent has the Center helped you develop or improve your own knowledge and skills in:	To a great extent	To a slight extent	To a moderate extent	Not at all	Did not receive assistance in this area
Developing a needs assessment (H, LM1) [programming note: display this item, if Q6_1=1]
Developing a logic model (H, LM2) [programming note: display this item, if Q6_2=1]
Reviewing or summarizing evidence to improve understanding of topics relevant to my agency’s educational needs (H, LM3)
Selecting practices, interventions, or efforts to address my agency’s educational needs (H, LM3) [programming note: display this item, if Q6_3=1]
Developing a plan to implement practices, interventions, or efforts to address educational needs (H, LM4) [programming note: display this item, if Q6_4=1]
Supporting the implementation of practices, interventions, or efforts to address educational needs (H, LM5) [programming note: display this item, if Q6_5=1]
Evaluating the results of practices, interventions, or s efforts to address educational needs (H, LM1/LM6)
Communicating rationale for evidence-based policy, intervention, or statewide efforts selected (H, LM6)

9b. Would you or your project team be able to [pipe in capacity dimension(s) from Q9a = “to a great extent”, “to a moderate extent”, or “to a slight extent”; repeat for each activities] activity without Center assistance in the future?

• Yes

• No

9c. Have you shared information about [pipe in capacity dimension(s) from Q9a = “to a great extent”, “to a moderate extent”, or “to a slight extent”; repeat for each activities] with someone who wasn’t part of the project team?

• Yes

• No

10a. To what extent has the Center helped your project team(s) develop or improve its ability in the following areas? “Project team” refers to you and your colleagues from your agency who worked on the same Center project.		To a great extent		To a slight extent		To a moderate extent		Not at all	Did not receive assistance in this area
Develop a shared understanding around project team’s visions and goals (O, LM1)
Increase human resources, staff hours, or funding devoted to addressing shared goals or educational needs (R, LM2)
Access tools, materials, guidance, experts, or resources devoted to addressing shared goals or educational needs (R, LM2)
Collaborate across different teams or units within the agency to address educational needs (O, PM5)
Coordinate services with LEAs to address educational needs (P, LM5)
Develop guidance, rules, regulations, or initiatives based on evidence (P, LM5)
Coordinate services with policymakers to address educational needs (P, LM5)
Use data to continuously improve processes, operations, or decision- making (O, LM6)

10b. Would your project team(s) be able to do this [pipe in capacity dimension in column 1 in boldfaced text] on their own without Center assistance in the future? [programming note: show only if Q10a = “to a great extent”, “to a moderate extent”, or “to a slight extent”]

• Yes

• No

• Don’t know

11. To what extent did the assistance you received from the Center enhance your project team’s knowledge and skills related to your project’s work?

• To a great extent

• To a moderate extent

• To a slight extent

• Not at all

11. Since fall 2019, what challenges, if any, have you experienced while working with the Center? Select all that apply.

• Unclear understanding of what the Center can offer

• Lack of alignment between agency’s needs and Center assistance

• Shifting priorities at the agency

• Resources, tools, and assistance activities not provided in a timely manner

• Difficulty coordinating project activities with other offices/divisions within the agency

• Staff turnover at agency

• Staff turnover at the Center

• Existing skills or content expertise of agency staff

• Existing skills or content expertise of Center staff

• Limited agency staff time to participate in Center activities

• Difficulty engaging policymakers or other key stakeholders

• Difficulty adapting project in response to COVID

• Other, please specify_____

• No challenges

11. Which of the following organizations did you or your project team(s) work with on a Center project?

Organizations	13a. Worked with this organization
1. Regional Educational Laboratories (RELs)	Yes No Don’t know
2. Other Regional Comprehensive Center	Yes No Don’t know
3. National Comprehensive Center	Yes No Don’t know
4. Equity Assistance Center	Yes No Don’t know
5. What Works Clearinghouse or another organization that rates evidence	Yes No Don’t know

COVID-19 and Your Work with the Center

For the following questions, please think about your project team’s work with the Center since the start of the COVID-19 pandemic.

[Programming note: Ask Q14 for each project for which Q1=yes]

11. How did the COVID-19 pandemic affect your work with the Center on [project name]? Select one response. [Internal note: project specific]

• The project was substantially changed (such as the project took a new direction)

• The project started specifically to address needs related to COVID-19

• The project was cancelled or delayed

• The project had minor changes (such as moved to virtual meetings only or made minor scheduling adjustments) or was unaffected by the pandemic

• Not applicable: the project did not exist before COVID-19

For this final set of questions, please think back about all your work with the Center since the start of the COVID-19 pandemic.

11. Did the Center provide your project team(s) with assistance in areas related to COVID-19?

• Yes

• No

[Programming note: Ask Q16 only if Q15=Yes]

Area of Need	16a. The Center provided my project team(s) with assistance in this area	16b. The Center was able to meet my project team’s (teams’) needs in this area [Programming note: show if Q16a = Yes]
1. Addressing achievement gaps	Yes No	Strongly agree Agree Disagree Strongly disagree
2.Technology access for students, including strategies to support remote learning for students with limited internet access	Yes No	Strongly agree Agree Disagree Strongly disagree
3.Teaching strategies for remote learning	Yes No	Strongly agree Agree Disagree Strongly disagree
4. Specific strategies to support continuity of learning	Yes No	Strongly agree Agree Disagree Strongly disagree
5. Supporting students’ social, emotional, and mental health needs	Yes No	Strongly agree Agree Disagree Strongly disagree
6. Plans to re-open schools, including strategies to support physical distancing and other health- or safety-related procedures	Yes No	Strongly agree Agree Disagree Strongly disagree
7. Engaging students and families	Yes No	Strongly agree Agree Disagree Strongly disagree
8. Specific ways to support diverse learners	Yes No (Programming note: If yes, display the following text and response questions) Select all that apply: English learners Students with disabilities Students experiencing homelessness Students in migrant families Students in foster care LGBTQ+ students American Indian/ Alaskan Native students Asian students Black students Hispanic students	Strongly agree Agree Disagree Strongly disagree
9. Strategies for developing attendance policies	Yes No	Strongly agree Agree Disagree Strongly disagree
10. Strategies for developing graduation or grading policies	Yes No	Strongly agree Agree Disagree Strongly disagree
11. Guidance on the use of COVID recovery funding	Yes No	Strongly agree Agree Disagree Strongly disagree
12. Guidance for meal delivery services	Yes No	Strongly agree Agree Disagree Strongly disagree
13. Other, specify	Yes No	Strongly agree Agree Disagree Strongly disagree

17. Overall, how responsive was the Center to your project team’s (teams’) needs related to COVID-19?

• Very responsive

• Moderately responsive

• Slightly responsive

• Not at all responsive

• Did not work with Center on needs related to COVID-19

Thank you for completing the survey! If you have any additional questions, please do not hesitate to contact a member of the study team at: [email protected]. If you are willing to answer follow-up questions, please click here [insert check box]

Abt Associates Inc. Appendix C: Primary Data Collection Instruments ▌pg. C-1