School District State
Kindergarten Entry Assessment Case Studies
Test Administrator Interview Protocol
Introduction and Purpose of Interview
Thank you for your willingness to participate in this interview. We are talking with state and district administrators, principals, kindergarten teachers, preschool administrators, and professional development staff in your state about the Kindergarten Entry Assessment, including the development and implementation of the assessment and how the data are used. We’d like to learn what is working well in addition to the challenges and how people are overcoming them. We believe information about your experiences will be valuable for other states and districts interested in implementing KEAs. Your experience also can help inform federal policy and help the U.S. Department of Education build technical assistance that is useful for all states tackling this important effort. A public report will be available at the end of the study to promote sharing of lessons learned but it will not identify any individual districts, schools, or people.
This interview will take approximately 45 minutes. We might ask questions that do not apply to your state, district, or schools. If so, please just let us know that this is the case.
Consent Process
Have each respondent read and sign the consent form. Ask permission to audio-record the conversation. ______________________________________________________________________________
We understand that your state has [selected or modified XX measure or developed its own measure XX] as your kindergarten entry assessment. First, we want to learn a few details about the [local KEA name] itself and the nuts and bolts of administering it.
Please describe the process for administering the [local KEA name] in your school. What is your role in the process?
When is the [local KEA name] administered (i.e., during what time frame of the school year)? How frequently is it administered? Is repeated use throughout the year an option?
What types of evidence are collected as part of the assessment? For example, are knowledge, skills, and abilities assessed through observation, checklists direct assessments, and/or parent reports? [Probe for each component of the KEA.]
Who administers or provides information about the student for the [local KEA name]?
How are families involved in providing information?
To what degree is technology used to support the administration or scoring of the [local KEA name]?
What did the state, district, or your school leadership communicate to you about the purpose of the [local KEA name] and what it is trying to assess? Who provided that information?
✪ Could you provide me with a sample of the information provided?
What training, supports, or resources did you receive on the administration of [local KEA name]? Please describe the various types of training. [Probe: What training, supports, or resources do you receive for using the data from [local KEA name]?
After getting a description, ask:
So would you say your training model uses [read the following list]? (Mark all that apply.)
Workshops
Webinars
Coaching
Online modules (PPTs on websites)
Resource documents such as administration manuals
Other (specify): ________________________________________
Who provides the training, supports, or resources (the state, the district, a contractor)?
How well prepared did you feel to administer the KEA?
Do individuals need to be certified before administering the assessment?
What other training, supports, or resources would have been helpful?
How does your principal, the district, or others monitor and evaluate whether the [local KEA name] is being implemented according to the state's recommended guidelines?
Are there ways implementation guidance from the state or district could be improved? If yes, how?
Can you describe the how your school submits the resulting data from the [local KEA name] to the district or state?
Who is involved in submitting the data?
In what form are data submitted from the school to the district or directly to the state?
Where are the data entered (e.g., test publisher’s website, statewide longitudinal data system, student permanent record/cumulative file)?
What is the data submission timeline?
Finding one assessment that can accurately identify young children’s development status across multiple learning domains is challenging. Ensuring that the measure is also linguistically appropriate for children whose primary language is not English adds another layer of complication, as does the need to accurately assess the strengths and needs of young children with developmental delays or disabilities.
Within this context, we’d like to ask about your thoughts concerning the use of the [local KEA name] with these particular populations.
To what degree do you believe the [local KEA name] accurately assesses the knowledge, skills, and abilities of English learners?
In what ways were you prepared/trained to use the [local KEA name] with English learners?
To what degree do you believe the [local KEA name] distinguishes between language differences and language delays?
What language support is there to conduct the assessment (e.g., translation, nonverbal stimuli, observations done by bilingual staff or adults)?
In what ways are expectations or interpretations of the findings adapted for English learners (who are learning their home language while also acquiring a second language)?
What additional training, supports, or resources would have been helpful in using the [local KEA name] with English learners?
To what degree do you believe the [local KEA name] accurately assesses the knowledge, skills, and abilities of children with developmental delays or disabilities?
In what ways were you prepared/trained to use the [local KEA name] with children with developmental delays or disabilities?
What accommodations are made and for whom? (For example, are children given augmentative or alternative communication or written systems, visual or sensory support, assistive equipment or devices, input is collected from special education staff?)
Are there children with disabilities for whom the [local KEA name] is not appropriate? If yes,
What, if alternate assessments are used with these children?
In what ways are the results used for screening and referral or IEP purposes?
What additional supports would have been helpful in using the [local KEA name] with children with developmental delays or disabilities?
The [local KEA name] may be used in conjunction with existing district and school assessments.
If this is the case in your district, could you describe how the [local KEA name] and the existing assessments work together (probe for existing K entry assessments)?
What are the main reasons for using the other K assessments in addition to [local KEA name]?
What have been the most important factors in supporting the implementation of [local KEA name]?
What have been the biggest challenges with implementing [local KEA name]?
(Use probes below.)
Administrator buy-in, If so, how?
The [local KEA name] instrument itself, If so, how?
Lack of training, If so, how?
Not enough or unclear administration directions, If so, how?
Political challenges, If so, how?
Timing challenges, If so, how?
Lack of shared vision among stakeholders about the purpose and use of the [local KEA name] or communication about its purpose and use with districts and schools, If so, how?
Technology difficulties (e.g., website for scoring and data entry not ready or functioning correctly), If so how?
Other (specify): ________________________________________
What solutions have you identified for addressing those challenges?
Now let’s move to how [local KEA name] results are used to inform policy and practice. Are you involved with the use of data from the [local KEA name]? If no, skip to question 21.
As someone who administers the [local KEA name], what information do you receive on [local KEA name] results? [Probe whether data are aggregated at class level, or broken down by students.]
What training, support, or resources do you receive to help use [local KEA name] results to inform your work with students?
Which training, supports, or resources were most useful?
What other training, supports, or resources would have been useful?
How do you, as someone who administers the [local KEA name], use [local KEA name] results?
After getting a description, ask: Do you use results to (mark all that apply):
Identify students in need of extra services or further assessment? If so, how?
Identify specific skills where students need improvement to inform teaching of individual students or groups of students or curriculum planning? If so, how?
Engage families in knowing children’s strengths and needs and ways to support learning at home? If so, how?
Promote dialogue between early childhood and elementary educators about ways to increase children’s readiness (e.g., align expectations and/or instruction)? If so, how?
Help teachers understand where children should be on a comprehensive set of domains and how to assess children's progress? If so, how?
Benchmark learning to track progress in kindergarten throughout the year, If so, how?
Make decisions about:
Curriculum? If so, how?
K-12 policies and practices? (e.g., School-wide planning for Kindergarten) If so, how?
After thinking about how you use the data, what information from the [local KEA name] do you think is most useful?
What other information would have been useful?
What information about [local KEA name] results do parents receive?
How is information shared? Do you use the [local KEA name] results in parent-teacher conferences?
Are there additional supports or resources you would want to be able to share results with families?
✪ Could you provide me with a sample report?
Describe any policies regarding how [local KEA name] results are not to be used, if there are any.
Listen for but don’t say the uses below, let the respondent say them. (Check all that apply):
Determine promotion or prevent kindergarten entry in any way
Principal evaluation, teacher evaluation
Evaluation of preschool programs
Screenings of children
Other (specify): ____________________________________________
How are these policies communicated to principals and teachers?
How are these policies enforced? Please give me a specific example.
What have been the most important factors in supporting the use of [local KEA name] results to improve students’ educational experience?
What have been the biggest challenges with trying to use the [local KEA name] results?
Use probes below.)
Lack of training, If so, how?
Reports hard to interpret, If so, how?
Lack of alignment with standards, If so, how?
Timing of when results are shared, If so, how?
Lack of perceived reliability, If so, how?
Lack of buy-in, If so how?
Other (specify): ____________________________________________
What solutions have you identified for addressing those challenges?
This is our final set of questions! In closing we’d like you to reflect on how your district and school administered and used [local KEA name] data and try to offer lessons learned.
Looking back over the [local KEA name] implementation, how well do you think it has gone? Has the KEA served the purposes it was intended to serve? [Probe: refer back to the general purpose they say has been communicated.]
What concerns did you have going into implementation? Do these concerns still remain now that you have begun implementing the [local KEA name]? Do you have new concerns that now that you are implementing the [local KEA name]?
What parts of implementation have gone especially well?
How could implementation be improved?
Are there any other lessons learned or recommendations you can share with other districts and schools about [local KEA name] selection, implementation, and use of [local KEA name] results to inform policy and practice?
Thank you for your time and thoughtful responses.
Appendix
H: Test Administrator Interview Protocol Date
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Shari Golan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |