Project Director's Collections

Appendix I
Project Director’s Interview, Project Characteristics Form, and Evaluation
Description Form

Project Director Interview
(90 minutes)
May 9, 2007
Paperwork Burden Statement
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless such collection displays a valid OMB control number. The valid OMB control number for this
information collection is 1875-NEW. The time required to complete this information collection is estimated to
average 90 minutes per response, including the time to review instructions, search existing data resources, gather
the data needed, and complete and review the information collection. If you have any comments concerning
the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S.
Department of Education, Washington, D.C. 20202-4700. If you have comments or concerns regarding the
status of your individual submission of this form, write directly to: Policy and Program Studies Service, U.S.
Department of Education, 400 Maryland Avenue, SW, Room 6W259, Washington, D.C. 20202.

Introduction (2 minutes)
Thank you for taking the time to talk with me. My name is ________ and I work for
Child Trends, an independent research organization. We are conducting a study for the
U.S. Department of Education to understand better the implementation and outcomes of
the Early Childhood Educator Professional Development Program (ECEPD) projects. As
part of this study, we are calling all of the Project Directors of the ECEPD grantees in the
2003 – 2005 cohorts to learn more about the project structure and activities, as well as to
identify successes, challenges, and promising practices that have emerged from the
projects.
I want to assure you that the information you provide today will be kept anonymous.
Specific comments will not be attributed to individuals or programs, nor will we share
your comments with anybody outside the research team. However, given the small
number of sites (18 in total), it is possible that some of the information summarized based
on the interviews will be recognized by those who know the sites. The information we
gather from the ECEPD programs across the country will be used to write a report for the
U.S. Department of Education. Our report will describe the range of program and
evaluation activities being implemented through the ECEPD grants as well as the
outcomes across programs.
We would like to record our interview with you so that we can transcribe your responses
and capture accurately the information you share with us. Do we have your permission to
record this interview? [Interviewer: Confirm respondent’s permission]
Do you have any questions before we get started?
Project Goals (5 minutes)

ECEPD Director Interview Protocol

1

1. In preparation for this call, we have thoroughly reviewed your project’s materials,
including your grant application and annual performance report(s). So, we are
aware of your program’s stated goals and activities. However, we would like to
start by asking what you consider to be the most important goals you are seeking
to accomplish through this project?
Rationale: Offers an Icebreaker; re-iterates Child Trends’ familiarity with the
grantees project
Project Structure (10 minutes)
2. One of the goals in our study of the ECEPD programs is to understand better the
state and local context that may have influenced the formation of the ECEPD
partnerships and the eventual outcomes of the project. Could you please describe
the circumstances and local context that led to the formation of your ECEPD
partnership?
Probe: What existing programs, policies, or initiative in your state or locality have
influenced your program?
Rationale: Provides insight into the state and local policy context component of
ECEPD conceptual model
3. Let’s talk now about the organizations working together on this project. We
know that partnerships are often a blend of partners formally identified in the
application and other community partners who may not be officially associated
with the grant. We are interested in understanding the roles that different partners
play in your project and how they contribute to the overall functioning of the
project.
Probe: [Note to interviewer: please probe for which partner or partners work on
different aspects of the project such as program or coursework development, selection
of faculty, outreach to program participants, program monitoring, program
management, evaluation, etc.?]
Rationale: Provides insight into the structure and functioning of ECEPD partnerships
with a specific focus on how projects are allocating various roles and responsibilities.
Who does what? Are number of partners and definition of roles related to
functioning?
4. What, if anything, would you like to see change about the type and level of
involvement of any of your ECEPD partners as you move forward in
implementing the project? [Alternative phrasing for programs that are finished:
What, if anything, would you like to have seen changed about the type and level
of involvement of any of your ECEPD partners as you moved forward in
implementing the project?]

ECEPD Director Interview Protocol

2

Rationale: Provides insight into director’s perceptions about the structure and
functioning of ECEPD partnerships and the characteristics that facilitate
implementation
5. Some ECEPD grantees have described the process of securing matching funds as
challenging. Could you describe the process you used to secure matching funds
for your grant and any difficulties or challenges you may have experienced?
Rationale: Addresses ECEPD goals and partnership structure/characteristics,
specifically how sources of matching funds may be related to implementation
decisions, availability of certain activities, etc
6. In what ways, if any, has the evaluation component of your project affected the
program component?
Rationale: Addresses the role of evaluation in the project and whether/how ECEPD
grantees are modifying their activities to meet the requirements of the evaluation
Program Implementation (10 minutes)
7. How has/did your ECEPD program evolve/d over time as you have implemented
it?
Probe: How has it changed?
Probe: Why do you think these changes have occurred?
Rationale: Provides insight into the evolutionary process of implementation;
highlights how project goals and implementation are modified to accommodate
constraints or opportunities (for example, inviting a new partner)
8. One of the key issues in program implementation is recruitment of staff to provide
professional development. Could you describe your experience finding staff for
your program with the qualifications you were looking for?
Probe: Is that the case for all the different professional development staff positions
that you have?
Note to interviewer: Make sure that you ask about all skills and characteristics of all
types of staff (mentors, workshop leaders, course instructors, etc.)
Rationale: Addresses implementation challenges and, to some extent, reflects on the
quality of instruction available in the project
9. What steps, if any, do you take to retain staff members in your ECEPD project?
Rationale: Addresses implementation challenges such as staff turnover

ECEPD Director Interview Protocol

3

10. Let’s talk now about the early childhood educators who are seeking to improve
their professional development by participating in your ECEPD program. We’d
like to know how the participants in your program are recruited and selected. It
may help to think about a typical participant in your program. Please walk us
through how a typical participant would find out about your program and sign up
to be involved.
Probe: From whom and in what ways would a typical participant find out about your
program?
Probe: What particular groups of educators are targeted?
[Try to ascertain the specific characteristics of the educator or the setting that are
important for targeting (for example, English language learners, home based child
care providers, or those working in Head Start)?]
Rationale: Addresses implementation and recruitment of participants. Also sheds
light on characteristics of the participants in the program
11. How does your program target early childhood educators for enrollment?
Probe: Note to interviewer: Try to ascertain the specific characteristics of the educator
or the setting that are important for targeting (for example, English language learners,
home based child care providers, or those working in Head Start)?
Rationale: Addresses educator characteristics in the conceptual model
12. What types of challenges do you face in recruiting participants?
Rationale: Addresses implementation challenges
13. What approaches do you use to encourage participation in professional
development activities?
Note to interview: If target groups are identified above, probe to find out what
incentives/supports are offered to those groups
Rationale: Addresses implementation, access to professional development, and
educator characteristics in the conceptual model
Program Activities (15 minutes)
I would like now to ask you for a brief overview of the main professional development
activities offered in your program.
14. What formats are used to deliver professional development through your
program? For example, do you provide college courses that are credit-bearing,

ECEPD Director Interview Protocol

4

workshops, individualized consultation, distance learning, or rely on other formats
such as on-site consultation or coaching?
Rationale: Elicits a complete picture of the program’s offerings and how they fit
together. This addresses the format and intensity of professional development
activities which is a component of the conceptual model.
15. Could you please walk us through the activities a typical participant would have
available to them in your program?
Note to interviewer: Pay attention to mention of different PD formats, such as
coursework, mentoring, and workshops and probe and ask about each in the
subsequent questions
Rationale: Addresses the package of activities that are available to participants. To
what extent do individual participants participate in different parts of what is offered
across the various offerings? This addresses the professional development activities
component of the conceptual model.
16. [For those who indicate that non-credit bearing workshops are offered:]
a. What is the structure of a typical workshop?
b. What different teaching strategies are used? [by teaching strategies we
mean things such as lectures, group discussions, collaborative activities,
use of video demonstrations, modeling of activities, role playing,
homework between sessions, one-on-one discussion, or other approaches]
c. What are some typical topics discussed or covered in a workshop?
17. [For those who indicate mentoring or consultation:]
a. What is the structure of a typical consultation session?
b. What different teaching strategies are used to deliver the on-site
consultation? [by teaching strategies we mean lectures, group discussions,
collaborative activities, use of video demonstrations, modeling of
activities, role playing, homework between sessions, one-on-one
discussion, or other approaches]
c. What are some typical topics discussed or covered in a consultation
session?
18. [For those who indicate credit-bearing coursework:]
a. What is the structure of a typical course?

ECEPD Director Interview Protocol

5

b. What different teaching strategies are used in the courses? [by teaching
strategies we mean lectures, group discussions, collaborative activities,
use of video demonstrations, modeling of activities, role playing,
homework between sessions, one-on-one discussion, or other approaches ]
c. What are some typical topics discussed or covered in a credit-bearing
course?
Rationale: Elicits details about the specific format, content, and instructional
strategies used in professional development activities
19. What degrees or certificates are offered through your program? For example, do
you offer a CDA, Associate’s Degree, Bachelor’s Degree, Master’s Degree,
Doctorate, Certification to teach young children, or Certification in a particular
curriculum or approach
Rationale: Addresses professional status which is within the Educator Outcomes
component of the conceptual model.
20. What content areas would you say are best covered in the professional
development your program provides?
Rationale: Addresses content of professional development activities.
21. You’ve described a number of options for activities for participants in your
program. To what extent were these activities available prior to your receipt of
the ECEPD grant? Or, would you say they were newly developed as a result of
the ECEPD funding?
Probe: What gap, if any, was your program filling with regard to professional
development offerings in the areas you serve?
Rationale: How is this program innovative? Does it fill a gap for PD in your area?
Addresses state and local policy context within the conceptual model as well as
characteristics of the partnership (for example, ability to respond to the needs of the
community)
Characteristics of Participants (10 minutes)
22. What demographic information did/do you collect about the participants in your
ECEPD program? And, how did/do you collect it?
Rationale: Addresses Educator Characteristics in the conceptual model.

ECEPD Director Interview Protocol

6

23. How does your program accommodate educators with different levels of
experience or prior professional development exposure?
Rationale: Addresses access and educator characteristics within the conceptual model.
Supports and Incentives (10 minutes)
24. What are the major challenges that participants in your program have to grapple
with in moving forward with their professional development?
Rationale: Addresses educator characteristics and the challenges that may prevent them
from achieving their professional development outcomes.
25. What supports, if any, are provided to educators by your program to address these
challenges?
Probe: Do you provide any kind of financial assistance for participating in your
program or purchasing materials like books or supplies?
Probe: Do you help with transportation, substitutes or release time to facilitate
participation?
Probe: Do you provide a guidance or mentoring system?
Probe: Do you provide workshops or coursework in languages other than English?
Rationale: Addresses access under professional development activities in our conceptual
model. Supports make the programs more accessible.
26. Which of these supports do you feel have been the most effective in assisting
educators in reaching their professional development goals?
Note to interviewer: Make sure the supports mentioned in response to this question
correspond to supports mentioned in the previous question.
Rationale: Addresses access under professional development activities in the conceptual
model. Supports make the programs more accessible.
27. What supports would you like to offer to the participants in your program beyond
those that your project currently provides?
Rationale: Addresses access under professional development activities in the conceptual
model. Supports make the programs more accessible.

ECEPD Director Interview Protocol

7

Program Outcomes (10 minutes)
I would now like to discuss your monitoring, reporting and evaluation activities for this
project.
28. What do you do to monitor how well the project is meeting its goals? Are there
specific pieces of information that you track on a regular basis?
Note to interviewer: Make sure you get specifics about the process or information they
track.
Rationale: Addresses how evaluation and feedback loops are being used to inform and
improve the program.
29. How do you track the progress of individual participants through your program?
Rationale: Addresses how evaluation and feedback loops are being used to inform and
improve the program.
30. How do you use the information that you collect?
Rationale: Addresses how evaluation and feedback loops are being used to inform and
improve the program.
31. Can you think of any changes to the program that you implemented because of
what this information indicated?
Rationale: Addresses how evaluation and feedback loops are being used to inform and
improve the program.
32. Was there any information that you wish you had collected that would have
informed your program?
Rationale: Addresses evaluation activities and how they could have been improved upon
to inform and improve the program.
Promising Practices (10 minutes)
My concluding questions relate to the grant’s future directions.
33. How would you sum up the grant’s major successes?
Rationale: Addresses sustainability and scalability of promising practices.

ECEPD Director Interview Protocol

8

34. What were your most significant challenges?
Rationale: Addresses sustainability and scalability of promising practices.
35. Please describe your plans, if any, to continue your ECEPD program after
Department of Education funding ends. What other sources of funding are
available to sustain your program?
Rationale: Addresses sustainability and scalability of promising practices.

36. Do you have plans for disseminating your model so that it can be replicated?
What types of products will be disseminated?
Rationale: Addresses dissemination of information about promising practices.

37. If you were able to communicate one thing to future program directors to help
them have successful programs, what would that be?
Rationale: Addresses key characteristics of promising practices – what works best to
address particular challenges
38. Given everything you have learned thus far from your ECEPD project, what
would you say are the professional development strategies that are most likely to
improve early childhood educator knowledge, practice, and/or child outcomes?
Note to interviewer: Make sure you are clear on which strategy affects which
outcome.
Rationale: Addresses perceptions of promising practices and how evaluation results
are informing strategies

Closing Statement (8 minutes)
I would like to thank you for taking the time from your busy schedules to provide us
information about your ECEPD project. We appreciate your contributions. We would
like to take another five minutes of your time to review the forms that we emailed you
prior to our call today? Do you have those forms available to review?

ECEPD Director Interview Protocol

9

[Confirm that the Director has the forms. Review any missing pieces of information and
ask the Director to help complete the form as s/he is able. Answer any questions that the
Director has about the form or the information that is requested.]
Thank you so much for your help with these details. We’d like for you to please conduct
a final review of this summary and correct or complete any information as necessary.
For some grantees we would like to do additional interviews to go into further depth on
specific issues. In anticipation, can you give us the names, e-mail addresses, and
telephone numbers of the key members of your partnership who could participate in a
conference call about the implementation and activities of your ECEPD Program?
__________________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
___________________________________________________________
____________________________________________________________
Could you also give us contact information for the lead members of the team conducting
the evaluation of your ECEPD program? We would ask them to participate in a
conference call focusing in greater detail on the evaluation of your program.
_____________________________________________________________
______________________________________________________________
______________________________________________________________
______________________________________________________________
_______________________________________________________________
Again, thank you very much for your time and for your participation in the study!

ECEPD Director Interview Protocol

10

ECEPD Project Characteristics Form
Child Trends reviewer: Please review each grantee’s Annual Performance Report(s) and fill out
this ECEPD Project Characteristics Form to the best of your ability. The purpose of this form is to
provide a record of the most current and the most comparable information from each project
regarding the demographics of the staff that provide professional development to early childhood
educators in each ECEPD program, the demographics of the program participants, and the
demographics on the children the early childhood educators serve. Highlight in yellow any areas
that need further clarification or are missing information. The ECEPD Project Director will review
this form and edit or add to ensure its accuracy.
Program ID and Name of Program:______________________________________
Staff Characteristics:
Staff refer to those ECEPD staff members who deliver professional development activities.
1. Estimate the number of staff members in the program who actually deliver
professional development. [Do not count any staff member more than one time,
even if he or she provided more than one course, workshop, etc.]. ___________

2. Estimate the percentage of staff members who delivered professional development
instruction in the ECEPD program during the past year who had the following levels
of education:
_______

Less than high school

_______

High school / GED

_______

CDA certificate

_______

Associate’s Degree (AA)

_______ Associate’s Degree (AA) in Child Development or Early Childhood Ed.
_______

Bachelor’s Degree (BA)

_______

Bachelor’s Degree (BA) in Child Development or Early Childhood Ed.

_______

Master’s Degree (MA)

_______

Master’s Degree (MA) in Child Development or Early Childhood Ed.

_______

Doctorate (Ph.D)

_______

Doctorate (Ph.D) in Child Development or Early Childhood Ed.

_______

Professional Degree (M.B.A., J.D., M.D.)

_______

Unknown

ECEPD Project Characteristics Form

1

3. Estimate the percentage of staff members actually delivering professional development in the
ECEPD program with the following years of experience:
_______

Less than 1 year

_______

1 – 4.99 years

_______

5 – 9.99 years

_______

10 or more years

4. Estimate the percentage of staff members providing professional development to early
childhood educators in the following languages:
_______

English only

_______

Spanish only

_______

English and Spanish

_______

Other [please describe]:

Characteristics of Participants:
Participants refer to those who are receiving professional development services/participating in activities.
1. Estimate the percentage of participants at each of the following education levels.
_______

Less than high school

_______

High school / GED

_______

CDA certificate

_______

Associate’s Degree (AA)

_______ Associate’s Degree (AA) in Child Development or Early Childhood Ed.
_______

Bachelor’s Degree (BA)

_______

Bachelor’s Degree (BA) in Child Development or Early Childhood Ed.

_______ Other:

ECEPD Project Characteristics Form

2

2. Estimate of the percentage of participants in each of the following age groups.
… Less than 18 years old

… 35 – 44 years old

… 18 – 24 years old

… 45 or older

… 25 – 44 years old
3. Estimate the percentage of current participants that are English language learners?

4. Estimate the percentage of participants working in each of the following types of care:
_______

State prekindergarten program

_______

Head Start

_______

Other center-based care

_______

Licensed family child care

_______ Legally unlicensed family, friend, neighbor care
Characteristics of the Children Served by the Participants in the ECEPD Program:
1. Estimate the percentage of children in each age range:
_______

Birth – 2.99 years

_______

3 – 5 years

2. Estimate the percentage of children who are English language learners.

ECEPD Project Characteristics Form

3

ECEPD GRANTEE EVALUATION DESCRIPTION FORM
Child Trends Reviewer: This form is used to summarize information from the Annual Performance Report(s) and Evaluation Reports for an ECEPD
grantee. For each construct of interest listed in the table, please indicate what measures were used to examine the construct, including any
adaptations for non-English speakers; the overall results of the study’s findings with regard to the construct thus far; whether the study addresses
subgroup differences and if so the patterns for subgroups thus far; and, whether the study is examining sustained change over time in the construct,
and if so, the patterns that are emerging thus far. Use as much space as is necessary to fully describe the research questions, research design plan,
and analysis plan. Use as much space as is necessary to fully describe the information within each cell of the table. Highlight in yellow any areas
that need further clarification or which are missing information. The ECEPD Project Director for this project will review this form and edit or add to
ensure its accuracy.
Program ID and Name of Program:
Research Question(s):
Research Design:
Analysis Plan:
Table Summarizing Measures and Results:

Construct of Interest:

Measures used (including Results thus far
adaptations for nonEnglish speakers)

Subgroup differences

Sustained change over
time

Fidelity of Implementation of
Training
Change in Educator Knowledge
Change in Educator Practice
Change in Educator’s sense of
enjoyment and competence
Child Outcomes – Cognitive
Development
Child Outcomes –
Reading/Literacy
Child Outcomes – Social
Development
Child Outcomes – Other
(specify)

ECEPD Evaluation Description Form

1