TAH Project Director Protocol

TEACHING AMERICAN HISTORY EVALUATION

SUMMER 2009 SITE VISITS

Project Director Protocol

A. Background and Work Responsibilities

1. Please tell me about your background. How long have you been the project director of this grant? How long has the project been in operation? Have you been the only project director? (If not, could you briefly describe the role of the previous director?) What do you consider to be your primary responsibilities associated with this project (e.g., administrative, recruitment, training-related, accountability)?

2. What did you do before directing the TAH project? What responsibilities do you have in addition to directing this TAH project?

B. Planning and Goals of the TAH Project

3. Why did you (or the grantee) decide to apply for this grant? How did you determine the need for this project prior to designing the program? What gaps in history education were you attempting to address? How were those gaps identified? Did you conduct a formal needs assessment? What input was solicited from teachers, history experts, or administrators?

4. Were there any prior grants or initiatives in this district(s) relating to teaching American history, history-related curriculum development, or to other professional development in history-related content? If yes, what were they? How did they inform the planning for and design of the TAH grant?

5. What are the overall goals for the project? How was the project designed to meet those goals? (Probe for district, state, and federal goals, use of assessment data for decision-making, and relevancy of improving teacher content knowledge/student achievement.) Did you consult research on effective professional development strategies to inform the design of the professional development and follow-up of project activities?

6. How did you decide what schools or districts to involve in the project and what grades to include?

7. What kinds of supports and barriers did you encounter in planning the project?

8. Does the district(s) administer a state or district-wide assessment of American history? How long has the assessment been administered? Can you describe the test? Does it include essay questions/writing? Is the test part of a social studies test? If so, can the U.S. history segment be separated out? To what extent was the state or district-wide assessment a factor in planning the TAH project and evaluation activities? Is it possible to connect the assessment results to the students of a given teacher? When do you receive test results? Are you able to use this information in planning, and if so, how?

9. Describe the implementation of the grant: How did you identify partners? When did the actual professional development activities begin? Did you encounter any additional supports and/or barriers as you implemented the project? Did you conduct a formative evaluation? If so, how were the results used? Overall, how have your goals and implementation activities changed over time?

10. How, specifically, do you plan to use this summer institute to improve history content knowledge and classroom instructional practice?

11. Have you received useful information, guidance, and/or technical assistance from ED’s program officers, technical assistance contractors, or the annual TAH Project Directors Conference? If so, can you describe the support you received and how it was useful? In what ways could federal support have been improved?

12. Have you coordinated efforts or exchanged information with your state’s Social Studies Coordinator or other State Education Agency personnel? If so, please describe.

C. Partners

13. Who are the grantee partners for this project and what roles do they play? (For each partner, probe for possible roles: planning, training delivery, fiscal, administrative roles, etc.) How were partners identified and brought into the grant?

14. What have been the benefits of working with these partner organizations? What have been some of the challenges?

D. Participating Districts, Schools and Teachers

15. Please explain how districts were chosen or came together for participation in this grant. Were the districts involved in the project design and planning? If so, how? What commitments to this project were solicited and provided by the participating districts?

16. What factors supported or impeded gaining support for the project at the district level? What has supported or impeded sustaining district support? Can you describe some ways district support has been demonstrated throughout the life of the project?

17. Please explain the role of schools in the grant: Did schools choose to have all eligible teachers participate? Were specific teachers selected to participate? Or did teachers self-select into the program? How was teamwork encouraged within schools? What role did the district play in encouraging participation by schools and teachers?

18. Please describe the demographic characteristics of students in the schools of TAH participants. (Note: Project directors will be asked in advance to locate, or advise SRI/BPA how to locate, this information.)

_______approximate percentage eligible for free and reduced-priced lunch

_______approximate percentage of special needs students

_______approximate percentage of English language learners

_______approximate percentage of minority students

_______approximate percentage of honors or AP students

19. Please describe the participating teachers:

1. To date, how many participants have received training? How many teachers participated this year? What are your targets for next year?

2. What grades do participating teachers teach? Approximately what percentage of American history teachers from each grade and school participate? How representative are participating teachers of all the American History teachers in the schools? (Probe for level of experience and expertise of teachers, minority teacher involvement, and new teacher involvement.)

3. How were teachers recruited, targeted and/or selected? What challenges do you face in the recruitment and retention of teachers? What is your overall attrition rate?

4. What factors motivated teachers to participate, or impeded their participation?

4. To what extent have school principals/administrators been involved in implementing training or recruiting teachers?

E. Activities

20. Is there an overall theme or historical period emphasized? Why did you choose to address this historical content? What types of history skills and teaching methodologies are emphasized? Why?

21. (BPA/SRI will obtain this information in advance of the interview from documents, to the extent possible, and will confirm or clarify in the interview.) Please describe the activities your project has offered.

Activity	Year, number of sessions and total hours	Number of participants	Where was training provided? Who taught the sessions?	Training methods used	Content addressed
Summer institute
Multi-session workshops with follow-up
Single session workshops, seminars, field trips
Lesson planning or project related to classroom teaching/ curriculum development

Activity	Number of sessions and total hours	Number of participants	Where was training provided? Who taught the sessions?	Training methods used	Content addressed
Review of lesson plans
Observation and feedback of teaching
Classroom-based support such as coaching or mentoring
Participation in network with historical research experts
Action research team related to historical topic
Online/ technology-based research
Other

22. On average how many hours do teachers participate per year? Do all participants participate in the program equally or are some participants more actively involved than others?

23. To what extent have teachers implemented what they have learned in the classroom? How do you know if new content and strategies have been implemented in the classroom? Have teachers received training materials, lesson plans, websites, or other resources for use in the classroom? Which of the above activities do you feel have been most effective and useful in improving instruction? Why? What have been key positive outcomes for teachers?

24. What is your assessment of the overall quality and relevancy of the training teachers have received in addressing the teacher needs identified for the project? What components of your project worked best? What were the most challenging? What would you do differently?

25. How typical is the training observed by SRI/BPA in terms of the kinds of activities that have occurred or will occur as part of your TAH project? (If yes, in what ways? If not, why not?)

26. Have there been positive outcomes for teachers or administrators who have not directly participated in the training activities? (Probe for teachers’ sharing of information with colleagues, school-wide access to website, lesson plans or other resources.)

F. Student Achievement

27. (BPA/SRI will collect information on a and b below in advance from documents, to the extent possible, and confirm in the interview.) How does the project assess the achievement of students taught by TAH project participants? (Include all methods of assessment, including experiment, quasi-experiment, single group pre-post, case study, focus groups, attitude surveys, etc.)

a. If you did not conduct a research study (experiment, quasi-experiment, or single group pre-post) what went into your decision-making not to do so? What have been the challenges in measuring student achievement?

b. If you did conduct a research study, please briefly describe the research design, type of analysis, any comparison groups, and the results. (If copies of written reports are available, please collect these documents.)

c. What were the challenges associated with conducting each of the evaluation activities? Will you conduct similar studies in coming years? What will you do the same or differently?

28. Do you have an external evaluator? What is his/her role? What have been the successes and challenges in working with the evaluator to assess student achievement?

29. Do you have suggestions for other ways TAH programs could better assess student achievement? Do you have suggestions for ways to better assess the relationship between TAH teacher participation and student achievement?

G. Assessment of Teacher Learning

30. (BPA/SRI will obtain this information in advance from documents, to the extent possible, and confirm in the interview.) How does the project assess what teachers learn during the program?

a. Do you use:

____standardized pre- and post-test of historical knowledge

____teacher surveys (what kind, pre and post?)

____teacher interviews or focus groups

____changes in student standardized test scores

____classroom observations

____reviews of lesson plans

____other (describe)

31. Please describe each assessment activity. How was it developed? What content did it cover? What have been the results/findings? Have you used the results to make any changes in the project? How?

32. How well do you feel these assessments (or the evaluation) have measured what teachers have learned by participating in the program? What have been the challenges in finding ways to assess what teachers learn by participating in your project? What have been the challenges of working with external evaluators in this process? Do you have any suggestions for other ways TAH programs could assess teacher learning of historical content, historical thinking skills, or changes in teaching methods?

H. Reflection and Wrap-Up

33. Will you sustain the training after the grant ends? What kinds of resources would you need to do so?

34. Overall, how has success been defined within your project? What factors would you say made the greatest difference in the success (or shortcomings) of your TAH grant?

35. Is there anything else we should know to better understand your experience with the Teaching American History Program?

36. Can we get a set of the training materials used for the sessions we observe? Would it be possible to collect syllabi and course materials from the training? Are there any copies of teacher or student assessments you are able to share with us?

Thank you for your participation!

7