National Evaluation of the Comprehensive Technical Assistance Centers

Status: Original and derived artifacts are available for this document.

Download: doc | pdf | html




Evaluation of the 
Comprehensive Technical Assistance Centers









Project Inventory Form
and 
Sample Completed Inventory Form





Technical Work Group Meeting
March, 2007
National Evaluation of the Comprehensive Technical Assistance Centers

Project Inventory Form



Dear Center Director,

Enclosed are a project inventory form and instructions to be used to compile a complete list of your center’s projects during the current project year (July 1, 2006 – June 30, 2007).

The national evaluation will use the completed project inventories to select a sample of projects for review by expert panels. Please designate [2 or 3] projects that you believe best represent the work of your Center; these projects will be included among the sample of projects to be reviewed by expert panels.  These panels will rate nominated and sampled projects for their quality and relevance.  

Please complete a draft inventory to review with a member of the national evaluation team during the site visit for the evaluation, scheduled to take place between April and June 2006 when [EVALUATION STAFF MEMBER] visits your center.  A final version of the project inventory should be completed as soon as possible after the site visit, but no later than July 1, 2007.  Please return the completed inventory form to your evaluation liaison, [name] at _________@___________.

If you have any questions about the inventory or the instructions for completing it, please contact _______________ at 1-xxx-xxxx or by email at _____@_________.

Thank you for your continued support of the national evaluation of the comprehensive center program.



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number.  The valid OMB control number for this data collection instrument is xxxx-xxxx.  The time required to complete these worksheets is estimated to average 16 hours per response, including the time to review instructions, search existing data sources, gather the data needed, and fill in the form.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to the U.S. Department of Education, Washington, DC 20202-4651.
Instructions for Completing an Inventory of Projects

	Each Comprehensive Center will prepare an inventory of all of the projects that were active during the current grant period.  The first inventory will cover the period from July 1, 2006 through June 30, 2007.  The second inventory will cover materials from July 1, 2007 through June 30, 2008, and the third inventory will cover July 1, 2008 through June 30, 2009.  

	Projects that began in the current project year but will not be complete by June 30 should be entered in the inventory and noted as “ongoing.”

Purpose
This inventory will serve as the sampling frame for the expert panel reviews to be conducted for the national evaluation.  A sample of projects will be selected in two ways.  Each Center will nominate [2 or 3] projects that they believe best represents the work undertaken by that Center.  The remaining 4 to 7 projects will be selected at random from each center’s inventory (excluding the projects nominated by the Center).  Samples will be drawn to reflect each center’s overall portfolio of work, as well as the work of all regional or content centers in key topic areas.  (More details on sampling are available in the evaluation’s sampling plan, or from your center’s liaison to the national evaluation.)  Items selected for review will be rated by one or more expert review panels in two areas:  quality and relevance.  
If a project is nominated or sampled for expert panel review and rating, the evaluation team will ask your center to collect and transmit all of the documents and other artifacts associated with that project (meeting agendas, briefing books, meeting summaries, training materials, white papers, web resources, etc.) for the expert panel to review.

Identifying Projects for Inclusion in the Inventory

The unit of analysis for the expert review panels will be the project.  For the purposes of this inventory, a “project” will ordinarily comprise a group of closely related activities and/or deliverables designed to achieve a specific outcome for a specific audience.  The content centers have some projects that consist of the development of a single product or deliverable, rather than a group of deliverables, but a project that comprises only a single product or deliverable will be the exception rather than the rule.  The inventory should include all of the projects developed or delivered by the center, including those developed or delivered in collaboration with other centers, during the reporting period.  

Because each project listed in the inventory could potentially be sampled for expert panel review, each project shown as an entry (or row) in the inventory form should be a relatively complete project that can be understood and rated on its own by expert panel members who may not know anything about other aspects of the center’s work.  Although a single project may include a number of deliverables and activities, it will be designed to achieve a specific outcome and address (in almost all cases) a single topic.  Where a group of activities and deliverables can be divided up into separate projects, each constituting a complete and coherent whole, the center should list these as separate projects in the inventory.

The following criteria should guide the centers as they identify projects (and their associated activities and deliverables) for recording in the inventory form.  They have been developed to ensure that projects constitute units that are large enough for review and rating, but focused enough for coherence.  Each project entered in the inventory should satisfy all three of the following criteria:
	
    • Complete and coherent whole.  Each project listed in the inventory should be able to stand on its own in an expert panel review.  Avoid listing activities and deliverables as separate projects in the inventory if they cannot be understood or evaluated without information about related activities and deliverables.   For example, a training event may require extensive planning (e.g., needs assessment, materials development) and follow-up activities (e.g., evaluation of the training, consultation on action plans).  These planning and follow-up activities would be very difficult for a panel to rate in the absence of information about the event itself.  Therefore the center should list these planning and follow up activities and the event itself as a single project (one row) on the inventory form.  Each phase of the project—planning, the event itself, and follow up—will be described briefly in the “activities and deliverables” column.  Similarly, ongoing work with a state-level task force should be listed as a single project rather than each meeting of the task force being listed as a project.

    • Common intended outcome.  Where a cluster of activities and deliverables is designed by the center to lead to the same outcome for the same audience(s), those activities and deliverables should be grouped as one project in the inventory form.  On the other hand, where a set of activities is intended to produce more than one distinct outcome—for example, helping the state to develop a strategic plan for improving reading instruction, and helping the same state recruit and train literacy coaches—those two activities should be listed as separate projects.  Where the Center replicates the same set of activities in each of several different states, that set of activities should be listed as a single project if the intended outcomes and processes do not differ materially from one state to another.  Where the intended outcomes do differ substantially from state to state, the work in each state should be listed as a separate project. 

    • Topic area focus.  With few exceptions, a project addresses just one of the 14 topic areas described in Exhibit A at the end of this document (e.g., state systems of support, reading/language arts).  Where it is possible to divide a group of related activities into two different projects according to the topic area addressed, centers should do so.  For example, a regional forum on interventions for low-performing students in reading and mathematics that offers separate strands of sessions in each subject area should be listed as two different projects, one under reading/language arts and one under mathematics.  In this case, all of the sessions on reading interventions would stand on their own as a complete and coherent whole for rating by an expert panel and should be listed as a separate project; the same would be true for the sessions on mathematics.  (If a project cuts across topic areas and the activities and deliverables which it comprises cannot be divided up by topic area into complete and coherent units that would make sense to a review panel, the project should be listed under the most relevant topic area with a note cross-referencing other topics, following the instructions for column IV below.)  

The sample inventory form at the end of this packet includes examples of the kinds of projects that should be listed by the centers.  

	 Some projects, activities, and deliverables should not be included on the inventory form at all.  These include:

    • Training or professional development for Comprehensive Center staff

    • Work on coordinating committees within the Comprehensive Center network 

    • Annual needs assessment activity or negotiations with states, unrelated to specific projects

    • Other internal working meetings or documents


Completing the Inventory Form

Centers should use the attached form to complete their inventory of projects.  Sample projects and examples of entries can be found in Exhibit B.    

	Centers might find it useful to review their annual project plans, technical assistance plans, management plans or technical assistance logs as a starting point since those documents typically provide an overview of the various projects and activities that were planned for or conducted during the year.  

	Once the inventory form is complete, Centers should designate which projects they would like to nominate for inclusion in the sample of projects reviewed by the expert panels, by inserting ** after the name of the project in Column II.  Centers can designate [either 2 or 3] projects, representing one-third of the projects that will be evaluated for the Center.

	List each project under the appropriate topic heading.  A list of topic headings, with definitions, is attached at the end of this document in Exhibit A.

Directions for completing each column are as follows:  

Topic Area Headings

List items under the appropriate topic area heading.  Where a project fits under more than one topic heading, list it once, under the topic heading that is most relevant.  Note the project’s relevance to other topics in Column IV.  Add rows to the table as needed.  Leave rows blank if your center does no work in a particular topic area.  

I.  Project number
Assign consecutive numbers to each item listed in the inventory.  (Centers may want to complete this column as a final step, after all of the items have been entered in the inventory.)

II.  Name
Enter the name of the project.  Projects nominated by Center staff for review should be followed by ** in this column.

III.  Description
Provide a concise description of the project.  (See examples for appropriate level of detail.)

IV. Additional topics addressed?
If the project addresses more than one topic area, note that here.  Use the list of topic areas provided in the appendix (this list corresponds with the row headings in the inventory).  Entering the appropriate number from the list will save space.

V.  Activities and Deliverables
List all of the activities and deliverables associated with the project.  (For products, include exact title, if applicable.  For services, include location and type of participant.)  Specific activities and deliverables may include:
  
    • Meetings/conferences (includes items such as workshops, conferences, institutes, forums, webinars)

    • Expert consultation/technical assistance (includes items such as assistance completing reports or applications, review of state plans, needs assessments, audits)

    • Facilitation/support of working groups or teams (includes items such as planning meetings, participation in meetings, drafting summary documents)

    • Guidance/information resources (includes items such as policy or issue briefs, fact sheets, congressional testimony, resource guides, planning tools, field guides, benchmarking rubrics, handbooks, exemplars, literature reviews/summaries, annotated bibliographies, case studies, websites)

    • Training (includes items such as professional development materials/services, software, training materials)

    • Other, specify ______________________

VI.  Start Date

Enter the start date for the project, including month and year.

VII.  End Date

Enter the end date for the project, including month and year.

If the project is currently ongoing, enter the note “ongoing.”

VIII: Major, Moderate or Minor Project 
Indicate whether you consider this a major, moderate or minor project in terms of the relative level of effort and/or resources devoted to it.

IX:  Target state(s), region(s), or regional center (s)
Regional comprehensive centers serving multiple states should note which state(s) participated in each project.  Regional centers serving a single state (Alaska, California, New York, and Texas) should note which region(s) within the state participated (e.g., New York City vs. rest of state).  Content centers should specify which regional centers participated in the project.

X:  Collaborations and Sources
If the center used materials developed by one of the content centers in the course of designing or delivering its own services, list these content centers as sources.

If the center collaborated with another regional center or content center on the design, development and/or delivery of products and services, list the names of those collaborating centers.

Note collaborations with other comprehensive centers only.  It is not necessary to record collaborations with other technical assistance providers, universities, or other agencies on this inventory form.
National Evaluation of the Comprehensive Technical Assistance Centers
Inventory of Projects

Center Name: ________________________________								

I.  Number
II.  Name
III.  Description
IV.  Additional topics addressed? (From list in Exhibit A—enter appropriate number)
V.  Activities and Deliverables
VI. Start Date
VII.  End Date (enter “ongoing” if project is not complete)
VIII. Major, moderate or minor project
IX. Target state(s), region(s) within a state, or regional centers(s)
1. State Systems of Suppor for Schools Identified for Improvement
 








 








2. Building District.Local Capacity to Support School improvement, or Address Corrective Action and Restructuring
 








 








3. Assessment (exc luding assessment of special education students or ELL students)
 








 








4. Reading/Language Arts Curriculum, Instruction, and/or Professional Development (includes Adolescent Literacy)
 








 








5. Mathematics Curriculum, Instruction, and/or Professional Development
 








 








6. Other Content Area Curriculum, Instruction, Professional Development, Standards-Based and Research-Based Instructional Frameworks
 








 








7. High School Reform

 








 








8. Special Education










 










I.  Number
II.  Name
III.  Description
IV.  Additional topics addressed? (From list in Exhibit A—enter appropriate number)
V.  Activities and Deliverables
VI. Start Date
VII.  End Date (enter “ongoing” if project is not complete)
VIII. Major, moderate or minor project
IX. Target state(s), region(s) within a state, or regional centers(s)
X.  Collaborations and Sources (other Comprehensive Centers only)
9. English Language learners
 








 
 








 
10. Highly Qualified Teachers
 








 
 








 
11. Parent and Community Involvement
 








 
 








 
12. Supplemental Educational Services
 








 
 








 
13. Other NCLB-related TA
 








 
 








 
14. Other
 








 
 
 
 
 
 
 
 
 
 
 

**Projects nominated by the Center for review by expert panels
Exhibit A
Topic Areas and Definitions

Topic Area
Definition
    1. State Systems of Support for Schools and Districts Identified for Improvement
Work that supports an SEA and/or its service delivery system carry out state-level responsibilities related to supporting district and schools identified for improvement.  Service may target School Support Teams (SSTs), Distinguished Educators (DEs), or regional education service centers (RSCs, ESCs).
    2. Building District/Local Capacity to Support School Improvement, or Address Corrective Action and Restructuring
Includes any work designed to build district- and school-level capacity to carry out school improvement and to make AYP, as required under NCLB, including remedies taken for schools or districts in “corrective action” or “restructuring.” 
    3. Assessment
Design and implementation of local assessment systems, such as benchmark assessments aligned to state assessments; classroom assessments designed to support instructional change; or support for analysis and use of data to drive instructional change.  Includes consultation on the design and implementation of state assessment systems.  Does NOT include alternate assessments for SPED or ELL.
    4. Language Arts Curriculum, Instruction, and Professional Development
Includes all work at all levels that addresses ELA, reading, and literacy.  Includes projects on adolescent literacy.  May include work related to curriculum alignment, research-based models or programs, standards and policy reviews, and teacher professional development.  
    5. Mathematics  Curriculum, Instruction, and Professional Development
Includes all work at all levels that addresses mathematics and numeracy.   May include work related to curriculum alignment, research-based models or programs, standards and policy reviews, and teacher professional development.  
    6. Other Content Area Curriculum, Instruction, and Professional Development; Standards-Based and Research-Based Instructional Frameworks
Includes all work at all levels that addresses specific content areas other than language arts or math, such as science and social studies.   May include work related to curriculum alignment, research-based models or programs, standards and policy reviews, and teacher professional development.  Includes work on curriculum alignment and policies related to academic standards that are NOT content area-specific. 
    7. High School Reform
Includes work related to smaller learning communities, 9th grade or high school transition, dropout prevention, high school policies.
    8. Special Education
Includes all work related to special education, such as alternate assessments for the “1 percent and 2 percent students” or instructional/assessment strategies that target needs of students with IEPs.
    9. English Language Learners
Includes all work related to ELL (i.e. LEP) students, such as work on Annual Measurable Achievement Objectives (AMAOs) for ELL students and instruction/assessment of ELL students.
    10. Highly Qualified Teachers
All work addressing the NCLB requirement that states have 100 percent of teachers licensed to teach in their subject area and 100% of paraprofessionals with 2-years of college.
    11. Parent and Community Involvement
All work with parents and community members EXCEPT outreach regarding Supplemental Educational Services or opportunities in association with schools in corrective action or restructuring
    12. Supplemental Educational Services
Includes work related to helping SEAs evaluate and monitor Supplemental Education Services, as well as setting up systems for notification and outreach to districts, schools, and parents.  
    13. Other NCLB-related TA
Completing miscellaneous federal applications or reports, convening SEA staff to address miscellaneous NCLB topics; ongoing consultation on NCLB topics, including migrant education.
    14. Other
Includes projects that do not fit under any other topic area, including websites, membership on SEA committees, and ongoing phone support.  Includes projects building general skills and knowledge of leaders at all levels.  Includes projects that assist clients with the review, collection, and translation of research, where there is no specific topic.  If the research addresses a specific topic area, put the project in the appropriate row.  Does not include internal or network meetings—these types of activities should not be included in the project inventory.


Exhibit B
Sample Project Inventory

Center Name: ______________________________								

Note: 	All examples in the table were taken from Year 2 baseline management plans.  

I.  Number
II.  Name
III.  Description
IV.  Additional topics addressed? (From list in Exhibit A—enter appropriate number)
V.  Activities and Deliverables
VI. Start Date
VII.  End Date (enter “ongoing” if project is not complete)
VIII. Major, moderate or minor project
IX. Target state(s), region(s) within a state, or regional centers(s)
X.  Collaborations and Sources (other Comprehensive Centers only)
1. State Systems of Suppor for Schools Identified for Improvement
1. 
State TA system for PI districts and schools
Design an integrated SEA TA system that reaches “program improvement” districts and schools

    • Convene coordinating council;
    • Inventory existing TA efforts;
    • Help create TA plan;
    • Give PD and TA to school support teams
7/2006 
Ongoing
Moderate

 
2. 
Statewide system of support**
Support development of Regional School Service Centers as a system of support for addressing NCLB

    • Help create a plan to guide the Regional Network Strategy for the next five to seven years;
    • Develop the next RFP for the RSSCs;
    • Support RSSC implementation;
    • Develop and help deliver PD;
    • Develop a protocol for collecting information on implementation 
7/2006  
6/2007
Major
State A
 
3. 
“Significant Change in School Improvement and Restructuring”
Develop a modular handbook and workshop on implementation of fast-paced significant school improvement, including restructuring
2 (Local capacity)
    • Draft 8 modules for handbook to be used with SEA systems of support;
    • Present at institute;
    • Revise and add workshop materials;
    • Consult at CC A regional meeting


Major
All Regional CCs
Regional CC A
4.
Support to School Improvement Team
Help the SEA School Improvement Team to identify resource and design training for schools in “monitored” and “priority” statuses

    • Attend meetings organized by SIP team and provide reflective syntheses;
    • Broker resources and services available through RRC and CC networks
7/2006  
6/2007
Minor
State A

2. Building District.Local Capacity to Support School improvement, or Address Corrective Action and Restructuring
5.
District tools for monitoring schools
Help the SEA develop processes and tools for districts to use in monitoring the implementation of school restructuring plans

    • Prepare draft processes and tools for monitoring;
    • Meet with SEA school improvement staff to review/revise tools
7/2006 
10/2006
Minor
State A
 
6. 
Leadership Institute on Helping Districts Assist Low-Performing Schools
Design and implement a PD plan to support the Local Education Agency Assistance Program

    • Convene planning team to design the Leadership Institute;
    • Help conduct institute;
    • Provide ongoing support to participants as they work with districts and schools
7/2006 
6/2007
Moderate


7.
Guidance to districts on restructuring schools
Assist SEA in developing guidance for districts with schools in restructuring

    • Assist state in writing guidance protocol;
    • Pilot guidance;
    • Collect and prepare research;
    • Provide initial training to districts with schools in restructuring
7/2006
12/2006
Moderate
State A
 
3. Assessment (excluding assessment of special education students or ELL students)
8.
Growth models
Build SEA understanding of assessment to help in decision making about accountability growth models

    • Identify state needs and create plan for growth model;
    • Assist states accepted as pilot states;
    • Assist ongoing development for other states
7/2006 
Ongoing
Major
All states in region

9.
Assessment data analysis**
Build SEA and other staff capacity by helping them train districts and schools in analyzing state assessment results and using them for instructional planning

    • Meet with SEA staff to plan PD;
    • Help host Statewide Assessment Conference;
    • Continue to plan and offer training in assessment analysis;
    • Establish a cadre of educators to provide PD to districts
5/2006 
6/2007
Major
State A
Content CC A
4. Reading/Language Arts Curriculum, Instruction, and/or Professional Development (includes Adolescent Literacy)
10.
Adolescent Literacy Cadres
Organize and deliver training of adolescent literacy cadres made up of SEA staff

    • Convene SEA work teams to analyze research;
    • Review literacy diagnostic tools;
    • Customize a literacy assessment;
    • Conduct PD for SEA work teams on customized assessments
10/2006 
6/2007
Moderate
All states in region
Content CC A; Content CC B;
Content CC C
11.
Effective Leadership in Literacy
Build SEA capacity to support “effective leadership in literacy for grades 6-12”

    • Convene work group and discuss research; 
    • Provide SEA with PD on Strategic Instruction Model;
    • Develop state literacy plan
7/2006 
6/2007
Major
State A; State B
Content CC A
5. Mathematics Curriculum, Instruction, and/or Professional Development
12.
K-2 Mathematics Inventory
Develop and provide PD to a cadre of educators to help schools implement the state k-2 Mathematics Inventory

    • Help SEA design PD;
    • Help design team pilot workshop;
    • Revise PD;
    • Help deliver regional trainings prioritized for schools in improvement;
    • Review feedback to redesign PD
7/2006 
6/2007
Major
State A
Content CC A
7. High School Reform
13.
Implementing High School Task Force Report
Develop capacity of SEA to implement recommendations of HS Task Force Report related to high school redesign

    • Work group and Steering Committee meet to consider task force recommendations, research, best practice, and review of prior initiatives;
    • Identify “knowledge management” components needed to support state implementation;
    • Identify policies and practices needed to promote redesign;
    • Develop multi-year implementation plan.
9/2006 
4/2007
Moderate

 
8. Special Education
14.
Assessing special needs students
Support SEAs in establishing assessment and accountability systems that include and support special education, ELL, and low-income students
10 (ELL)
    • Collect information from Content CC A on research- based practices;
    • Conduct SEA Needs Assessment;
    • Provide regional training(s) on Scientifically Based Research on assessment and accountability systems for special needs students
7/2006 
Ongoing
Major
All states in region
Content CC A
15.
Developing an Alternate Assessment
Assist SEAs in developing Alternate Academic Assessments for the “1%” of students who have significant cognitive disabilities 
3 (Assessment)
    • Identify needs and broker assistance in developing assessment;
    • Provide TA on development of assessment;
    • Provide ongoing TA on implementation
7/2006 
6/2007
Moderate
State A;
State B
Content CC A
16.
Research Synthesis on Special Needs Students
Publish Research Synthesis 1 focusing on teaching special-needs students 
12 (HQT)
    • Identify and evaluate existing research and syntheses;
    • Conduct gap analysis to identify research synthesis needs;
    • Write synthesis (50-60 pages);
    • Post on on-line resource forum..
7/2006 
9/2007
Major
All Regional CCs

9. English Language learners
17.
District guidance on the education of ELL students
Help develop a comprehensive framework for educating English Language Learners to guide district work, including guidance on the use of formative assessments to improve instruction and on family and community engagement  
13 (Parents)
    • Facilitate meetings of the SEA task force responsible for developing and disseminating a comprehensive framework for the education of ELLs;
    • Assist in integrating effective formative assessment practices for ELL students;
    • Assist in formulating guidance on ELL family and community engagement strategies;
    • Provide consultation and resources from Content CC A
7/2006 
6/2007
Major
State A
Content CC A
18.
Addressing AMAOs
Assist SEA in aligning Annual Measurable Achievement Objectives, standards, and assessments for ELL students

    • Conduct a study on alignment of English Language Proficiency standards for grades K-12 with the state assessment for ELL students
    • Help state reconfigure AMAOs to align with state test
8/2006 
12/2006
Minor
State A;
State B;
State C
 
10. Highly Qualified Teachers
19.
“TQ Source Project”
Develop online resource clearinghouse that provides clients with policy and research data specifically related to teacher quality, as addressed in NCLB

    • Develop “interactive data tool” based primarily on NCES data;
    • Update “policy database” for  state-by-state policy areas, including policies related to PD and teacher prep, recruitment, retention, certification, etc.;
    • Publish next issue of “Tips and Tools: Emerging Strategies to Enhance Teacher Quality”;
    • Enhance “Publications database” focusing on teacher quality research
7/2006 
Ongoing
Major
All Regional CCs
 
20.
Teacher Preparation and Licensure in literacy
Assist SEA in reviewing teacher preparation and licensure requirements related to literacy  
4 (Language arts)
    • Assist the Task Force on Licensure/Professional Development in revisiting preparation and licensure requirements related to literacy instruction for school leaders and teachers;
    • Discuss findings and possible next steps related to engagements with representatives from institutes of higher education
10/2006 
2/2007
Minor
State A
 
12. Supplemental Educational Services
21.
“State Evaluation of Supplemental Educational Services” Manual
Update “State Evaluation of SES” manual and assist with its use by Regional CCs 


    • Draft updates to SES evaluation manual
    • Present at September 2006 Institute
    • Revise based on evaluations and usage
    • submit evaluation report to ED
9/2006 
4/2007
Major
All Regional CCs
Regional CC A
13. Other NCLB-related TA
22.
Enhancing Schoolwide Planning Materials
Assist SEA in updating resources and training for district Title I directors on the topics of schoolwide planning, plan implementation, and district monitoring of school plans

    • Help update T1 schoolwide application and evaluation rubric;
    • Ensure that North Central Association Commission on Accreditation and School Improvement school improvement processes include schoolwide plan components;
    • Review need assessment results from district/schools;
    • Update materials based on needs
7/2006 
6/2007
Major
State A
 


**Projects nominated by the Center for review by expert panels