Logic Model

Appendix A. NES Program Logic Model

NASA Explorer Schools Project Conceptual Framework

The NASA Explorer Schools Project

Responding to recommendations from the National Research Council committee that reviewed NASA’s elementary and secondary education projects,¹ NASA embarked on a redesign of the NASA Explorer Schools (NES) project.² The new NES model represents a coherent effort by NASA to help engage students in science, technology, engineering, and mathematics (STEM) and inspire them to pursue STEM careers, or at a minimum, become part of a STEM-literate citizenry. The redesigned NES model targets the middle and high school levels, and includes four core elements: (1) STEM curriculum support materials; (2) electronic professional development (ePD); (3) virtual NASA news events; and (4) teacher, student, and school recognition opportunities.

NES staff has worked with the Abt evaluation team to develop a project logic model that depicts the project’s theory of change (Exhibit 1). The logic model surfaces the assumptions about why the NES program is expected to work. It identifies the short- and long-term outcomes of the project and the links to the project’s inputs and activities.

NES’s Theory of Change

NES seeks to build student interest and engagement in STEM by involving students in NASA-related STEM activities in classrooms, and supporting teacher use of the materials. To help engage students in STEM, NES provides NASA-developed STEM curriculum support materials and NASA Now video events for use in STEM classrooms. NES provides electronic professional development to train teachers on the use of the NES materials and the related STEM content. To promote the use of best practices, NES recognizes teachers and schools use of best practices in the areas of curriculum integration, student engagement, technology use, community outreach, and family involvement.

Exhibit 1 displays the NES theory of change. Moving from left to right, the model makes explicit the links between the project inputs and activities through NES intended short-term and long-term outcomes of NES. Each of these elements in the logic model is described below.

Exhibit 1: Logic Model Depicting NASA Explorer School’s Theory of Change

Short-Term

Outcomes

Inputs and Activities

NES Inputs and Activities

The NES project is run by staff from both NASA and implementing partners, who are contractors involved in the implementation of various components of the NES project. The NES virtual campus website is the central hub for the project; it provides the interface between the NES project and the teacher, school, and student participants. NES, with its strategic partners, is involved in advertising the NES project in order to increase visibility and recruit teachers and schools to participate. Teachers sign up to participate via the NES website and they access the NES curriculum materials, ePD training, NASA Now events through the virtual campus. The virtual campus also contains links to the NES social networking components and other STEM-related opportunities.

NES offers 20 NASA-developed modules for middle and high school teachers to use as supplementary materials in their science, mathematics, engineering, or technology classrooms. The curriculum modules were selected for their potential student and teacher appeal, classroom relevance and grade appropriateness, applicability across STEM areas, intellectual rigor and incorporation of scientific inquiry or engineering design, and ease of use and flexibility. To assist with the use of these materials, NES provides teachers with content and implementation training on the NES curriculum modules via ePD segments. This training is further enhanced by support that NES offers to teachers via an email and phone help desk.

In addition, NES offers NASA Now videos related to on-going NASA research and programs. The videos are intended to be used by teachers in classrooms with students to engage students and expose them to NASA-related content, missions, and careers. NES also fosters the development of teacher communities through the moderation of online collaborations and access to social networking tools (e.g. NASA Educators Online Network (NEON)).

Another component of the NES experience is the recognition program. The recognition program identifies best practices in five areas: curriculum integration, student engagement, technology use, community outreach, and family involvement. This component is being structured to recognize the expression of best practices among participants. NES hopes to motivate teachers in each of these areas by highlighting them in the recognition program.

Teacher Inputs and Activities

Teachers self-select into the program and so bring their own motivation and desire to participate. As participants they contribute time to plan and implement curriculum modules, view ePD, participate in the recognition program and social networking. Social networking is one mechanism by which NES and NASA teacher networks may be established. Teachers may also promote NES among their colleagues, thus broadening the participation in NES and laying the groundwork for networks.

Teachers drive the use of the NES materials, both modules and NASA Now, in the classrooms. Although NES modules identify the materials that are necessary to implement a unit, teachers provide any supplies for activities as necessary. Also, teachers provide feedback to the NES project via surveys that correspond to the individual project components. Finally, schools and teachers provide the technology necessary to access the NES products and use them in classrooms.

Partner Inputs and Activities

Strategic partners—external stakeholders that are collaborating with NES to support the NES model—also serve an integral role in NES. Partners provided reviewers to validate and make the final selection of curriculum modules. Partners also provide access to networks of teachers and they assist in the recruitment of teachers to NES via these established networks. In addition, NES has linked to resources that strategic partners offer, so that NES teachers and students might have access to resources that extend beyond the NES project.

Outputs

NES outputs can be measured at the student, teacher, and school levels. These outputs include the number of teachers, students and schools that participate in NES and in particular activities. At the student level, outputs include the number of students engaged in NES, through teachers’ use of curriculum modules and NASA Now events in classrooms. It is expected that a subset of NES students will apply for and be selected to participate in the recognition program, therefore additional student outputs include the number of students who apply for and participate in the recognition program.

Outputs related to teachers include the numbers that are enrolled in the NES project. NES expects that enrolled teachers will participate in the ePD related to the curriculum modules, and will then use the NES modules in their classrooms. These teachers will also complete the surveys related to the curriculum modules and NASA Now events used in the classroom. NES teachers will also demonstrate elements of the best practices identified by NES. A subset of NES teaches will also access the linked resources from the NES website. A subset of teachers will also apply for and some will attend the NES rewards opportunities related to the recognition program. Additional outputs include the number of teachers engaged in each of these NES components.

NES is designed so that participation may occur at the school level. Therefore, a school-level outcome is the number of schools with multiple teachers participating. These schools will demonstrate the integration of NES beyond a single classroom and the use of best practices. Additional outputs include the number of these schools that apply for and participate in the NES recognition program.

Short-Term Outcomes

If the theory of change underlying NES is correct and teachers and schools are able to correctly implement the program, then the inputs and activities will result in expected outcomes for students and teachers. Students should have more positive attitudes about STEM, increased self-efficacy in STEM areas, as well as increased interest in NASA STEM careers and educational opportunities, and increased access to other NASA opportunities. All of these outcomes are supported by the curriculum modules and NASA Now events, which are designed to create opportunities for students to engage positively with STEM around NASA themes, expose them to the inquiry process, and NASA content and careers. Students’ increased access to and participation in other NASA opportunities is further supported by the NES recognition program and access via participating teachers and schools.

Teachers themselves should be better able to implement and use NES products, have an increased familiarity and confidence with NES STEM materials and content, integrate these products into their curriculum, and use best practices relating to curriculum integration, student engagement, technology use, community outreach, and family involvement.

The NES materials were selected through a systematic process that was designed to identify high-quality materials that were appropriate for middle-school and high-school STEM classrooms, and that would be engaging to students. The NES virtual campus provides access to resources that support the implementation of NES. The professional development and helpdesks in particular are expected to provide the knowledge and skills to support teacher ability to use the materials and confidence with the implementation of these. Further, the NES social networking opportunities may help establish the foundation for NES communities of practice that will both support the implementation of NES as well as expand the reach of NES.

The NES recognition structure is designed to drive the use of best practices, both by identifying and making salient what these practices are, as well as by recognizing best practices that are exhibited among NES participants at the teacher, student, and school levels.

Long-Term Outcomes

The final premise underlying the NES project theory is that experiences with NES will lead to outcomes that reach further than the short-term outcomes described above. As teachers integrate the NES materials into their classrooms year after year, they expose additional classrooms of students to NES’s engaging materials. Increased interest and engagement among students would lead to their pursuit of additional NASA-related STEM experiences. Familiarity with a wider range of STEM careers and the importance and relevance of NASA missions would lead more students to pursue NASA-related STEM degrees and careers. Even among those individuals who do not pursue STEM careers, their familiarity with NASA and interest in STEM will continue throughout their lifetimes.

As designed, NES will continue to incorporate discovery into the project, and will add new content and increase its offerings. In this way, teachers may incorporate initial NES materials into their curriculum year after year, and add additional modules that are current and relevant to the content in their classrooms. Further, in collaboration with NEON a NES/NASA teacher community will develop to support the incorporation of NASA content and materials into STEM classrooms. NES will be incorporated into larger school-wide STEM efforts in schools where NES communities exist within schools. NES will also serve as a gateway for teachers to purse other NASA opportunities that they are exposed to via the recognition program, the virtual campus, social networking, and NES-provided linkages.

Contextual Factors

Finally, the model recognizes that there are contextual factors that may influence the implementation and related outcomes of the project. For example, for the project to work as intended, some other factors may need to be in place, such as high-quality and motivated teachers with the ability to understand and teach the materials that NASA offers. In addition, students’ ability to grasp the concepts taught may rely on their previous STEM knowledge and experiences in the classroom. Parental involvement, one of the project’s best practices because it reinforces student learning and interest in STEM, will not be present in all settings. Additional factors may include the school’s technological capability, instructional environment, and climate, as well as principal leadership and focus on instruction.

1 National Research Council. (2008). NASA’s Elementary and Secondary Education Program: Review and Critique. Committee for the Review and Evaluation of NASA’s Precollege Education Program, Helen R. Quinn, Heidi A. Schweingruber, and Michael A. Feder, Editors. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences Education. Washington, D.C. The National Academies Press.

2 The original NASA Explorer Schools (NES) project consisted of three-year partnerships between NASA and selected schools. The project focused on whole schools and provided financial investment, professional development, and curricular support designed to provide engaging student STEM educational experiences and sustained professional development, and to enhance family involvement in science education.