Crosswalk Between Evaluation Questions and Site Visit Protocol Questions

APPENDIX B

CROSSWALK BETWEEN EVALUATION QUESTIONS AND

SITE VISIT PROTOCOL QUESTIONS

Evaluation Questions	PI /co-PIs	Staff		Faculty		Students		Administrators	Partners
1) How is the CPATH program (a) infusing computational thinking into a wide range of disciplines serving undergraduate education and (b) reaching a wide range of students to prepare them for STEM careers?
What are the curricular and pedagogical models that have been created and developed through this program?	4	4		4		1,2		4	3
How, and to what extent, are the individual CPATH projects being implemented as planned?	8	8		8				6
How has the CPATH program supported the formal identification and specification of the core elements and competencies of computational thinking?	5	5		5		6			5
What are the factors that have supported successful implementation of strategies?	10	11		10				8
What do institutional leaders understand about the goals of the CPATH program and project(s)?	9,10	9		10				4,8
Who is benefitting from the projects? Students? Faculty? Departments?	7	7		7		3,7		5	4
How successful has the program been in targeting and engaging traditionally underrepresented groups (i.e., minorities, females, disabled and non-traditional computer majors) in classes or programs that promote computational thinking?	7	7		7		5		5,11
2) What is the evidence that university and community college departments and faculty are integrating computational thinking into their courses?
What have been the program’s successes and highlights across projects in integrating best practices in computational thinking into courses across disciplines? What patterns do these indicate?	6,10	6		6		4,5,7		8	6
What have been the program’s barriers and challenges across projects to infusing computational thinking across disciplines? What patterns do these indicate?	6, 8	8		6,8				6	7
What do institutional documents (course catalogues, outlines of departmental majors, etc.) reveal about how IHEs are adapting to change by integrating computational thinking across the disciplines?								12
To what extent do faculty within university and community college departments produce educational scholarship to support grassroots efforts to promote computational thinking in and across the undergraduate curriculum?	12			11				10
What are the different social and resource supports within departments available to faculty bolstering their curricular reform efforts around computational thinking?	9			9				7
3) What is the evidence that the program is supporting the development of promising models of institutional change?
How is the program supporting increases in student enrollment and course taking in computing over time?	12	7		11		5,8		10
How is the program catalyzing institutional change through faculty and administrative leadership governance in IHEs?	11,13	10		12				11
To what extent has the program influenced faculty culture and the rewards/incentive structure of IHEs?	9,11, 13	10		9, 12				7,11
What are some examples of promising models developed for infusing computational thinking across disciplines and institutions?	4,14	4		13				4,12	3,8
How have grantee institutions leveraged grant resources to support and sustain their reform models over time?	11	12						9
How are promising models sustained and replicated in new institutional settings?	14			13				12,16	15
4) What is the evidence that the program is developing communities of practitioners (among the different program stakeholders) that regularly share best practices across communities?
How is the program supporting community building to share best practices in computational thinking?	15		14				13		9
How has the program promoted the continued growth of the community of practitioners outside of the formal CPATH grantees?	15	13	14				13		14
How have the program’s grantees contributed to a common understanding of computing competencies among the different stakeholder groups?	15		14				13		10
How is the program supporting the inclusion of different stakeholders in these learning communities around computational thinking and education?	15		14				13
What role has the program played in promoting shared learning about computational thinking among industry and professional/disciplinary associations?									10
To what extent is the program creating strong links and synergy among the grantees involved in collaborative activities?	20								11
5) How has the CPATH program promoted sustainable multi-sector partnerships that represent a broad range of stakeholders (i.e. industry, higher education, K12)?
To what extent have the program’s grantees created opportunities to develop multi-sector partnerships around computational thinking?	17	14	15				14		12
How has the program supported pre-existing relationships between different sectors and promoted further buy-in to develop strong alliances around computational thinking?	18	15	16				15		13
How has the program shaped the goals and theories of change of the different partnerships supporting computing education?	18	15	16				15		13
What has the program done to define the roles, processes, and outcomes generated by the partnerships it supports?	19	16	17						14
What has the program done to sustain multi-sector partnerships that hold promise for infusing computational thinking throughout the field?	20						16		15