Administrator Interview Protocol

Administrator¹ Interview Protocol²

CPATH Project Title:

Grantee Institution(s):

Interviewee Name/Title/Contact Information:

Interview Date:

Interviewer(s):

Introductory Remarks:

My name is ______ and I work for SRI International, a non-profit research firm, who is responsible for the evaluation of the NSF CPATH Program. The purpose of this interview is to learn more about your CPATH project, how it is being implemented, the organizations and stakeholders involved, and the effect that it has been having on various groups and organizations. Beneficiaries of this project may include students, faculty members, administrators, institutions of higher education, K-12 school districts, professional associations, government and non-profit organizations, and businesses and industry. Throughout our discussion, please feel free to interrupt me to ask any questions you may have or include information that you believe we should know.

Before we start, I need to have you review and sign this consent form. SRI International’s Institutional Review Board (IRB) has approved this project as well as the U.S. government’s Office of Management and Budget. Your signature on this form indicates that your participation in this interview is voluntary and that you understand that we will never directly identify or quote you based on what you tell us today.

(After they review consent form, ask: Do you have any questions before we get started?

Introduction and Background

1. Please tell me about yourself. Probe for:

1. What are your roles and responsibilities?

2. How long have you worked at this institution?

3. What other positions have you previously held?

4. Describe your current role related to the CPATH project.

2. What is the history of undergraduate reform at your institution? Probe using the following:

   1. What do you view as pressing issues in improving undergraduate education?

   2. What are the current undergraduate reform initiatives on this campus?

   3. How did the CPATH project come about?

3. How do reforms related to computational thinking and the CPATH project fit in with your institution’s overall vision or mission? Probe using the following:

1. How important is this project to your institution?

Project Strategies

4. (1a, 1e, 3d) Please describe the nature of the teaching/learning environment for computing you are trying to create through the CPATH project. What is it designed to do at your institution (and other partner sites, if applicable)? Probe using the following:

1. How might this environment look different from traditional settings for computing?

2. How will you know you’ve been successful in creating this environment?

3. What are the core strategies you’re using to create this environment?

5. (1f, 1g) Who do you think are the primary beneficiaries of your project? Probe using the following:

1. Which groups of students are you targeting for participation?

2. Are you targeting any underrepresented groups for participation? If yes, please explain. (Probe for traditionally underrepresented groups including minorities, females, disabled and non-traditional computer majors.)

Implementation Factors

5. (1b, 2b) From your perspective, how do you think project implementation gone thus far? Probe using the following:

1. What have been the highlights or successes?

2. What failures has the project experienced? Please describe.

3. What barriers and challenges has the project faced? What do you think still needs to be done to address these challenges

5. (2e, 3c) What support do faculty at your institution have to help with curriculum development or pedagogical strategies for teaching computational thinking? Probe using the following:

1. Describe the current incentive and rewards structure at your institution.

2. What incentives are being offered through the CPATH project to faculty to develop curriculum, to devise pedagogical strategies and/or to participate in the project?

3. Are there any additional incentives to encourage faculty to develop innovative ways to teach computational thinking?

5. (1d, 1e, 2a) What are some of the factors that have supported project implementation? Probe using the following:

1. Culture of committed faculty involvement and participation

2. Strong institutional support from department head/deans/administrators

3. Innovative curricular and pedagogical strategies

4. Student demand for course changes

5. (3e) How has the institution leveraged funding to provide additional support to the project? Probe using the following:

1. What other ongoing initiatives are focused on computing reform?

2. What other projects, if any, do you envision emerging from this project? Please explain.

Outcomes

10. (2d, 3a) Overall, what influence has the CPATH project has had on students and faculty? Probe using the following:

1. How has the project influenced student enrollment in computing courses? Have there been any changes in the demographics or population of students enrolled in computing courses?

2. How has the CPATH project prepared students for STEM and other careers?

3. How has the project influenced faculty members? Has it led to changes in faculty culture?

4. Have faculty members published any articles related to computational thinking in peer-reviewed journals? (Note: this may or may not be related to the CPATH project)

10. (3b, 3c) Are there any institutional changes that you would attribute to the CPATH project? Please describe. Probe using the following:

1. Has the CPATH project integrated computational thinking into other disciplines?

2. Has the CPATH project influenced the rewards/incentive structure of your institution? If yes, please explain.

3. How might any institutional changes be documented? For example, have there been any changes in institutional documents (course catalogs, outlines of departmental majors, etc.) as a result of integrating computational thinking across disciplines? Probe for changes in the way majors are described, description of computing courses, and the role of computing in non-computing disciplines.

10. (3d, 3f) Do you think your CPATH project has created a model that could be used at other institutions? If yes, please describe the model and how you are supporting its implementation at other institutions. Probe using the following:

1. Are other institutions implementing your models?

2. Are you providing materials or guidance?

3. What mechanisms are in place to sustain the relationship between you and the institutions that have adopted your model?

Community Building and Partnership Development

10. (4a, 4b, 4c, 4d) Who are the stakeholders for the CPATH project? How is information shared with this community of stakeholders? Note: Stakeholders may include other CPATH grantees, faculty within computing-related disciplines (e.g., computer science, informatics), faculty in other disciplines, university administrators, other higher education institutions, K-12 teachers, professional associations, industry and businesses, and government and non-profit organizations.

Probe using the following:

1. How inclusive is this group of stakeholders?

2. Would you say there is a shared understanding about computing competencies among stakeholders in this community?

3. How do stakeholders communicate with and learn from each other? (How is the project supporting learning and sharing of best practices around computational thinking and education amongst these stakeholders?) Probe for online sharing and communication, conferences and colloquia, informal networks and other venues for sharing best practices.

10. (5a) Are there other organizations with which the project has a significant ongoing relationship as a part of the CPATH project? Probe using the following:

1. Post secondary institutions

2. K-12 school districts

3. Government offices

4. Private companies/non-profit organizations

5. Professional membership organizations

6. Any other groups that have had or will have a significant connection to the project.

(IF YES to Q14, then ask questions 15-16; IF NO, then go to Q17)

15. (5b,5c) Please describe these partnerships and how they came about. Probe using the following:

1. Has NSF funding (for the CPATH project) opened new opportunities for partnerships between multiple sectors (e.g. industry, K-12, professional associations) around computing? If yes, please explain.

2. To what extent have these partnerships leveraged pre-existing relationships or new opportunities in the local science, technology and economic development environment?

15. (3f, 5e) How can multi-sector partnerships support the replication of promising models of computational thinking over the long term? What are the barriers that would inhibit sustaining these partnerships?

Conclusion

15. Is there anything else you would like to share with us about your institution or the CPATH project?

Thank you for your time.

1 Administrators include department chairs, deans, provosts, or other university administrator familiar with CPATH grant activities.

2 Throughout this interview protocol, numbers and letters in parentheses—e.g., (1a, 3d) refer to research questions and are included as a tool for evaluation analysts.