Staff Interview Protocol

Staff¹ Interview Protocol²

CPATH Project Title:

Grantee Institution(s):

Interviewee Name/Title/Contact Information:

Interview Date:

Interviewer(s):

Introductory Remarks:

My name is ______ and I work for SRI International, a non-profit research firm, who is responsible for the evaluation of the NSF CPATH Program. The purpose of this interview is to learn more about your CPATH project, how it is being implemented, the organizations and stakeholders involved, and the effect that it has been having on various groups and organizations. Beneficiaries of this project may include students, faculty members, administrators, institutions of higher education, K-12 school districts, professional associations, government and non-profit organizations, and businesses and industry. Throughout our discussion, please feel free to interrupt me to ask any questions you may have or include information that you believe we should know.

Before we start, I need to have you review and sign this consent form. SRI International’s Institutional Review Board (IRB) has approved this project as well as the U.S. government’s Office of Management and Budget. Your signature on this form indicates that your participation in this interview is voluntary and that you understand that we will never directly identify or quote you based on what you tell us today.

After they review consent form: Do you have any questions before we get started?

Introduction and Background

1. Please tell me about yourself. Probe for:

1. What’s you academic background and previous experience?

2. How long have you worked at this institution?

3. Please describe your role and responsibilities on the CPATH project.

2. How did you become involved in the CPATH project? Probe using the following:

1. Posted job opening, invitation from PI, graduate student funding opportunity, personal interest, etc.

2. Prior to the CPATH project, were you engaged in undergraduate reform activities at your institution? If so, please describe those activities.

3. In your view, what are the principal goals of the CPATH project? Probe for understanding of concept of computational thinking.

Project Strategies

4. (1a, 3d) Please describe the nature of the teaching/learning environment for computing you are trying to create through the project. What is it designed to do at your institution (and other partner sites, if applicable)? Probe using the following:

1. How might this environment look different from traditional settings for computing?

2. How will you know you’ve been successful in creating this environment?

3. What are the core strategies you’re using to create this environment?

5. (1c) What are the core computing concepts that the CPATH project focuses on? Probe using the following:

1. How has participating in the CPATH program helped you identify and refine those concepts?

2. How would you define the concept of computational thinking?

6. (2a, 2b) How has the project integrated these core concepts into courses outside of traditional computing disciplines (i.e. biology, chemistry, mathematics, etc.)? Probe using the following:

1. What disciplines are involved?

2. What have been the challenges the project has faced in efforts to integrate these core concepts outside of traditional computing disciplines?

7. (1f, 1g, 3a) Who do you think are the primary beneficiaries of the project? Probe using the following:

1. Which groups of students are you targeting for participation?

2. Are you targeting any underrepresented groups for participation? If yes, please explain. (Probe for traditionally underrepresented groups including minorities, females, disabled and non-traditional computer majors.)

Implementation Factors

If project is at multiple campuses, probe for information on each participating institution using the probes provided with questions in this section.

8. (1b, 2b) How has project implementation gone thus far? Probe using the following:

1. What have been the highlights or successes?

2. What failures has the project experienced? Please describe.

3. What have been the challenges to the implementation of the project? Probe about the relevant actors – students, faculty, administrators, partners, etc. What have been the lessons learned from addressing these challenges?

9. (1d, 1e, 2a) What are some of the factors that have supported project implementation? Probe using the following:

1. Culture of committed faculty involvement and participation

2. Strong institutional support from department head/deans/administrators

3. Innovative curricular and pedagogical strategies

4. Student demand for course changes

10. (3e) How has the project leveraged funding from other sources to provide additional support to the project? Probe using the following:

1. What are the specific funding sources, if any (e.g., state, federal, private)?

2. What other ongoing initiatives are focused on computing reform?

3. What other projects, if any, do you envision emerging from this project? Please explain.

Outcomes

11. (2d, 3a) Overall, what influence do you think the CPATH project has had on students and faculty? As appropriate, probe using the following:

1. To what extent has the CPATH project influenced student engagement in computing? Please describe.

2. How has the project influenced student enrollment in computing courses? Have there been any changes in the demographics or population of students enrolled in computing courses?

3. How has the CPATH project prepared students for STEM and other careers?

4. How has the project influenced faculty members? Has it led to changes in faculty culture?

11. (3d, 3f) Do you think the CPATH project has created a model that could be used at other institutions? If yes, please describe the model and how you are supporting its implementation at other institutions. If appropriate, probe using the following:

1. Are other institutions implementing your models?

2. Are you providing materials or guidance?

3. What mechanisms are in place to sustain the relationship between you and the institutions that have adopted your model?

Community Building and Partnership Development

11. (4a, 4b, 4c, 4d) Who are the stakeholders for the project? How is information shared with this community of stakeholders? Note: Stakeholders may include other CPATH grantees, faculty within computing-related disciplines (e.g., computer science, informatics), faculty in other disciplines, university administrators, other higher education institutions, K-12 teachers, professional associations, industry and businesses, and government and non-profit organizations.

As appropriate, probe using the following:

1. How inclusive is this group of stakeholders?

2. Would you say there is a shared understanding about computing competencies among stakeholders in this community?

3. How do stakeholders communicate with and learn from each other? (How is the project supporting learning and sharing of best practices around computational thinking and education amongst these stakeholders?) Probe for online sharing and communication, conferences and colloquia, informal networks and other venues for sharing best practices.

11. (5a) Are there organizations with which the CPATH project has a significant ongoing relationship as a part of the CPATH project? Probe using the following:

1. Post secondary institutions

2. K-12 school districts

3. Government offices

4. Private companies/non-profit organizations

5. Professional membership organizations

6. Any other groups that have had or will have a significant connection to the project. Probe if there is a key person in this organization that we should talk to.

(IF YES to Q14, then ask questions 15-16; IF NO, then go to Q17)

11. (5b,5c) Please describe these partnerships and how they came about.

1. Has NSF funding (for the CPATH project) opened new opportunities for partnerships between multiple sectors (e.g. industry, K-12, professional associations) around computing? If yes, please explain.

2. To what extent have these partnerships leveraged pre-existing relationships or new opportunities in the local science, technology and economic development environment?

11. (5d) How much does the work done by each partner depend on the work of other members? As appropriate, probe using the following:

1. To what extent is the work shared equally among the partners?

2. How do you assess the effectiveness of the partnership? What measures are you using to make that assessment?

3. How would you describe any challenges to building and maintaining effective partnerships?

Conclusion

17. Is there anything else you would like to add that might help us get a better understanding of the CPATH project?

Thank you for your time.

1 Project staff could be any individual (either a part-time or full-time position) supported by the CPATH grant to assist the principal investigator with the management and administration of the project.

2 Throughout this interview protocol, numbers and letters in parentheses—e.g., (1a, 3d) refer to research questions and are included as a tool for evaluation analysts.

A-4