The NSF Eddie Bernice Johnson INCLUDES National Network Member Survey - Group C

***

The NSF Eddie Bernice Johnson INCLUDES National Network Member Survey

Introduction

The National Science Foundation (NSF) Eddie Bernice Johnson INCLUDES National Network (National Network) aims to support systems change efforts to broaden participation in Science, Technology, Engineering, and Math (STEM) career pathways and opportunities through a collaborative approach that brings together member resources, expertise, and connections.

This survey is an opportunity for you to provide feedback on your experiences with the National Network. This includes your perspectives on desired outcomes and ways in which the National Network is informing and supporting your efforts to change systems to broaden participation in STEM. This survey is administered by The NSF Eddie Bernice Johnson INCLUDES Coordination Hub (Coordination Hub), which provides cross-National Network leadership and coordination, and develops and disseminates resources across the National Network.

Your feedback will help improve National Network functioning, advance collaboration and shape National Network activities and the Coordination Hub’s initiatives in the future to best support National Network members.

Pursuant to 5 CFR 1320.5 (b), an agency may not conduct or sponsor, and a person is not required to respond to an information collection unless it displays a valid OMB control number. The OMB control number for this collection is 3145-0256. Public reporting for this collection of information is estimated to average 10 – 25 minutes per response, including the time for reviewing instructions.

Your participation in this survey is voluntary, and you may opt out of the survey at any time. While the survey is not anonymous, it is confidential; this means that any and all findings shared beyond the Coordination Hub will be reported in aggregate without any personally identifiable information (PII). If we want to use a direct quote from you or share any data from your organization beyond the Coordination Hub, we will ask permission first. We will also not report any statistics for groups with fewer than 10 responses and will destroy all PII at the end of the Coordination Hub’s current NSF award in August 2028.

ONE QUESTION PER PAGE UNLESS OTHERWISE NOTED

ALL QUESTIONS REQUIRE A RESPONSE UNLESS OTHERWISE NOTED

CONNECTION TO THE NETWORK

1. Which of the following best describes your current connection to the National Network?

I am part of …

[CLUSTER 1]

1. An INCLUDES Alliance

2. An INCLUDES Collaborative Change Consortia

[CLUSTER 2]

3. An INCLUDES Design and Development Launch Pilot (DDLP) project

4. An INCLUDES Early-concept Grants for Exploratory Research (EAGER) project

5. An INCLUDES Conference grant

6. An INCLUDES Planning grant

[CLUSTER 3]

7. An NSF-funded project that is not part of the INCLUDES initiative (including but not limited to Louis Stokes Alliances for Minority Participation (LSAMP), ADVANCE: Organizational Change for Gender Equity in STEM Academic Professions, Alliances for Graduate Education and the Professoriate (AGEP), Broadening Participation in Computing (BPC) Alliance, or Innovative Technology Experiences for Students and Teachers Resource Center (ITEST RC))

8. Another broadening participation in STEM project that is not funded by NSF

9. An NSF staff member or program officer

10. A staff member from another federal agency (not NSF)

11. A national, state, or local policymaker

12. A private funder or foundation program officer

13. Something else (e.g., part of another industry, professional association, nonprofit, advocacy organization) (OPEN END, VALIDATE RESPONSE)

14. I no longer consider myself connected to or participate in the National Network (SKIP TO Q30)

QX. In what state is the university, organization, or other institution that you are primarily affiliated with located? If you are not affiliated with a university, organization, or other institution please indicate where you are located. (INSERT DROPDOWN LIST OF STATES ALPHABETICALLY, including an N/A option)

[DROPDOWN OF STATES ALPHABETICALLY]

NETWORK ENGAGEMENT AND VALUE

There are various ways to engage with the National Network. For example, National Network members can attend NSF INCLUDES convenings and webinars, access resources, and connect with other members through and outside of the Online Community (www.includesnetwork.org) and NSF INCLUDES events. For the following questions, think about your experience in the National Network and its value to you and your work.

NOTE: The Coordination Hub’s working definition of equity is the “intentional act of creating policies, environments, and systems that disrupt inequities and seek to advance opportunities in partnership with historically marginalized people...”

NSF defines broadening participation as the “discovery and nurturing of talent wherever it may be found…[and] in terms of 1) individuals from underrepresented groups, 2) institutions and 3) geographic areas that do not participate in NSF research programs at rates comparable to others. “

If helpful, keep these definitions in mind when answering the following survey questions.

2. Reflect on the ways that you’ve engaged in the National Network in the last year.

Since the beginning of [CURRENT CALENDAR YEAR] I have… Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Attended Coordination Hub-organized events (e.g., in person and online conferences/convenings, virtual events, capacity building webinars) 

2. Engaged or communicated with other National Network members through the Online Community/www.includesnetwork.org (e.g., via affinity groups, member messenger function, creating or commenting on blog posts)

3. Engaged with other National Network members outside of the Online Community

4. Accessed National Network resources or content stored on the Online Community/www.includesnetwork.org (e.g., data, research, literature)

5. None of the above (EXCLUSIVE RESPONSE)

(DISPLAY Q3 – 5 ON ONE PAGE WHEN APPLICABLE)

3. [ASK OF ALL] The Coordination Hub wants to understand what engaging with the National Network has made possible for you. Reflecting on your NSF-funded project or the work that brought you into the National Network, how much do you agree with the following statements? 

Participating in the National Network has enabled me or my project to… 

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

[RANDOMIZE ITEMS]

1. Strengthen capacity (understanding and abilities) to approach systems change work in new ways 

2. Work with other National Network members whom I wouldn’t have had the opportunity to connect with otherwise

3. Stay informed about issues related to access in STEM fields 

4. Gain new knowledge and/or skills for our work or to better engage with our constituencies

5. Be inspired to take action toward our broadening participation goals

6. Be inspired to further contribute to the National Network

7. Do something else (please specify) (OPEN ENDED, VALIDATE RESPONSE)

[END RANDOMIZE]

4. [ONLY ASK Q4 IF Q1 = 1-6; CLUSTER 1 OR CLUSTER 2] Participating in the National Network has enabled me or my project to…

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

[RANDOMIZE ITEMS]

1. Develop the ability to articulate how my project fits within the goals of the National Network 

2. Feel a sense of ownership and responsibility to contribute to broadening participation in STEM fields  

3. Apply promising practices and strategies to make our work more effective 

4. Share lessons learned and promising practices/strategies through National Network events and communications (e.g., webinars, blogs, social media posts)

5. Gain visibility through INCLUDES events for collaboration with other National Network members 

6. Secure new and/or diversified funding to support our broadening participation work

7. Share something I learned from the National Network with another colleague in my organization/partnership

8. Provide new professional learning opportunities to our staff around diversity, equity and inclusion in STEM fields

[END RANDOMIZE]

5. [ONLY Q5 ASK IF Q1 = 1 OR 2; CLUSTER 1] Participating in the National Network has enabled me or my project to…

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

[RANDOMIZE ITEMS]

1. Explore new approaches, strategies, and activities  

2. Conduct new research to inform broadening participation work

[END RANDOMIZE]

(RESUME ASKING ALL)

6. The Coordination Hub also wants to learn about the extent to which network engagement is supporting the development of collective power within the National Network to catalyze equitable systems change in the STEM fields. How much do you agree with each of the following statements?

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

[RANDOMIZE ITEMS]

1. Participating in the National Network has allowed me to be part of a bigger movement to support access to STEM careers 

2. By participating in the National Network, members are able to accomplish more together than they can alone

3. Participating in the National Network is increasingly meeting my needs

[END RANDOMIZE]

7. [ONLY ASK Q7 IF Q1 = 1-6; CLUSTER 1 OR 2]

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

[RANDOMIZE ITEMS]

1. The National Network increasingly has the expertise it needs from members in different sectors (e.g., policy, business) to catalyze equitable systems change in the STEM fields

2. Leadership in the National Network is increasingly diverse and representative of broadening participation goals

3. I see others in the National Network holding themselves accountable in contributing to broadening participation outcomes

4. My project/work is aligned to the goal of the National Network to catalyze equitable systems change in STEM fields

5. The National Network has articulated policy priorities to influence broadening participation in STEM fields

[END RANDOMIZE]

[RESUME ASKING ALL]

8. The Coordination Hub is committed to providing high-quality and relevant support to National Network members. Do you have any suggestions for how the National Network could better support you in your work? (OPEN ENDED, DO NOT REQUIRE RESPONSE)

ROLE IN BP EFFORTS

9. Is your organization an institution of higher education?

1. Yes

2. No à SKIP TO Q11

10. [ONLY ASK Q10 IF Q9 = 1, YES] Is your institution one of the following minority-serving institutions? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Historically Black Colleges and Universities (HBCUs)

2. Hispanic-Serving Institutions (HSIs)

3. Tribal Colleges and Universities (TCUs)

4. Asian American and Pacific Islander Serving Institutions (AAPISIs)

[RESUME ASKING ALL]

11. Does your organization partner with one or more minority-serving institutions?

1. Yes

2. No

3. I don’t know à SKIP TO Q13

12. [ONLY ASK Q12 IF Q11 = 1, YES] If yes, which type of minority-serving institution? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Historically Black Colleges and Universities (HBCUs)

2. Hispanic-Serving Institutions (HSIs)

3. Tribal Colleges and Universities (TCUs)

4. Asian American and Pacific Islander Serving Institutions (AAPISIs)

[RESUME ASKING ALL]

13. Who is/are the focus population(s) for your broadening participation work? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Pre-K to 12 students

2. Undergraduate students

3. Graduate students

4. Pre-K to 12 teachers/faculty

5. Postsecondary faculty

6. K-12 schools/districts à IF ONLY, SKIP TO Q21

7. Postsecondary departments àIF ONLY, SKIP TO Q21

8. Employers or industry professionals à IF ONLY, SKIP TO Q21

9. Governmental agencies à IF ONLY, SKIP TO Q21

10. Individuals impacted by the carceral system à IF ONLY, SKIP TO Q21

11. Nonprofits or other community-based organizations à IF ONLY, SKIP TO Q21

12. Another focus population (please specify) (OPEN ENDED, VALIDATE RESPONSE) à IF ONLY, SKIP TO Q21

14. [ONLY ASK Q14 IF Q13 = 1, 2, OR 3; STUDENTS] Which of the following describes any student group you may be focusing on with your broadening participation in STEM work? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. African American or Black

2. American Indian or Alaska Native

3. Asian or Asian American

4. Hispanic, Latinx, or Spanish origin

5. Middle Eastern or North African

6. Native Hawaiian or Pacific Islander

7. White

8. Prefer not to answer

9. First-generation college students

10. Low-income students

11. Students with disabilities

12. Students who have been impacted by the carceral system

13. Women and people who are gender diverse

14. Another student group (please specify) (OPEN ENDED, VALIDATE RESPONSE)

15. [ONLY ASK Q15 IF Q13 = 4 OR 5; FACULTY] Which of the following describes any focus pre-K to 12 and/or postsecondary faculty group you may be addressing? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Women and people who are gender diverse

2. People of color

3. New faculty/teachers

4. Individuals with disabilities

5. Another faculty group (please specify) (OPEN ENDED, VALIDATE RESPONSE)

16. [ONLY ASK IF Q13 = 1-5; STUDENTS OR FACULTY] Which of the following goals does the broadening participation in STEM initiative that you are affiliated with address? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Equitable access to high-quality STEM education programs

2. Success in STEM coursework

3. Increasing awareness of STEM education/career opportunities

4. Representational diversity of those participating in STEM education opportunities

5. Representational diversity in STEM careers

6. Equitable institutional programs, policies, and practices that support access

7. Alignment of career education and pathway opportunities

8. Creation or change of policies to support access to and broadening participation in STEM education and/or careers

9. An inclusive culture in STEM workplaces and/or educational settings

10. Another broadening participation goal (please specify) (OPEN ENDED, VALIDATE RESPONSE)

[RESUME ASKING ALL]

PROJECT INFORMATION

[ONLY ASK Q17-20 IF Q1 = 1- 6; CLUSTER 1 OR 2]

The following questions about your NSF INCLUDES project will help the Coordination Hub support the scope and scale of your work and your partnerships with others in the National Network. For the following questions, think of “partners”, as any independent contributors and/or organizational partners you are currently working with in your NSF INCLUDES-funded project.

17. Which of the following categories best describes your role in your NSF INCLUDES-funded project?

1. Principal investigator/co-principal investigator

2. Evaluator

3. Backbone Member

4. Alliance or project team member

5. Another role (please specify) (OPEN ENDED, VALIDATE RESPONSE)

18. Under your funded award, how many partners are engaged in implementing the project?

1. 0-5

2. 6-10

3. 11-20

4. More than 20
   
   

19. Of your project partners articulated in the previous question, what percentage are new as a result of participating in the National Network?

1. All are new partners

2. More than 75% are new

3. About half are new

4. Less than 25% are new

5. None are new partners

20. What kinds of groups are your project partners members of? Please select all that apply (ACCEPT MULTIPLE RESPONSES)

1. Higher education institutions

2. Pre-K to12 schools/districts

3. Nonprofit and/or community-based organizations

4. Professional associations or networks (outside of the National Network)

5. Local or state government agencies

6. Federal government agencies other than NSF

7. Accrediting agencies

8. Funders

9. Museums/science centers

10. Business/industry/trade organizations

11. Federally funded research and development centers (FFRDCs) (e.g., National Laboratories, Jet Propulsion Lab, National Radio Astronomy Observatory)

12. Another type of partner (please specify) (OPEN ENDED, VALIDATE RESPONSE)

[COLLABORATIVE INFRASTRUCTURE]

[RESUME ASKING ALL]

Foundational to the INCLUDES National Network’s approach is collaborative infrastructure, which is a model that includes five elements: shared vision; partnerships; leadership and communication; goals and metrics; and expansion, sustainability, and scale. A goal of the National Network is for all its members to operationalize these elements to accelerate and scale systems level change to make STEM fields more equitable and accessible for people who have been marginalized and minoritized by STEM systems.

In the following section, the Coordination Hub seeks to understand the extent to which different National Network members are engaging in collaborative infrastructure approaches. In answering the following items, please reflect either on your NSF-funded project or the work that brought you to the INCLUDES National Network. For the items that don’t apply to your work, please select “Not applicable.”

21. Within my NSF-funded project/ work to broaden participation in STEM…

SCALE:

1. Strongly disagree

2. Disagree

3. Agree

4. Strongly agree

5. I don’t know

6. Not applicable

All partners who are funded within my award/all my project team members:

1. Agree on the goal(s) of our project

2. Agree on the solution(s) needed to address our project goal(s) 

All project leads funded within my award/in my project team:

3. Agree on the problem(s) that our work is aiming to solve

4. Agree on the root causes of our work 

In my funded project/project team:

5. We have a diversity of partners (e.g., that reflect different expertise, knowledge and perspectives) 

6. We continually address power dynamics with our project partners/within our project team (e.g., maintain open, consistent dialogue; engaging in self-reflection; communicate through disagreements) 

7. We work equitably (e.g., assign roles according to an individual's expertise, interests, and capacity and clearly communicate role expectations and responsibilities, have transparent decision-making structures)

8. We create opportunities to collaborate with people most affected by STEM inequities (e.g., they participate as advisors, offer feedback on strategies/activities, share in decision-making)  

9. We share resources (e.g., promising practices/strategies, data, lessons learned) with others (e.g., the National Network, the field, our project partners)

10. We have dedicated expertise to measure progress on project outcomes 

11. We use shared data to track progress on our goals  

12. We collect data that are used to inform continuous improvement (e.g., of strategies, activities) 

13. Project lead(s) can articulate how our project’s impact is contributing to equitable systems change (e.g., shifts in policies, practices, resources) 

14. We disaggregate data by different factors (e.g., race, gender, abilities, geography) 

15. We foster relationships with different sectors (e.g., philanthropy, nonprofits, business, government) to secure funding for the sustainability of our work

16. We have plans to sustain our work after our NSF funding ends  

17. We adopt and adapt promising practices from the National Network to scale our work  

18. We amplify promising practices and lessons to people outside of the National Network   

19. We focus our activities on achieving equitable systems change (defined as changing policies, programs, practices, mindsets, and other constructs that hold inequities in place) 

SYSTEMS CHANGE

22. The Coordination Hub also wants to learn about the extent to which National Network members are contributing to changing the systems and structures that limit access to STEM fields. One way to understand system change is through the Water of System Change framework, developed by global nonprofit consulting firm, FSG, which describes six conditions for systems change. While there are many possible frameworks that could be used, the Coordination Hub is using this particular framework—which has been adopted more broadly within NSF—to help create shared language to support collaborative work.

The Water of Systems Change framework outlines six conditions of systems change: structural change of policies, practices, and resource flows; relational change of relationships and connections and power dynamics; and transformative change of mental models. Please reflect on the work that brought you to the National Network and share how much progress you have made in the following areas. 

SCALE:

1. We have not made any progress but plan to

2. We have made some progress

3. We have made a lot of progress

4. We have achieved our goals

5. I don’t know

6. Not applicable

1. Policy: Changing government, institutional, or organizational rules, regulations, and/or priorities that guide your entity’s and others’ actions in a manner that is equitably broadening participation of underrepresented students in STEM.

2. Practices: Changing activities, procedures, guidelines, or information-sharing habits in a manner that is equitably broadening participation of underrepresented students in STEM

3. Resource Flows: Changing how money, people, knowledge, information, and/or other assets such as infrastructure are allocated and distributed in a manner that is equitably broadening participation of underrepresented students in STEM 

4. Relationships and Connections: Changing the quality and kinds of connections and communications among actors in the systems, especially among those with differing histories and viewpoints, in a manner that is equitably broadening participation of underrepresented students in STEM

5. Power Dynamics: Changing the distribution of decision-making power, authority, and formal and informal influence among individuals and organizations in a manner that equitably broadens participation of underrepresented students in STEM

6. Mental Models: Changing deeply held beliefs and assumptions, and taken-for-granted ways of operating that influence how we think, what we do, and how we talk, in a manner that is equitably broadening participation of underrepresented students in STEM 

23. If you prefer to share any examples via interview instead of completing open-ended questions, please share your contact information here: (OPEN ENDED, VALIDATE EMAIL ADDRESS)

24. [ONLY ASK IF Q22A = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes to policies and why this shift will support more equitable participation in STEM. (OPEN END, DO NOT REQUIRE RESPONSE)

25. [ONLY ASK IF Q22B = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes to practice and why this shift will support more equitable participation in STEM. (OPEN END, DO NOT REQUIRE RESPONSE)

26. [ONLY ASK IF Q22C = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes made to resource flows and why this shift will support more equitable participation in STEM. (OPEN END, DO NOT REQUIRE RESPONSE)

27. [ONLY ASK IF Q22D = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes made in connections and communications and the types of people with differing histories and viewpoints who were included as a result. (OPEN END, DO NOT REQUIRE RESPONSE)

28. [ONLY ASK IF Q22D = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes made to power, authority, and influence and the types of people with differing histories and viewpoints who were included in decision making as a result. (OPEN END, DO NOT REQUIRE RESPONSE)

29. [ONLY ASK IF Q22F = 2 – 4; MADE AT LEAST SOME PROGRESS] Please describe the changes made to beliefs and assumptions and the types of actions related to equitable participation in STEM that shifted as a result. (OPEN END, DO NOT REQUIRE RESPONSE)

FORMER NETWORK MEMBER

30. (ONLY ASK IF Q1 = 14; NOT CONNECTED TO THE NETWORK) Please explain why you no longer consider yourself connected to or participate in the National Network. (OPEN ENDED)

31. (ONLY ASK IF Q1 = 14; NOT CONNECTED TO THE NETWORK) Please explain how the Coordination Hub can best support you in continuing to participate in the National Network. (OPEN ENDED)

THANK YOU AND END

We thank you for your time spent taking this survey.

END OF SURVEY

  

15