Supporting Statement Part A. JUSTIFICATION
Title of Collection: The NSF Eddie Bernice Johnson INCLUDES National Network Survey
OMB Number: 3145-0256
Expiration Date of Approval: August 31, 2024
Type of Request: Intent to seek renewal of an established information collection.
A.1. Necessity of the Information Collection
The proposed information collection, The National Science Foundation (NSF) Eddie Bernice Johnson Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (INCLUDES) Initiative National Network Survey, is a renewal of an established information collection request. The data to be collected by the proposed survey are not available elsewhere. The information collection activities are planned to start immediately after OMB approval is received and will be conducted annually through the end of the project. NSF is requesting OMB approval for The NSF Eddie Bernice Johnson INCLUDES Coordination Hub (“Coordination Hub”, an NSF awardee) to collect information from members of The NSF Eddie Bernice Johnson INCLUDES National Network (National Network) on their experiences in the National Network and with the Online Community (i.e., website, www.includesnetwork.org).
SRI, HCM Strategists (HCM), Quality Education for Minorities (QEM) Network, and WestEd, are partnering to form the Coordination Hub, with ORS Impact (ORS) serving as the formative evaluation team. The Coordination Hub leads and supports the National Network (see the detailed description of the National Network in Section A1.1) by brokering connections among members; elevating best practices within the National Network and field; and working with National Network members to improve collaborative efforts aimed at enhancing the preparation, participation, and contributions of the full spectrum of diverse talents in science, technology, engineering, and mathematics (STEM). The information from members completing the National Network Survey will help to provide the Coordination Hub and NSF with a better understanding of the health, development, and expansion of the National Network.
The National Network Survey is designed as a time-series instrument to collect data from National Network members and provide aggregated data about the National Network as a whole. Currently, membership is defined as anyone registered for the Online Community. For the purposes of this information collection, anyone who is registered for the Online Community for at least six months prior to survey administration and has not updated their subscription settings to “no email” will be contacted to participate.
The National Network survey is part of the developmental evaluation of the Coordination Hub’s work over the next five years and will (1) shape the Coordination Hub’s activities (e.g., to identify support needs in the coming year; inform Shared Measures and National Network communication, engagement, learning, and community building; and guide strategy and priorities of the National Network’s expansion goals); (2) assess the development and progress of the National Network; and (3) inform the Coordination Hub’s assessment of progress toward its theory of change. While this survey was designed specifically as part of the Coordination Hub’s internal developmental evaluation, data collected is used for other purposes, and is shared in the aggregate with NSF and with National Network members themselves to help them track the evolution of the work and growth of the National Network.
Based in part on findings from the first three years of survey administration (2021, 2022, and 2023), the Coordination Hub has progressively learned more about National Network members’ needs and the continuum of participation. This Coordination Hub is applying these learnings and working to operationalize a new vision and implement a new model for the National Network, which has resulted in the need to adjust the National Network Survey. In collaboration with the Coordination Hub, ORS has revised the survey to reflect those adjustments. These revisions align with the current vision and model for the National Network and advance responsive programming from the Coordination Hub, while streamlining questions to reduce respondent burden.
Survey items, such as information on affiliation with the National Network, project goals and area of focus, and perceived impacts of the National Network on respondents’ efforts, have largely stayed the same. Questions regarding participation and activities have been significantly shortened. Questions relating to the use of aspects of collaborative infrastructure to advance project work will now only be asked of respondents who are funded by NSF INCLUDES. Questions about motivations and capacity building needs have been removed. Questions to better understand system change approaches and impacts of respondents have been added.
A1.1 Background of NSF INCLUDES
NSF INCLUDES is a comprehensive national initiative designed to enhance U.S. leadership in STEM discoveries and innovations by focusing on broadening participation in these fields at scale. The initiative supports the development of a robust network of partnerships among stakeholders across multiple sectors (public, private, academic). This National Network, has a registration list of over 4,000 people representing:
NSF INCLUDES-funded projects.
Other projects funded by NSF that benefit from and contribute to the broadening participation resources available to the National Network. These include, but are not limited to, projects supported by Louis Stokes Alliances for Minority Participation (LSAMP), Engineering Research Centers (ERCs), Alliances for Graduate Education and the Professoriate (AGEP); ADVANCE: Organizational Change for Gender Equity in STEM Academic Professions (ADVANCE), Historically Black Colleges and Universities Undergraduate Program (HBCU-UP), Improving Undergraduate STEM Education (IUSE); Broadening Participation in Computing (BPC) Alliances, Advancing Informal STEM Learning (AISL), Innovative Technology Experiences for Students and Teachers (ITEST), Discovery Research PreK-12 (DRK-12) programs; and engineering education projects such as the Women in Engineering Proactive Network and NSF Engineering Research Visioning Alliance.
Federal Coordination in STEM (FC-STEM) agencies.
Broadening participation research efforts.
Organizations that support the development of talent from all sectors of society to build an inclusive STEM workforce.
The National Network is open to anyone interested in the goals of Broadening Participation in STEM, and is purposefully designed to have a low barrier to entry for anyone who would like to participate. However, participation in the National Network is required of NSF INCLUDES-funded projects and strongly encouraged for other related projects/organizations that work toward similar goals.
As mentioned in Section A.1, the National Network is lead and supported by the Coordination Hub with the vision of helping to make the National Network a go-to entity for people from a wide range of constituency groups to learn about strategies, practices, and models to broaden participation in STEM. In service of the Coordination Hub’s vision for the National Network, the Coordination Hub (1) provides useful supports so National Network members can have a greater impact in their broadening participation projects, (2) learns from collective wisdom, experiences, voices, and data across National Network members, and (3) shares insights, stories, and evidence of the National Network’s impact.
A.2.
Purposes and Uses of the Data/Information
As
described in Section A.1, the National Network survey is part of the
developmental evaluation of the Coordination Hub’s work over
the next five years and will (1) shape the Coordination Hub’s
activities (e.g., to identify support needs in the coming year;
inform Shared Measures and National Network communication,
engagement, learning, and community building; and guide strategy and
priorities of the National Network’s expansion goals); (2)
assess the development and progress of the National Network; and (3)
inform the Coordination Hub’s assessment of progress toward its
theory of change.
This is a renewal of an established information request. The original information request was a time-series survey designed to provide developmental input on the Coordination Hub’s operating model and reporting relevant data to NSF. The survey articulated in this information request is modification of that time-series instrument. It has been adjusted to align with the new vision and model for the National Network described in Section A.1. The information collected through the revised survey may be applicable to supplement the external evaluation of the National Network, the work of National Network members themselves, or other related purposes.
A2.1 Who Will Use the Information
The information collected will primarily be used by the Coordination Hub to understand the utility of the National Network in supporting individual project successes, and for informing design decisions the Coordination Hub will make regarding future programming and support provided to National Network members, as part of internal developmental evaluation and the Coordination Hub’s strategic learning. This information will be used by the Coordination Hub to refine its activities in support of the National Network and to track the evolution of the National Network. Aggregated findings will be shared with NSF and the National Network. NSF will use aggregated findings in presentations and learning opportunities with directorates and offices within NSF, as well as in meetings with constituencies, such as the Committee on Equal Opportunities in Science and Engineering (CEOSE) and the National Science Board (NSB). The Coordination Hub will use aggregated findings and insights for the purposes outlined above in section A.2. National Network members may use the data for their own project purposes, as applicable.
A.3. Use of Information Technology
The Coordination Hub and ORS take the responsibility to minimize burden on respondents very seriously and have revised the original survey accordingly. The information will be collected using web-based survey technology (i.e., Qualtrics), which provides an efficient mechanism to capture responses and eliminates the time required for respondents to return completed surveys by mail. Electronic administration enables ORS to send reminders only to non-respondents, supporting improved response rate. Skip patterns in the web-based survey move respondents past questions that are not relevant, based on responses to previous items, which minimizes burden on the respondent. The electronic survey, and all related communication, will be compliant with Section 508 of the Rehabilitation Act.
Information
technology will also be employed in requests to complete the survey,
support of non-response follow up, and distribution of survey
findings. De-identified and aggregated results will be shared
annually via the Online Community, along with other virtual sessions
and in-person meetings as applicable.
A.4. Efforts to Identify Duplication
There are no other data collection efforts implemented by the Coordination Hub that provides this information on National Network members’ experiences in the National Network and the Online Community. NSF routinely collects information from funded projects through designated reporting requirements for each grant such as annual, semi-annual, evaluation, and final reports . However, these reports are controlled unclassified information and are not made available to the Coordination Hub. These also do not include questions that are part of the proposed survey capturing the affiliation, experiences, and reported impacts of each individual National Network member (as opposed to each project), or their perception of the value of the National Network. Additionally, grant reporting and other data collection sources do not include perspective of members who are not part of NSF-funded projects. As such, there is no similar ongoing data collection being conducted that duplicates this proposed effort.
A.5. Methods Used to Minimize Burden on Small Businesses
There may be small business entities involved in this data collection; however, burden is limited to voluntary participation in the survey.
A.6. Consequences of Less Frequent Data Collection
This
information obtained via this voluntary annual data collection is
essential to provide the Coordination Hub and NSF with quantitative
and qualitative measures to gain valuable insights into the
development and health of the National Network and ways in which the
Coordination Hub can better serve National Network members.
Conducting the collection less frequently would impede the
Coordination Hub’s and NSF’s ability to track the
evolution of the National Network and responsively adjust supports
and programming, and deprive National Network members, their
partners, and other interested parties of an opportunity to learn
more about the contributions of the National Network to broadening
participation in STEM careers.
A.7. Special Circumstances
The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6 (Controlling Paperwork Burden on the Public-General Information Collection Guidelines). There are no special circumstances that require deviation from these guidelines.
A.8. Consultations Outside the Agency
NSF received no public comments from the Federal Register notice dated December 21, 2023.1
A.9. Payments or Gifts to Respondents
No payments or gifts will be given to respondents.
A.10. Assurance of Confidentiality
The Coordination Hub has measures, detailed below, in place to address and maintain confidentiality of information collected and that respondents understand these provisions.
IRB Review: The Coordination Hub is submitting the survey for institutional review board (IRB) review by HML IRB Research and Ethics, a third party. The IRB human subjects review will assess:
the purpose of the data collection and data collection methods being used;
the respondent populations and how they will be identified and accessed;
whether the data will be anonymous, confidential, or neither, and if the data are confidential or neither, explanation of why identifiers are necessary;
how data will be stored (e.g., electronic files, hard copies);
who will have access to data and for how long; and
potential risks and burdens of the project to participants.
Voluntary Participation: In all communications about the survey, as well as in the instructional materials for the survey itself, respondents will be reminded that participation is voluntary, they can opt out at any time.
Removal of Personally Identifiable Information (PII): Individual responses will not be reported except in aggregate and without PII. Findings will not contain names, addresses, and organizational affiliation that could permit disclosure or identification of respondents directly or by inference. Any information shared outside of the Coordination Hub will not link a participant’s responses to their individual or organizational identities. All PII will be destroyed at the end of the Coordination Hub’s funding period.
Aggregated Reporting: Results will be reported only in aggregate form as written summaries, counts, data visualizations, frequencies, etc. rather than in any individual response format. No statistics for disaggregated groups with fewer than 10 responses will be reported.
A.11. Justification for Sensitive Questions
The survey does not include sensitive questions. PII collected will include individuals’ affiliated organizations and home states. These PII are collected to categorize responses based on respondents’ roles in the National Network, track responses to avoid burdensome reminders, and understand response rates and representation. PII will be accessed only by the Coordination Hub. Any public reporting of data will be in aggregate form, and any PII will be removed.
A.12. Estimates of Hour Burden to Respondents/Table
Table 1 provides burden estimates for approximately 4,000 survey respondents. Skip logic is used in the survey to for three distinct groups of respondents, with Group A receiving all 29 questions, Group B receiving 28 questions and Group C receiving 22 questions. For more information on the composition of each group, see Section B.1.
The table below reflects the maximum estimated burden of the survey, if all National Network members (i.e., all those who have registered for the Online Community, includesnetwork.org, at least six months prior to the survey administration and have not adjusted their subscription settings to “no email”) respond to the survey.
Table 1. Maximum Burden Estimates for Survey Respondents
Type of Collection |
Category of Respondent |
No. of Respondents |
Participation Time (minutes) |
Annual Burden Estimate (hours) |
National Network Survey: Group A (survey in its entirety*) |
NSF INCLUDES-funded Alliances or Collaborative Change Consortia participating in the National Network |
680 |
25 (max) |
283 |
National Network Survey: Group B (excluding Q5) |
Other NSF INCLUDES-funded projects listed participating in the National Network, including but not limited to:
|
680 |
25 (max) |
283 |
National Network Survey: Group C (excluding Q4-5, 7, 17-20) |
All other National Network members including, but not limited to:
|
2640 |
10 (max) |
440 |
*Q30 and Q31 of the survey are only asked of those who “no longer consider [themselves] connected to or participate in the National Network,” and are excluded from the definition of “survey in its entirety” for timing and other purposes |
||||
Total Estimated Respondent Burden Hours Annually |
1006 hours |
|||
A.13. Estimates of Cost Burden to Respondents
Cost estimates are based on average associate professor salary as reported by the American Association of University Professors (https://www.insidehighered.com/news/2018/04/11/aaups-annual-report-faculty-compensation-takes-salary-compression-and-more). The average associate professor salary is $81,274 annually/$40.48 hourly.
For respondents in Group A (n=680), burden is estimated to be 25 minutes maximum per respondent, yielding the estimated total cost of $11,444 or $16.83 per respondent, if all possible respondents participate.
For respondents in Group B (n=680), burden is estimated to be 25 minutes maximum per respondent, yielding the estimated total if $11,444 or $16.83 per respondent, if all possible respondents participate.
For respondents in Group C (n=2640), burden is 10 minutes per respondent, yielding the estimated total cost of $17,714 or $6.71 per respondent, if all possible respondents participate.
A.14. Estimates of Cost to Federal Government
This
effort is accounted for in the Coordination Hub’s award. In the
award budget, the Coordination Hub allotted $40,000 for design and
$10,500 for yearly administration of the survey, or $151,500 total
over the three-year OMB control period.
A.15. Reason for Program Changes or Cost Adjustments
This
is a renewal of an existing information collection request, and the
number of respondents and burden estimates reflect what was learned
from the first approval cycle. We have better defined the projects
for surveying and the respondents.
A.16. Project Plan and Timeline
The National Network Survey will be administered annually starting in 2024. The survey will be deployed via Qualtrics, an online survey platform. Data from the survey will be analyzed descriptively using frequencies and measures of central tendency and dispersion to show patterns. Aggregated results will be shared with the National Network annually within six months of the completion of data collection, beginning in 2025.
A.17. Request to Not Display Expiration Date
The Coordination Hub will display the expiration date of OMB approval and OMB approval number on all instruments associated with this information collection, including forms and questionnaires.
A.18. Exceptions to the Certification
No exceptions are necessary for this information collection.
1Federal Register :: Agency Information Collection Activities; Submission to the Office of Management and Budget for Renewal; Comment Request; NSF's Eddie Bernice Johnson INCLUDES Initiative National Network Survey
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rebecca Valenchis |
| File Modified | 0000-00-00 |
| File Created | 2024-09-06 |