NASA STEM Engagement K-12 Stakeholders Needs Assessment SHORT FORM

REQUEST FOR APPROVAL under the Generic Clearance for NASA STEM Engagement Performance Measurement and Evaluation, OMB Control Number 2700-0159, expiration 09/30/2024

_____________________________________________________________________________________

1. TITLE OF INFORMATION COLLECTION:

NASA National K-12 Stakeholders Survey

2. TYPE OF COLLECTION:

	Attitude/Behavior Scale
	Baseline Survey
	Cognitive Interview Protocol
	Consent Form
	Focus Group Protocol
	Follow-up Survey
	Instructions
	Satisfaction Survey
	Usability Protocol

GENERAL OVERVIEW: NASA Science, Technology, Engineering, and Mathematics (STEM) Engagement is comprised of a broad and diverse set of programs, projects, activities and products developed and implemented by HQ functional Offices, Mission Directorates and Centers. NASA’s Office of STEM Engagement (OSTEM) along with NASA’s Mission Directorate Offices deliver participatory, experiential learning and STEM challenge activities for young Americans and educators to learn and succeed. NASA STEM Engagement seeks to:

• Create unique opportunities for students and the public to contribute to NASA’s work in exploration and discovery.

• Build a diverse future STEM workforce by engaging students in authentic learning experiences with NASA people, content, and facilities.

• Strengthen public understanding by enabling powerful connections to NASA’s mission and work.

To achieve these goals, NASA STEM Engagement strives to increase K-12 involvement in NASA projects, enhance higher education, support underrepresented communities, strengthen online education, and boost NASA's contribution to informal education. The intended outcome is a generation prepared to code, calculate, design, and discover its way to a new era of American innovation.

However, the beneficiaries of NASA’s investments and work in STEM engagement are students at the grades K-12, undergraduate, and graduate levels. To serve K-12 students, NASA STEM Engagement employs the following three strategies:

1. Develop and deploy evidence-based opportunities to engage students in NASA-unique learning experiences beyond the classroom.

2. Enhance student STEM experiences in schools and other educational venues, using evidence-based strategies, NASA STEM practitioners, learning opportunities, content, and resources.

3. Create and deploy authentic learning experiences and research opportunities for students to bolster their STEM studies and stimulate further interest and achievement.

The NASA National K-12 Stakeholders Survey for this information collection is to gather external stakeholder perceptions regarding NASA STEM Engagement and to learn more about external experiences regarding using NASA STEM engagement K-12 activities to spark student interest in STEM.

3. INTRODUCTION AND PURPOSE: The NASA National K-12 Stakeholders Survey is comprised of demographic items, experience with NASA resources items, NASA awareness and perceptions items, and future of STEM engagement opportunities items. Our interest is to measure to determine stakeholder perceptions of strengths and weaknesses of NASA STEM Engagement K-12 activities/resources, challenges to using activities/resources, and thoughts regarding needs, gaps, and suggestions for future programming. Thus, the purpose for testing is to validate the instrument and its reliability to examine stakeholder perceptions regarding NASA STEM Engagement K-12 activities’: a) areas of strength, b) areas of weakness, c) challenges, and d) potential future opportunities.

1. RESEARCH DESIGN OVERVIEW: NASA’s work in STEM Engagement is focused on serving students. It is recognized that providing support and resources to educators and educational institutions is vital to effectively engage students. This study will examine the impact of NASA STEM Engagement K-12 activities on sparking student interest in STEM. In this study, the experiences of stakeholders who have utilized NASA STEM K-12 activities and/or resources will be investigated with the goal of revealing important evidence on the fidelity of implementation of NASA content, as well as the perceived outcomes regarding student interest in STEM.

Quantitative data will be summarized using descriptive statistics such as numbers of respondents, frequencies and proportions of responses, average response when responses categories are assigned to a Likert scale (e.g., 1 = “Never Used” to 4 = “Used Every day”), and standard deviations. Emergent coding will be used for the qualitative data to identify the most common themes in responses.

Descriptive survey item analysis. Descriptive statistics are used to describe survey participant responses to individual items that do not form a construct or scale. Charts or tables with average participant responses and/or frequencies and percentages will be reported. Percentages of responses will be reported in charts unless the percentage for a category is <5%, in this case the percentage will not be identified.

Construct survey item analysis. Rasch (1960/1980) measurement will be employed for the survey analysis. Thus, items in different survey construct sections are first descriptively analyzed and an average scale score is then computed for the purpose of looking for significant differences in each construct by intern demographic variables (e.g., demographics, grade level, school location). Because all grouping variables are dichotomous, independent samples t-tests will be conducted to investigate group differences by varying composite variables (i.e., scales).

4. TIMELINE: Testing of the NASA National K-12 Stakeholders Survey will take place April 2022 – March 2023 with the NASA K-12 STEM Engagement participating stakeholders (e.g., previous NASA OSTEM program external adult/teacher participants).

5. SAMPLING STRATEGY: A purposeful sampling technique will be employed to select the NASA K-12 STEM Engagement participating stakeholders (e.g., previous NASA OSTEM program external adult/teacher participants) from which data will be collected. Purposeful sampling is used to select an appropriate sample for the purpose of developing generalizable findings that can be used to inform changes in practices, programs, and policies (Patton, 2015). Specifically, the form of purposeful sampling that will be used is utilization-focused. Utilization-focused refers to the samples’ ability to provide sufficient, credible data with the depth and detail necessary to identify key factors that can be used to inform future decision-making (Patton, 2015). The selected NASA K-12 STEM Engagement participant stakeholders is 1000 or below. NASA National K-12 Stakeholders Survey items will be placed into Survey Monkey online software, and a survey link will be distributed through email to ~1000 NASA K-12 STEM Engagement participant stakeholders.

Table 1. Calculation chart to determine statistically relevant number of respondents

Data Collection Source	(N) Population Estimate	(A) Sampling Error +/- 5% (.05)	(Z) Confidence Level 95%/ Alpha 0.05	(P) *Variability (based on consistency of intervention administration) 50%	Base Sample Size	Response Rate	(n) Number of Respondents
Sample K-12 STEM Engagement participating stakeholders	1000	N/A	N/A	N/A	1000	N/A	1000
TOTAL							1000

6. BURDEN HOURS: Burden calculation is based on a respondent pool of individuals as follows:

Data Collection Source	Number of Respondents	Frequency of Response	Total minutes per Response	Total Response Burden in Hours
Sample K-12 STEM Engagement participating stakeholders	1000	1	20	333.33
TOTAL				333.33

7. DATA CONFIDENTIALITY MEASURES: Any information collected under the purview of this clearance will be maintained in accordance with the Privacy Act of 1974, the e-Government Act of 2002, the Federal Records Act, and as applicable, the Freedom of Information Act in order to protect respondents’ privacy and the confidentiality of the data collected.

8. PERSONALLY IDENTIFIABLE INFORMATION:

1. Is personally identifiable information (PII) collected? Yes  No

– NOTE: First and Last Name are not collected but background, demographic information and state is collected (i.e., current work position description, gender, ethnicity, racial category and current state or U.S. territory)

2. If yes, will any information that is collected by included in records that are subject to the Privacy Act of 1974? Yes  No

3. If yes, has an up-to-date System of Records Notice (SORN) been published?

Yes  No

Published March 17, 2015, the Applicable System of Records Notice is NASA 10EDUA, NASA STEM Engagement Program Evaluation System - http://www.nasa.gov/privacy/nasa_sorn_10EDUA.html.

APPLICABLE RECORDS:

4. Applicable System of Records Notice: SORN: NASA 10EDUA, NASA STEM Engagement Program Evaluation System - http://www.nasa.gov/privacy/nasa_sorn_10EDUA.html

5. Completed surveys will be retained in accordance with NASA Records Retention Schedule 1,

Item 68D. Records will be destroyed or deleted when ten years old, or no longer needed, whichever is longer.

9. PARTICIPANT SELECTION APPROACH:

1. Does NASA STEM Engagement have a respondent sampling plan? Yes  No

If yes, please define the universe of potential respondents. If a sampling plan exists, please describe? The selected NASA K-12 STEM Engagement participant stakeholders is 1000 or below. NASA National K-12 Stakeholders Survey items will be placed into Survey Monkey online software, and a survey link will be distributed through email to ~1000 NASA K-12 STEM Engagement participant stakeholders.

If no, how will NASA STEM Engagement identify the potential group of respondents and how will they be selected? Not applicable.

10. INSTRUMENT ADMINISTRATION STRATEGY

Describe the type of Consent:  Active  Passive

6. How will the information be collected:

 Web-based or other forms of Social Media

 Telephone

 In-person

 Mail

 Other

If multiple approaches are used for a single instrument, state the projected percent of responses per approach.

7. Will interviewers or facilitators be used?  Yes  No

11. DOCUMENTS/INSTRUMENTS ACCOMPANYING THIS REQUEST:

 Consent form

 Instrument (attitude & behavior scales, and surveys)

 Protocol script (Specify type: Script)

 Instructions NOTE: Instructions are included in the instrument

 Other (Specify ________________)

12. GIFTS OR PAYMENT:  Yes  No  If you answer yes to this question, please describe and provide a justification for amount.

13. ANNUAL FEDERAL COST: The estimated annual cost to the Federal government is $1200. The cost is based on an annualized effort of 20 person-hours at the evaluator’s rate of $60/hour for development and administering the survey instrument, collecting and analyzing responses, and editing the survey instrument for ultimate approval through the methodological testing generic clearance with OMB Control Number 2700-0159, exp. exp. 09/30/2024.

14. CERTIFICATION STATEMENT:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low burden for respondents and low cost for the Federal Government.

3. The collection is non-controversial and does raise issues of concern to other federal agencies.

4. The results will be made available to other federal agencies upon request, while maintaining confidentiality of the respondents.

5. The collection is targeted to the solicitation of information from respondents who have experience with the program or may have experience with the program in the future.

Name of Sponsor: Richard Gilmore

Title: Performance Assessment and Evaluation Program Manager, NASA

Office of STEM Engagement (OSTEM)

Email address or Phone number: [email protected]

Date: 9/27/2022

Bibliography

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4^th ed.). Los Angeles, CA: SAGE

Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research), with foreward and afterword by B.D. Wright. The University of Chicago Press.

NASA Office of STEM Engagement 8