0648-0658 SUPPORTING STATEMENT Part A

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Bay Watershed Education and Training Program National Evaluation System

OMB Control No. 0648-0658

SUPPORTING STATEMENT PART A

Abstract

This is a request for an extension of an existing information collection.

The National Oceanic and Atmospheric Administration’s (NOAA) Office of Education is sponsoring data collection efforts on its Bay Watershed Education and Training (B-WET) program. B-WET advances NOAA’s mission by awarding education grants that foster an environmentally literate citizenry who have the knowledge, attitudes, and skills needed to protect watersheds and related ocean, coastal, and Great Lakes ecosystems. B-WET currently funds projects in seven regions (California, Chesapeake Bay, Great Lakes, Gulf of Mexico, Hawaii, New England, and the Pacific Northwest). To ensure high-quality educational activities funded by B-WET, we created a cross-region, internal evaluation system that collects information from B-WET awardees and their teacher participants to provide ongoing monitoring of program implementation and to identify opportunities for improved program outcomes. B-WET awardees and teacher participants complete online survey forms to provide data for the evaluation system. Awardees are surveyed once per year of the project and teachers are surveyed twice, once after completing professional development programs and once after teaching their students about watershed. Information collected from awardees includes program elements such as program duration, format, audience, location, support and/or materials offered, and topics covered. Information collected from teacher professional development participants includes teaching methodologies, program satisfaction, program coverage, suggestions for improvement, and teaching confidence. This data collection effort benefits those who receive B-WET grants and project participants of those grants by providing regular program improvements that they can leverage to implement successful projects.

The B-WET evaluation system is designed to collect data on a continuous basis (annually for grantees and as described above for teachers). However data collection is currently paused due to the fact that the national evaluation instruments are not designed to assess project implementation during the ongoing impacts of the pandemic.

Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The NOAA Office of Education’s Bay Watershed Education and Training (B-WET) program seeks to contribute to NOAA’s mission by helping to foster an environmentally literate public who make informed environmental decisions (http://www.noaa.gov/office-education/bwet). B-WET currently funds projects in seven regions: California, Chesapeake Bay, Great Lakes, Gulf of Mexico, Hawaii, New England, and the Pacific Northwest.

The NOAA B-WET program is authorized under 33 U.S.C. § 893a(a), the America COMPETES Act. The Administrator of the National Oceanic and Atmospheric Administration is authorized to conduct, develop, support, promote, and coordinate formal and informal educational activities at all levels to enhance public awareness and understanding of ocean, coastal, Great Lakes, and atmospheric science and stewardship by the general public and other coastal stakeholders, including underrepresented groups in ocean and atmospheric science and policy careers.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

B-WET created a cross-region, internal evaluation system to monitor program implementation and outcomes on an ongoing basis. Based on a review of annual evaluation system results, B-WET has made program improvements by adjusting its Federal Notice of Funding Opportunities and program guidelines. On-going data collection enables assessment of the benefits of continuous improvements and, thus, supports adaptive management of the program. This effort is consistent with the goals and plans outlined in the NOAA Education Strategic Plan 2021-2040¹. See in particular Objective 5.2.1 on page 28 as a part of “Organizational Excellence.”

To meet evaluation needs, B-WET’s evaluation system was designed to answer the following questions:

1. To what extent do regional B-WET programs support grantees in implementing Meaningful Watershed Educational Experiences (MWEEs)?

2. How are MWEEs implemented by grantees and teachers?

3. To what extent do B-WET-funded projects increase teachers’ knowledge of watershed science concepts, their confidence in their ability to integrate MWEEs into their teaching practices, and the likelihood that they will implement high quality MWEEs?

4. To what extent do B-WET-funded projects increase students’ knowledge of watershed concepts, attitudes toward watersheds, inquiry and stewardship skills, and aspirations towards protecting watersheds?

The annual Grantee Survey collects information on grantee satisfaction with the B-WET program, project goals and implementation, and other descriptive information about the projects. The Teacher Professional Development (PD) Survey asks participating teachers to provide information about their experience in professional development, their intentions for instruction as a result, and their anticipated student outcomes. At the end of the following school year these same teachers receive the follow-up Teacher MWEE Survey, which asks for information about how they actually implemented MWEEs with their students and student outcomes. Surveys include multiple choice and open-ended questions.

The primary users of the B_WET evaluation system are the B-WET staff members who administer the B-WET grant program, and its national coordinator. These individuals review the evaluation system’s results annually to determine what changes may be necessary to the grant program to maximize benefits for K-12 teachers and students. The system automatically generates results in the form of aggregate descriptive statistics (at the national and regional level) to inform decisions about the program at both of these tiers.

B-WET staff members will share findings with secondary users, including staff members in the NOAA Office of Education and other parts of the agency who may choose to use information to improve other NOAA education programs. Evaluation findings will also be used at the national level to report on agency performance measures and respond to other Administration data collection activities, as appropriate. Tertiary users are grant recipients who are provided with access to a synthesis of findings so that they may identify ways to improve their respective environmental science and education programs.

Preliminary results of an initial analysis of data were discussed at an interactive session at the North American Association for Environmental Education (NAAEE) national conference in Madison, Wisconsin, in October 2016, and additional information was shared as part of the NAAEE virtual conference in 2017. Program managers regularly include review of evaluation results in regional grantee workshops and evaluation findings inform the development of regional funding opportunities. Evaluation system results were also presented to the NOAA Education Council in 2018. In the future, results associated with each of the evaluation system’s questions will continue to be shared online and through professional conferences, reports, and peer-reviewed journal articles.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The evaluation system data collection is electronic. Study participants (i.e., B-WET grantees and teachers who participate in professional development offered by B-WET grantees) receive email prompts to complete the online instruments accessed through Qualtrics, an online survey delivery and collection platform. The Qualtrics surveys have built-in “logic” prompts so respondents complete only items relevant to their experience. Data are stored on Qualtrics’ server that automatically generates descriptive statistics. The Qualtrics online platform is designed to meet Federal Information Security Management Act (FISMA) security guidelines to ensure all data provided by respondents is secure. The proposed data collection process minimizes costs, while also being sensitive to issues of respondent burden, accuracy, and efficiency. It is assumed that most respondents (i.e., grantees, K-12 teachers) have access to the Internet at work, home, on a smartphone, or at a public institution such as a local library.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Question 2

In some cases B-WET-funded projects that have additional funding or partnerships with other parts of NOAA may be asked to report in to other NOAA data collections; however the B-WET system is the only NOAA data collection taking place that is focused on Meaningful Watershed Educational Experiences and specific characteristics of B-WET awards. NOAA education programs and evaluation efforts are coordinated through the NOAA Office of Education and the NOAA Education Council, and data collection is coordinated to ensure that individual survey items are not duplicative.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

The evaluation system asks individuals working for non-profit organizations and some businesses, state and local government employees, and teachers in K-12 schools to participate by completing online questionnaires. The study minimizes burden on respondents because completion of the proposed questionnaires is voluntary. In addition, an iterative item review process was used to eliminate any non-essential questions, thus keeping the questionnaires as streamlined as possible while ensuring that sufficient data are collected to answer the evaluation questions.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The evaluation system contributes to ensuring that federal funding is used in an effective and efficient manner to educate teachers and students about watershed science and environmental issues. The evaluation system provides B-WET with scientific data to assess the effectiveness of its grant funded programs (i.e., B-WET-funded teacher professional development and student MWEEs). The results of the evaluation system also provide insights into how to improve watershed education programs.

If the evaluation system were not conducted, B-WET would not have the needed data to scientifically assess the effectiveness of its program/MWEEs and/or to scientifically determine how best to improve its program/MWEEs. The continuous data collection of the evaluation system allows on-going monitoring of outcome results and, thus, on-going program/MWEE improvements.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with OMB guidelines.

This collection will be conducted in a manner consistent with OMB guidelines.

8. If applicable, provide a copy and identify the date and page number of publications in the Federal Register of the agency's notice, required by 5 CFR 1320.8 (d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A Federal Register Notice published on Tuesday, February 8, 2022 (87 FR 7157), solicited public comments. No comments were received.

During the development of the B-WET evaluation system, the ILI-UM team solicited input from a range of individuals including B-WET grantees, evaluation experts, watershed scientists, and statisticians on all aspects of the proposed evaluation system. Their suggestions informed the design of the proposed study (e.g., type of data collection, frequency and timing of data collection, reporting formats, etc.). Their feedback was also used to improve the questionnaire items and led to confirmation of their face and content validity.

In addition, the grantee and teacher questionnaires include several measures at the end of the respective instruments to allow respondents to comment on the data collection process and content. This on-going feedback will continue to be used to improve both the data collection process and instruments over time.

Limited feedback has been collected during the current clearance period, since data collection was paused in January 2021 due to the fact that the instruments are not designed to assess project implementation during the ongoing impacts of the pandemic. While we are not proposing to make changes at this time, we will consider this feedback in the design of future instruments and approaches that will accompany a planned revision to this information collection.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No incentives for participation are provided by the NOAA B-WET program. We encourage grantees to ask teachers to complete the surveys as part of their professional development responsibilities. If the grantees provide stipends to their professional development teachers, they could include a requirement that teachers complete the questionnaire to receive the payment.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy. If the collection requires a systems of records notice (SORN) or privacy impact assessment (PIA), those should be cited and described here.

An assurance of confidentiality is not provided to respondents. B-WET grantees and teachers who respond to the questionnaires, however, remain anonymous to B-WET and NOAA. Only aggregate statistics at the national, regional, and organizational level are reported. Thus, individual sources of data are not disclosed and study participants remain anonymous.

Anonymity is guaranteed in the following ways:

• Neither B-WET grantees nor teacher respondents are asked to provide information that can identify them as individuals as part of the questionnaire.

• Information that is needed to link data, that is (1) award numbers to link data provided by grantees with teachers participating in their professional development, and (2) teacher-generated codes to link responses to their initial and subsequent questionnaires, are not associated with any of the other data they provide.

• Email addresses, used to (1) invite prospective participants to participate in the study with a link to the questionnaire, and (2) track response rates and prompt non-respondents, are not associated with any of the data provided by respondents.

• Results are only presented in aggregate form (across all grantees or teacher respondents), not by individual.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

No questions of a sensitive nature are asked.

12. Provide estimates of the hour burden of the collection of information.

The data collection is currently paused due to the fact that the national evaluation instruments are not designed to assess project implementation during the ongoing impacts of the pandemic. Therefore, the burden information from the previous submission is being used, with the exception of the hourly wage rate, which has been updated with the most recent information from BLS.

Information Collection	Type of Respondent (e.g., Occupational Title)	# of Respondents/year (a)	Annual # of Responses / Respondent (b)	Total # of Annual Responses (c) = (a) x (b)	Burden Hrs / Response (d)	Total Annual Burden Hrs (e) = (c) x (d)	Hourly Wage Rate (for Type of Respondent) (f)	Total Annual Wage Burden Costs (g) = (e) x (f)
Grantee Questionnaire	Education & Childcare Administrators (11-9030)	86	1	86	1	86	$48.64	$4,183.04
Grantee Nonresponse	Education & Childcare Administrators (11-9030)	15	1	15	0.17	3	$48.64	$145.92
Teacher PD	Secondary School Teachers (25-2030)	1003	1	1003	.5	501.5	$32.93	$16,514.40
Teacher PD NonResponse	Secondary School Teachers (25-2030)	376	1	376	.083	31	$32.93	$1,020.83
Teacher MWEE	Secondary School Teachers (25-2030)	752	1	752	.5	376	$32.93	$12,381.68
Teacher MWEE NonResponse	Secondary School Teachers (25-2030)	351	1	351	.083	29	$32.93	$954.97
Totals				2583		1026.5		$35,200.84

The May 2020 National Occupational Employment and Wage Estimates from BLS was used to determine the mean hourly wage rate for the type of respondent. https://www.bls.gov/oes/current/oes_nat.htm#00-0000

For Secondary School Teachers: Calculated from U. S. Department of Labor, Bureau of Labor Statistics. May 2014. National Occupational Employment and Wage Estimates, United States: Secondary School Teachers (mean hourly wage not available; mean annual salary $67,240) https://www.bls.gov/oes/current/oes_nat.htm#25-0000 and Krantz-Kent, Rachel. 2008. Teachers’ work patterns: when, where, and how much do U.S. teachers work? U. S. Department of Labor, Bureau of Labor, http://www.bls.gov/opub/mlr/2008/03/art4full.pdf (“On average for all days of the week, full-time teachers worked 5.6 hours per day” = 39.2 hours per week = 2,038 hours per year @ 52 weeks/year = $32.93 per hour)

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

There are no direct costs to participants. The only costs are the opportunity costs of respondents’ time required to provide information as explained in Question 12 above. No capital equipment, start-up, or record maintenance requirements are placed on respondents.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

Cost Descriptions	Grade/Step	Loaded Salary /Cost	% of Effort	Fringe (if Applicable)	Total Cost to Government
Federal Oversight	ZA-IV	$187,000	5%		$9,350
Contractor Cost		$150,000	50%		$75,000
Travel					$0
Other Costs: Survey mgmt platform license & support					$25,000
TOTAL					$109,350

The estimated cost to the federal government to implement the NOAA B-WET National Evaluation System is based on the government's cost for yearly maintenance of the data collection, periodic analysis, interpretation and reporting activities, and personnel cost of government employees involved in oversight of the collection system. The overall cost to the government for the data collection activities described herein is annualized to $109,350. These estimates are based on B-WET program management’s previous experience managing these data collection activities and costs observed from 2014-2021.

This includes:

• $75,000 for an estimated half FTE for an internal program evaluator (contractor) who will facilitate the data collection activities, conduct periodic data analysis, and produce reports at least once per three-year period, and as much as annually

• $25,000 annually for FedRAMP approved online survey management platform license and support

• $9,350 annually for 5% FTE government personnel costs in overseeing the evaluation activity

Total annualized cost: $109,350.

15. Explain the reasons for any program changes or adjustments reported in ROCIS.

There are no changes to the information collection since the last OMB approval. The respondent’s labor costs were updated with current wage rates. The burden response time for the Post-PD Teachers and Post-MWEE Teachers was updated from 0.1 hr to 0.83 hrs to match the footnotes captured at Q12. This accounts for a reduction of 14 burden hours.

Estimated costs to the Federal Government (Q14) were updated to reflect recent changes to reporting requirements.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Depending on the availability of the necessary funding, regular syntheses of the main findings as related to the evaluation system questions will be prepared to meet the needs of different stakeholder groups. These stakeholders include the NOAA Office of Education which seeks information to improve its education grant programs, and external stakeholders such as B-WET grantees and teacher participants seeking ways to improve their MWEE practices, as well as tertiary members of the public. B-WET will prepare these syntheses, ensuring that they meet respective stakeholders’ needs both in terms of content and format. These syntheses will be made available online and will include descriptive statistics such as numbers of survey responses, means (averages) of responses items, counts of responses when appropriate. Reports will also include inferential statistics (i.e., tests of statistical significance) when appropriate.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency plans to display the expiration date for OMB approval of the information collection on all instruments.

18. Explain each exception to the certification statement identified in “Certification for Paperwork Reduction Act Submissions."

The agency certifies compliance with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).

1 https://www.noaa.gov/education/explainers/2021-2040-noaa-education-strategic-plan

6