U.S. Department of Commerce
National Oceanic & Atmospheric Administration
Partner Probabilistic Snowfall Messaging Survey
OMB Control No. 0648-XXXX
SUPPORTING STATEMENT PART A
Abstract
The National Weather Service (NWS) and the National Center for Atmospheric Research (NCAR) will work together to conduct new data collection, specifically a survey across the NWS Central Region to determine different core partners’ decisions, needs, preferences, understanding, and usefulness regarding probabilistic snowfall forecasts. The core partners of particular interest for this survey are emergency managers, school officials, and transportation officials. Results from this survey will inform how probabilistic snowfall information will be used in future NWS products and services. The benefit is that this evidence-based knowledge will help NWS provide winter weather forecast information in a way that is meteorologically sound as well as user-driven, which will improve core partners’ ability to make improved decisions for the protection of life and property.
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
For decades, the NWS has provided deterministic (i.e., single-value) snowfall forecasts or snowfall uncertainty forecasts with set, narrow ranges (e.g., 2-4, 4-6 inches). Recent advancements in numerical weather prediction (NWP) model ensembles have allowed for the calculation of probabilistic snowfall information through a probability distribution of snowfall amounts that is specific for a given winter weather threat (i.e., it varies from event to event). This forecast uncertainty information can be extracted and messaged in a number of ways, for example, as the probability that a location will receive some amount of snowfall, as the probability the snowfall will be within a certain range, or a range of times when snowfall will begin and end. Although such probabilistic snowfall information is statistically skillful, it is unknown whether and in what ways the information is best understood or helpful to different end-users for their decision-making.
Legal justification for the proposal data collection comes from H.R.353, the Weather Research and Forecasting Innovation Act of 2017, which calls for (a) improved understanding about the receipt, interpretation, and responses to warnings and forecasts of high weather events that endanger life and property and (b) use of advanced NWP forecasting tools and techniques that improve the forecasting of intensity and severity of high-impact weather.
Multiple NOAA (National Oceanic and Atmospheric Administration) documents further support the need for such data collection. NWS Directive 10-1801, Section 5 states “Partner and public feedback regarding NWS products and services is critical. It enables the NWS to constantly improve our services/products, respond to changing needs of partners, design and create new products and services, and to routinely evaluate effectiveness.” Moreover, the NWS Strategic Plan (available here) includes goals to reduce the impacts of weather events by improving users’ ability to receive, understand, and act on information, including with social sciences research to improve the understanding and utility of forecasts and warnings. The strategic plan also calls for harnessing cutting-edge science and technology--such as ensembles--to provide the best forecasts and warnings. Most recently, the NOAA Science Advisory Board released a report on Priorities for Weather Research for NOAA for the next decade (available here). The report includes many recommendations that are relevant to this survey, among them: (a) the need for the development and delivery of user-oriented forecast information, (b) a call to “Examine for whom, in what hazard scenarios, when, and how forecast uncertainty (probabilistic) information is advantageous versus when it is not”, and (c) a recommendation that, “NOAA should partner with other agencies and the private sector to develop, test, and evaluate probabilistic and deterministic hazard information delivery capabilities for a broad spectrum of end-users.”
To achieve the goals of the project, which are part of the NWS’s broader strategic goals, data collection via a survey is necessary to collect generalizable data to determine core partners’ decisions, needs, preferences, understanding, and usefulness regarding probabilistic snow forecasts.
The questions asked are designed to better understand core partners’ decisions, needs, preferences, understanding, and usefulness regarding probabilistic snow forecasts. Answers to three demographic questions also will be requested but not required, including gender, level of education, and which local NWS office serves the respondent's area.
As explained above, there are multiple NOAA-relevant policies and reports that motivate this type of data collection. Results from this survey will be used to determine what and how probabilistic snowfall information will be derived and provided in future NWS products and services, with the ultimate goal of providing information in a way that improves core partners’ ability to make informed decisions for the protection of life and property for snowfall events.
Results may be shared throughout the NWS in various forms, including formal presentations as part of webinars, conferences, or other meetings. Data also will be shared with National Center for Atmospheric Research scientists for analysis and may be used in scientific publications, future research, and professional conferences. The data collected will be anonymous; no identifiable information will be collected.
Collection will occur annually for the length of the approved survey.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.
The survey collection will be conducted electronically in its entirety, as an Internet-based survey. Respondents will be invited via email, and they will be provided with a link to respond to the Web-based survey that will use the online survey platform QuestionPro. No in-person or paper-based methods will be used.
Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Question 2
As noted above, the NWS is harnessing recent advancements in NWP model ensemble capabilities to derive probabilistic snowfall information in different ways, with the potential to provide this information to its core partners. The provision of such information should be informed by partners’ decisions, needs, preferences, understanding, and perceived usefulness. Yet, very little work regarding winter forecast information has been done with NWS’s core partners.
Five years ago, the Eastern Region Group (ERG) was contracted by NOAA and did seven focus groups in four geographic locations with different decision-makers (NOAA 2018). Three of the focus groups included emergency managers (two of those focus groups were done in Eastern Region, one was done in Central Region), and two focus groups included school and DOT officials (one of those focus groups was done in Eastern Region and one was done in Central Region). The survey proposed here expands on the ERG focus groups by (a) asking about additional types of information needs and perceptions of probabilistic snowfall forecast information and (b) collecting data from a much larger sample of NWS core partners in a way that can be generalized to all emergency managers, school officials, and transportation officials in NWS’s Central Region.
NOAA (2018). Communicating Probabilistic Information for Decision‐Makers: A Case Study Using Experimental Snow Forecast Products. Summary of Decision‐Maker Focus Groups, Simulations, and User Journey. Available online at https://repository.library.noaa.gov/view/noaa/29009
If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The collection of this information will not impact small businesses or other small entities.
Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
The NWS is a service-based agency. Without direct input from our core partners, the products and services that are provided may be of little use or value–or, worse, be detrimental–in the decision-making process as it relates to hazardous weather. Services must be received, understood, and acted upon in a manner consistent with the agency’s mission of protecting life and property. Collecting this information will help achieve this goal and is explicitly endorsed by National Weather Service Directive 10-1801 and by the recent report that articulates NOAA’s Priorities for Weather Research for the next decade.
Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with OMB guidelines.
This collection will be conducted in a manner consistent with OMB guidelines.
If applicable, provide a copy and identify the date and page number of publications in the Federal Register of the agency's notice, required by 5 CFR 1320.8 (d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.
The 60-day Federal Register Notice that solicited public comment on this request was published on October 28, 2021 (86 FR 59696). No comments were received.
NWS is collaborating on this data collection effort with Dr. Julie Demuth from the National Center for Atmospheric Research. Dr. Demuth has her PhD in Communication with a focus on mass communication, science communication, and risk communication. She has expertise in quantitative social science methods, including extensive training and experience in survey design, sampling, data collection, and statistical analysis.
Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
No payments or gifts will be provided to those who complete the survey because most are public service employees.
Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy. If the collection requires a systems of records notice (SORN) or privacy impact assessment (PIA), those should be cited and described here.
No Personal Identifiable Information (PII) will be collected as part of the survey. All survey respondents will be anonymous and their answers will be analyzed and reported in aggregate or in ways they cannot be identified. If any respondent provides a response to an open-ended question that could identify them, the identifying information will be redacted so they remain anonymous. This assurance information is part of the informed consent language at the beginning of the survey that a participant must read before beginning the survey.
The raw survey data will be shared with a core group of NWS and NCAR personnel who will lead data analysis. Additional NWS personnel may choose to analyze data as they deem of interest or important beyond the analysis done by the core group. Additionally, the raw data may be shared with an undergraduate student, graduate student, or intern (e.g., a NOAA Hollings Scholar, or an NCAR SOARS protege) whom we may recruit to help with data analysis and who would benefit from the opportunity to conduct applied, user-oriented, interdisciplinary research. All data will be stored on password-protected computers. Anonymized and aggregated survey results will be shared within NOAA through a report and presentations and through conference presentations and a peer-reviewed publication.
Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
The survey does not contain any sensitive questions as described in the PRA guidance.
Provide estimates of the hour burden of the collection of information.
The NWS Central Region has 38 Weather Forecast Offices (WFOs), which serve the local emergency management, school, and transportation partners in their County Warning Areas (CWA) as well as some regional and state partners. The total number of these partners varies by WFO (e.g., depending on population in their CWA), but the NWS WFO collaborators involved in this survey estimate an average of N=400 partners per WFO. We estimate a conservative response rate of 50%, which translates to an average of N=200 partners for each of 38 WFOs.
Information Collection |
Type of Respondent (e.g., Occupational Title) |
#
of Respondents / Year |
Annual
# of Responses / Respondent |
Total
# of Annual Responses |
Burden
Hrs / Response |
Total
Annual Burden Hrs |
Hourly
Wage Rate (for Type of Respondent) |
Total
Annual Wage Burden Costs |
Winter Wx Survey |
General Public |
7600 |
1 |
7600 |
0.25 |
1,900 |
$27.07 |
$51,433 |
Totals |
|
|
|
7600 |
|
1900 |
|
$51,433 |
The “All Occupations” occupation title (00-0000) was used to determine the mean hourly wage rate since respondents will cross a broad range of occupations. See https://www.bls.gov/oes/current/oes_nat.htm
Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).
There are no capital/start-up costs or ongoing operation/maintenance costs associated with this information collection.
Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.
Four NWS personnel in CR (3 Warning Coordination Meteorologists and 1 Science and Operations Officer) have been involved in the survey design thus far and will lead analysis of the data. The total number of hours of each person working on the survey is estimated to be 40. Their respective grade/steps and annual salary costs are provided in the table below, along with the corresponding total cost to the government. As noted above in response to Q10, the raw data will be shared with six additional NWS personnel who may choose to analyze the data. These possible costs are accounted for at differing GS grades and steps.
In addition, it is estimated that each WCM or point-of-contact who sends the survey emails to their partners in each WFO will spend up to 10 minutes sending each survey. We anticipate each POC will send the initial survey invitation and up to two reminders, thereby each spending a total of 0.5 hours. An average grade/step and salary cost is provided in order to calculate the total cost to the government.
The funding for NCAR collaborators Julie Demuth and Robert Prestley to contribute work time for the survey development and data analysis is covered by NOAA JTTI grant NA18OAR4590362 for which Dr. Demuth is the grant Principal Investigator. Therefore, it is not accounted for in the table below.
Cost description and personnel |
Grade/step |
Annual Salary cost |
Total Annual cost to government |
|
NWS personnel #1 developing and analyzing the survey |
40 |
GS 13-5 |
$106,955 |
$2,056.83
|
NWS personnel #2 developing and analyzing the survey |
40 |
GS 13-6 |
$110,101 |
$2,117.33 |
NWS personnel #3 developing and analyzing the survey |
40 |
GS 13-10 |
$122,683 |
$2,359.29 |
NWS personnel #4 developing and analyzing the survey |
40 |
GS 14-5 |
$129,728 |
$2,494.77 |
(Possible) Additional NWS personnel analyzing the survey, 3 personnel at this grade/step, for 5 hours each |
15 |
GS 13-10 |
$122,683 |
$884.73 |
(Possible) Additional NWS personnel analyzing the survey, 3 personnel at this grade/step, for 5 hours each |
15 |
GS 14-5 |
$129,728 |
$935.54
|
38 WCMs or points-of-contact sending up to 3 10-minute emails to partners |
0.5 |
Midpoint between GS13-6 and 14-5 |
$119,915 |
$1,095.38 |
Total |
|
|
|
$11,943.86 |
Explain the reasons for any program changes or adjustments reported in ROCIS.
This is a request for a new data collection of this information.
For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
The survey data will be analyzed primarily descriptively, with frequency distribution and measures of central tendency (i.e., mean, median, mode). Multivariate analyses also will be conducted to evaluate how partners’ decisions and informational preferences are associated with their perceptions and uses of different probabilistic snowfall forecast information.
Results from the survey will be shared internally within the NWS and externally with the National Center for Atmospheric Research. Collection of the information is scheduled to occur upon OMB approval, likely in Spring 2022. Data analysis will occur throughout the subsequent six months in 2022.
A report of the findings will be provided to NWS Central Region by September 30, 2022, or within 6 months after the survey closes, whichever is later. The data also will be presented at a future conference (e.g., American Meteorological Society or the National Weather Association).
NCAR research scientists will analyze and synthesize portions of the survey data in conjunction with other data being collected as part of an externally competed and funded NOAA JTTI grant. Those data will be disseminated in a final grant report to NOAA by January 2023 and via a peer-reviewed publication, to be submitted within a year after the survey closes.
If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
The expiration date for OMB approval will be displayed on all survey forms.
Explain each exception to the certification statement identified in “Certification for Paperwork Reduction Act Submissions."
The agency certifies compliance with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).
Page |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dumas, Sheleen (Federal) |
| File Modified | 0000-00-00 |
| File Created | 2022-03-11 |