0648-PSMS Supporting Statement Part B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Partner Probabilistic Snowfall Messaging Survey

OMB Control No. 0648-XXXX

SUPPORTING STATEMENT PART B

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The universe of the population of particular interest is emergency management, school officials, and transportation officials who work in and serve the National Weather Service’s Central Region. These groups are considered NWS core partners who utilize NWS forecast and decision support information to make risk assessments and risk management decisions on behalf of society when hazardous weather threatens. In particular, these groups make critical decisions when winter weather, namely snowfall, threatens. Thus, surveying them about their decisions, preferences, understanding, and usefulness regarding probabilistic snowfall information will help guide the development, refinement, and provision of probabilistic forecast information for them to enhance their decision-making.

As described in response to Q12 of Part A, the NWS Central Region has 38 Weather Forecast Offices (WFOs), which serve the local emergency management, school, and transportation partners in their County Warning Areas (CWA) as well as some regional and state partners. The total number of these partners varies by WFO (e.g., depending on population in their CWA), but the NWS WFO collaborators involved in this survey estimate an average of N=400 partners per WFO.

NWS recently fielded event-driven Impact-Based Decision Support Services surveys with their partners. CR’s response rates were above 60% (personal communication with Vankita Brown). However, for this survey, we estimate a more conservative response rate of 50% given the significant factor of COVID on many of these users in their professional and likely personal roles, which may translate to them having less time to respond to a survey. Nevertheless, we will work to achieve a 50% response rate in two key ways: (1) by having the survey sent to NWS core partners by the Warning Coordination Meteorologist (WCM), who is the trusted, local liaison between a WFO and the partners it serves in the region, with long-established relationships, and (2) by sending up to two survey reminders.

Survey Region	Target Population (38 WFOs × 400 partners) (a)	Response Rate (b)	Expected Number of Respondents (c)=(a) × (b)
NWS Central Region WFOs	15,200	50%	7,600

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

As described above and in response to Q12 of Part A, the NWS Central Region has 38 Weather Forecast Offices (WFOs), which serve the local emergency management, school, and transportation partners in their County Warning Areas (CWA) as well as some regional and state partners. The total number of these partners varies by WFO (e.g., depending on population in their CWA), but the NWS WFO collaborators involved in this survey estimate an average of N=400 partners per WFO. There will be no sampling nor stratification of the sample; all partners will be invited to participate. As also described above, we estimate a conservative response rate of 50%, which translates to an average of N=200 partners for each of 38 WFOs and up to 7600 responses across all of CR.

Although some data analyses will be conducted at the full Region level, many data analyses will be conducted at the level of each WFO. Based on a statistical power analysis for a fixed model multiple linear regression to assess small effect sizes (e.g., 0.1) with up to 8 tested predictors with power of beta=0.9 at statistical significant level of alpha=0.05, a sample size of N=199 is needed for each WFO. This corresponds with our average estimated number of responses per WFO.

We do not anticipate any problems that would require specialized sampling procedures. It is possible that we may have limited participation from a group of interest (e.g., school officials) due to the continued, many effects of COVID. In this case, our sampling approach would be the same in that all school partners will be invited to participate. However, it’s possible that additional reminders and additional time fielding the survey will be needed to capture sufficient responses.

The need for collecting longitudinal data to assess changes in the effects of NWS’s evolving products and services has been documented, for instance, in the NOAA Science Advisory Board’s report on Priorities for Weather Research for NOAA for the next decade (see here). Therefore, we seek to field the survey annually so that we can collect this important empirical data to understand partners’ decisions, needs, preferences, understanding, and usefulness regarding probabilistic snowfall forecasts, including as the provision of these forecasts are changing.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Each NWS WFO in CR has developed a sampling frame of their core partners, with their partners’ names, job categories and roles, and contact information. Per the power analysis outlined in the response to the prior question, all school officials, emergency managers, and transportation officials will be invited to participate in the survey. Some WFO partner lists may reach partners beyond the three groups of particular interest, and the survey has an “Other” job category option for this reason.

In order to maximize response rates and reduce non-response, (1) the survey will be sent to NWS core partners by the Warning Coordination Meteorologist (WCM), who is the trusted, local liaison between a WFO and the partners it serves in the region, with long-established relationships, and (2) up to two survey reminders will be sent, one after 2 weeks, and the second after 3 weeks.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

After the Web survey has been programmed, we will pretest it with two emergency managers, two school officials, and two transportation officials, all of whom will be from a different NWS Region but that nevertheless manage risks from winter weather. The pretests will be conducted as cognitive think alouds, with a primary focus on ascertaining whether any questions are confusing or interpreted differently than intended.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

As described in response to Q8 of Part A, NWS is collaborating on this data collection effort with Dr. Julie Demuth from the National Center for Atmospheric Research. Dr. Demuth has her PhD in Communication with a focus on mass communication, science communication, and risk communication. She has expertise in quantitative social science methods, including extensive training and experience in survey design, sampling, data collection, and statistical analysis. Accordingly, Dr. Demuth has helped design the survey, determine the sample size needed and sampling approach, and determine data analyses.