Supplemental and Part B questions

Haz Simp FG Supp Questions 2018.docx

NOAA Customer Surveys

Supplemental and Part B questions

OMB: 0648-0342

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0648-0342 can be found here:

Document [docx]

Download: docx | pdf

Supplemental Questions for DOC/NOAA Customer Survey Clearance
(OMB Control Number 0648-0342)

Explain who will be conducting this survey. What program office will be
conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

This survey will be conducted by the National Weather Service’s (NWS) Analyze, Forecast and Support Office (AFSO) using in person, open-ended discussion in the form of focus groups. The Forecast Services Division (FSD) which is primarily responsible for the development of requirements, policy, and procedures for all NWS forecast and warning services, will be leading the effort.

The purpose of the focus groups is to collect feedback on possible alternatives to the NWS current hazard messaging system (Watch, Warning, Advisory (WWA)). A recent social science study was conducted that tested public knowledge of the WWA terms and the public’s response to alternative language. Using the final prototypes developed from these study results, the focus group engagements will gather internal NWS and core partner feedback on preferences and potential impacts to these alternative systems. With that in mind, the audience for these focus groups will be Warning Forecast Office (WFO) forecasters and NWS core partners including but not limited to emergency managers (EMs) and broadcast meteorologists (BM).

Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

This survey was developed by FSD Chief, Elliott Jacks, and Hazard Simplification (Haz Simp) Project Manager and Social Scientist, Dr. Danielle Nagele. They consulted other risk communication specialists within NOAA and FSD Branch Chiefs during the creation of the survey to ensure not only the integrity of the questions, but also the accuracy of the meteorological/hydrological content.

Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate?

The survey will be conducted using in person focus groups in select locations throughout the US. Because this is qualitative data collection in the form of focus groups, we do not expect our results to be generalized to the broader population. Instead the results of the focus groups will inform and guide next steps for the Haz Simp project including a Workshop in Spring 2019.

At least one location will be chosen in each of the six NWS Regions (Eastern, Southern, Central, Western, Alaska, Pacific). Focus groups will be held at a NWS Warning Forecast Office (WFO) and will open to local/regional partners. Participants will be recruited with the assistance of WFO staff who have developed relationships with core partners in their area. We expect between 8 and 15 participants for each focus group which is a typical and reasonable number for this methodology.

Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

The results of the focus groups will be analyzed by FSD staff and contracted Social Scientist, Dr. Danielle Nagele. The goal of the focus groups are to assess whether using alternative warning terms would improve hazard communication for core partners and NWS forecasters. The focus group notes will be examined and coded to identify concepts, trends, and themes. We will be looking for strengths, weaknesses, and opportunities to improve the prototypes based on the feedback gathered during the focus groups.

These results will inform a Workshop in Spring 2019 that will draw out the technical and policy challenges and requirements for implementation of a new system to replace WWA.

This survey is not intended to measure a GPRA performance measure.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe would be NWS WFO staff and their core partners. Table 1 shows an approximate number of EM entities, Local broadcast stations, WFO staff members, and other relevant stakeholder entities for each WFO County Warning Area (CWA).

In order to gain a broad understanding of perspectives, preferences, and impacts, we will choose at least one location within each of the six NWS Regions (Eastern, Southern, Central, Western, Alaska, Pacific) in which to hold focus groups. Each location will be centered around a WFO CWA, though other entities within driving distance will be welcome. We will be in close collaboration with WFO staff to recruit relevant focus group participants, leveraging pre-existing user groups and networks. There will be 3-5 focus groups per location and each group will have no more than 15 participants. Assuming an an average of 4 focus groups per location, 6 locations, and a maximum of 15 people per focus group, we are expecting ~360 respondents. Each focus group will last ~3 hours.

Table 1: Approximate number of participants per focus group in each NWS Region

Location	EM Offices and Public Safety Entities	Local TV/Radio Stations	WFO Staff	Other Relevant Stakeholder Groups*
Estimated number of potential respondents in each CWA	3-5	5-15	8-15	3-5
Estimated number of focus group participants	2-4 participants from each Entity (6-20 total)	Staff from 2-3 of the major stations (8-15 total)	5-10 total	1-2 other groups selected with 8-15 participants in each (8 - 30)+

*Location dependent specialized user groups (e.g. commercial fisherman)

+This box indicates that 1-2 other relevant stakeholder groups may be chosen in each Region, but each additional focus group would only have 8-15 participants.

Figure 1: Map of NWS WFO CWAs

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Focus group participants will be recruited based on their professional affiliation and proximity to focus group location. We will work closely with the local WFO to recruit the most relevant and appropriate number of participants. At least one location will be chosen in each of the six NWS Regions (Eastern, Southern, Central, Western, Alaska, Pacific). There will be 3-5 focus groups per location and each group will have no more than 15 participants. Assuming an an average of 4 focus groups per location, 6 locations, and a maximum of 15 people per focus group, we are expecting ~360 respondents. Each focus group will last ~3 hours.

We do not intend to generalize the results of these focus groups to the broader population. This survey alone will not premise any changes to products and services, instead it will provide feedback from NWS staff and core partners in the form of broad themes. These themes and feedback will allow us to further refine the alternative prototypes to WWA and develop more thorough and accurate Workshop topics.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

In order to maximize response rates, we will be working closely with the local WFO, including the Warning Coordination Meteorologist (WCM) who collaborates often with core partners in the community. We have chosen to host focus groups in at least one location within each of the six NWS Regions - this will allow us to get broad representation across the US and across hazard types. Each WFO hosting focus groups will be chosen based on their availability, strong connection to a broad set of user groups, proximity to other WFOs (opportunity to bring in others within driving distance), and typical hazards experienced. In each location we will gather feedback from the WFO staff, emergency managers/public safety officials, and broadcast media (core stakeholders). Other groups will be added depending on availability and location (e.g. commercial fisherman in a coastal location).

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Due to its proximity to NOAA Headquarters, the Sterling WFO will act as an initial test of the focus group script and structure. The forecasters and other office staff will participate in the first focus group in order to refine the process. All participants/respondents will be federal employees.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Elliott Jacks (Division Chief)

NOAA - Federal

NWS Analyze, Forecast, and Support Office, Forecast Services Division

[email protected]

301-427-9351

Danielle Nagele, Ph.D. (Project Manager/Social Scientist)

NOAA - Affiliate (INNOVIM)

NWS Analyze, Forecast, and Support Office, Forecast Services Division

[email protected]

301.427.6919

Michelle Hawkins (Branch Chief)

NOAA - Federal