Supplemental and Part B questions

OMB Control No. 0648-0342

Expiration Date: 04/30/2015

Extra-tropical Surge and Inundation Social Science Research Project

And

Arrival of Tropical Storm Force Winds Social Science Research Project

Amendment to Add Focus Groups

April 15, 2015

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) predicts tropical and extra-tropical (ET) storm movement and impacts; issues warnings, watches, and other information products; and interacts with a variety of audiences (e.g., the media, local officials, emergency managers, and the public) regularly when a tropical storm or ET storm is threatening the coast.

The NWS is looking to develop or issue three new products:

• A storm surge watch/warning, to communicate life-threatening storm surges;

• A storm surge inundation graphic for ET storms; and

• A map depicting the arrival of tropical storm force winds.

With OMB approval, the NWS has already interviewed a select group of partners about these products (see (ICR Reference Number 201111-0648-005; approved 1/13/15 for the tropical storm force winds graphic research and ICR Reference Number 201111-0648-005; approved 1/23/15 for the storm surge watch/warning and ET storm surge map research).

In the case of the storm surge work, the NWS interviewed only NWS Weather Forecast Offices (WFOs). These forecasters helped to explain their regions vulnerabilities and their preferences, but they do not represent NWS partners, such as emergency managers and broadcast meteorologists, who may have different perspectives. Additionally, the perspectives of the WFOs were split, with no clear conclusions: about half of the WFOs were in favor of the new watch/warning; about half were not. The forecasters also expressed some concerns about the utility and design of the storm surge map (e.g., that the map does not visualize ET threats that are often more severe than inundation, such as waves; that the water level ranges should be tailored to local conditions; and that the map could be difficult to explain) that would be useful to explore with NWS partners—to see if they have similar or other concerns.

In the case of the arrival of tropical storm force winds graphic, the NWS spoke with six broadcasters, mostly at a national level, and six emergency managers in a few locations. The input from these exploratory interviews provided enough feedback for the NWS to revise the maps by changing color schemes, the way time intervals are expressed, titles, and legends. The NWS is now in the process of incorporating this feedback to create new prototypes and would now like to test these revised graphics with a larger group of emergency managers, forecasters, and broadcast meteorologists in key cities and coastal locations where this information is critical for planning and evacuation purposes.

To further this research, the NWS seeks to conduct additional data gathering to explore key issues that have come to light in the interviews. Because all three of these new NWS products are relevant to some of the same communities, the NWS seeks to conduct focus groups on all three products in the following locations:

• New York City, Long Island, NY (July 18-19 proposed)

• Charleston, SC (June 8-9 proposed)

• Melbourne, FL (July 6-9 proposed)

• Wakefield, VA (August 5-7 proposed)

Additionally, the NWS is looking to conduct an additional interview on only the storm surge warning/storm surge inundation map for extra-tropical storms on the West Coast, and so is planning a focus group in Portland, OR (August 18-19, proposed).

The NWS is also looking to conduct an additional interview on the Gulf Coast for just the arrival of tropical storm force winds graphic, and so is looking at focus group in Houston, TX (May 21-22, proposed).

The numbers and types of participants are as follows:

• New York City, Long Island, NY: 2 groups of state/local/tribal emergency managers, 1 group of broadcast meteorologists

• Portland, OR: 1 group of state/local/tribal emergency managers; 1 group of broadcast meteorologists

• Charleston, SC: 1 group of WFO forecasters, 1 group of state/local/tribal emergency managers; 1 group of broadcast meteorologists

• Melbourne, FL: 1 group of WFO forecasters, 2 groups of state/local/tribal emergency managers; 1 group of broadcast meteorologists

• Wakefield, VA: 1 group of state/local/tribal emergency managers; 1 group of broadcast meteorologists

• Houston, TX: 1 group of WFO forecasters, 1 group of state/local/tribal emergency managers; 1 group of broadcast meteorologists

2. Explain how this survey was developed. With whom did you consult regarding content during the development of this survey? Statistics? What suggestions did you get about improving the survey?

The NWS contracted with Eastern Research Group, Inc. (ERG) on the development of the focus group scripts. ERG has significant experience assessing technical assistance provided by federal agencies through detailed interviews, focus groups, and surveys that focus on customer satisfaction and outcome attainment. To develop the scripts, ERG worked with Dr. Betty Morrow, a well-known and respected social scientist, who has conducted many risk communication and risk behavior studies related to the public’s response to hazardous weather warnings and barriers to taking the appropriate protective actions. ERG also worked with Robbie Berg, a forecaster at the National Hurricane Center, to develop the focus group script for assessing the arrival of tropical storm force winds graphic.

Suggestions for improving the focus group scripts included developing maps showing storm surge and arrival of tropical storm force winds tailored to each region and beginning the focus groups with a conversation about recent events to engage participants.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

The NWS WFO in each location will invite attendees from communities most impacted by TC/ET storms. WFOs have strong working relationships with emergency managers and broadcast meteorologists in their regions, and will reach out to these individuals via phone calls and emails to personally invite them to the focus groups. We anticipate each focus group will consist of 12 to 15 individuals.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

The NWS will use the data resulting from the storm surge research to assess regional vulnerability and understanding of storm surge associated with ET storm surge on the Eastern and Western U.S. coasts and to discern any barriers to implementing a new storm surge warning for life-threatening storm surge events. The focus groups will help illuminate whether the NWS should replace coastal flood products with the storm surge watch/warning—or if the NWS should retain the coastal flood products and add the storm surge watch/warning to the current suite of products. The NWS will also use information gleaned from the focus groups to determine if a storm surge map for ET events would be useful, and if so, who should produce it and how should it be tailored for different regions.

The NWS will use the data on the arrival of tropical storm force winds to tailor its graphic to the needs of its users and ensure its compatibility with the use of a computer program called HURREVAC, a privately-developed software program used by government emergency managers to track and analyze the threat for hurricanes. The HURREVAC program is supported by the Federal Emergency Management Agency, the U.S. Army Corps of Engineers, and NOAA.

The data do not directly contribute to a GPRA measure.

NWS is not using any statistical methods to select participants from the population and will select all population members in the discussion group and interviews.

2. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes 45 federal government employees (15 each in SC, TX, and FL), 120 emergency managers (30 in NY, 30 in FL, 15 in TX, 15 in OR, 15 in VA, and 15 in SC) and 90 broadcast meteorologists (15 in each of the 6 locations).

Category of Respondent	No. of Respondents	Location	Participation Time (hours)	Burden (hours)
Federal Government	45	SC, TX, FL	2	90
State, Local, Tribal Government	120	NY, OR, SC,FL, VA, TX	2	240
Private Sector	90	NY, OR, SC, FL, VA, TX	2	180

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Each WFO will invite attendees from communities most impacted by TC/ET storms. WFOs have strong working relationships with emergency managers and broadcast meteorologists in their regions, and will reach out to these individuals via phone calls and emails to personally invite them to the focus groups. We anticipate each focus group will consist of 12 to 15 individuals and be two hours long.

Statistical Method for Stratification and Sample Selection

The NWS is not using statistical methods for collecting these data.

Estimation Procedure and Accuracy

The NWS does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates in not meaningful to calculate

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The WFOs will utilize their existing relationships with the emergency managers and broadcast meteorologists to personally invite each individual to attend the focus groups. To further maximize attendance of emergency mangers, we will schedule the focus groups at convenient locations and on dates that do not correspond to major events in the cities (such as sporting events or festivals that might require emergency management oversight). To maximize broadcaster attendance, we will schedule the focus group between the morning and evening broadcasts and at venues close to news stations.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The focus group scripts for the storm surge research were informed through a series of 12 interviews that ERG conducted with WFOs in regions most impacted by ET storms (ICR Reference Number 201111-0648-005; approved 10/23/14). For the arrival of tropical storm surge research, the scripts were informed by a series of interviews we conducted with six emergency managers and six broadcast meteorologists. These interviews have served to clarify the wording of the questions to be asked in the focus groups, as well as highlight areas that warrant additional probing in a focus group setting.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Eastern Research Group, Inc (ERG) of Lexington, MA to design the interview guide and discussion questions, implement the data collections, and analyze the results. ERG will present a summary of the results to the NWS (including all participating WFOs) for their review and feedback. ERG’s project manager for this work is Linda Girardi (703-841-0501; [email protected]).

6