Nws 2015 Hurricane Products Social Science Research Project

NOAA National Weather Service

NWS 2015 Hurricane Products Social Science Research Project

Part A and B DOC/NOAA Customer Survey Clearance

OMB Control No. 0648-0342

Expiration Date: 05/31/2018

NWS 2015 hurricane products social science research project

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) and National Hurricane Center (NHC) predict tropical cyclone storm movement and impacts. The NHC issues tropical cyclone warnings, watches, and other information products through websites, through NOAA Weather Radio, and through interactions with partners such as the media, government officials, emergency managers, and community groups.

Potential Storm Surge Flooding Map for Hurricane Arthur

In 2014, the NHC issued a new, experimental storm surge graphic (the “Potential Storm Surge Flooding Map") for the first time during Hurricane Arthur. The NHC will issue the map, which is still experimental, again this summer for named storms that have the potential for life-threatening storm surge.

The NWS is looking to gather input, via telephone interviews, on how the map was actually used during these storms by its partners (primarily decision-makers, broadcast meteorologists, emergency managers, and community members). These interviews will therefore take place a few weeks after an event occurs.

The NWS also seeks to gather feedback from these partners on several local hurricane products, namely the Hurricane Local Statement (HLS)/Tropical Cyclone Valid Time Events Code (TCV/VTEC), and Hurricane Threats and Impacts (HTI) Graphics—should a storm occur this season where these products are issued. The NWS has recently modified these products and are interested in gathering input from partners on the modifications made.

2. Explain how this survey was developed. With whom did you consult regarding content during the development of this survey? Statistics? What suggestions did you get about improving the survey?

NWS contracted with Eastern Research Group, Inc. (ERG) on the development of the interview guide and questions. ERG has significant experience in conducting detailed interviews, focus groups, and surveys for federal agencies that focus on customer satisfaction and outcome attainment. To develop the interview guide and discussion questions, ERG worked with Dr. Betty Morrow, a well-known and respected social scientist, who has conducted many research projects with NWS stakeholder groups related to communicating and interpreting tropical cyclone hazards, as well as evacuation studies and risk behavior/communication studies. ERG and Dr. Morrow also worked closely with Rick Knabb, the Director of the NHC, Jamie Rhome, the NHC Storm Surge Team lead, Robert Berg, an experienced NHC forecaster, and Pablo Santos, the meteorologist-in-charge of the local Miami Weather Forecast Office (WFO).

The interview questions focus on how organizations or individuals accessed the products, how they used the products, and whether the products had any effects on decision-making.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

The survey will be in the form of one-on-one telephone interviews in a webinar format. Interviewees will be able to view static screen shots of the products issued through a Go-To-Meeting type of format. For each event, the NWS/NHC will provide an initial list of 20 individuals to interview based on its knowledge of the partner community and how the products were used during any storm that occurs in the 2105 season. The list will be derived from conversations that the NWS/NHC has had with these individuals during or after the storm; these interviewees include 6 emergency managers, 6 broadcast meteorologists, and 8 government officials. The interviewees with broadcast meteorologists will include individuals from private entities, such as The Weather Channel, that used the map on air during the hurricane.

ERG will contact all of interviewees to introduce the project, our role in the project, and request that an interview be scheduled at a convenient time. NWS expects that 100 percent of the interviewees will schedule an interview with ERG, given the importance and level of interest in this product.

In scheduling each interview, ERG will introduce ourselves, explain our affiliation, describe what information we are seeking, and how long the interview will take:

My name is XX. I am with Eastern Research Group (ERG), and I am calling to schedule an interview with you on behalf of NOAA’s National Weather Service. This interview is voluntary, [emphasize] and it focuses on hurricane forecast products used during Hurricane X in X LOCATION in 2015. Did you see the Tropical Storm/Hurricane XX Storm Surge Flooding Map from the National Hurricane Center during this storm? [If no, thank participant but discontinue scheduling. If yes, continue scheduling interview.] The interview should be 40 to 60 minutes long.  Do you have any availability over the next couple of weeks to schedule this interview? We will not release your name or information that could identify you as part of this interview process or in our subsequent reports to the National Weather Service.

ERG will generate detailed notes from the interviews.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

NWS/NHC will use the information resulting from this data collection to help guide refinements (e.g., exceedance levels such as 10 percent versus 30 or 50 percent; flooding threshold such as under one foot as opposed to under 18 inches or under three feet) for the “Potential Storm Surge Flooding” map while the graphic is still in experimental use. This information will also help NWS/NHC determine if additional education, marketing, training, or partner coordination is warranted to ensure that organizations and individuals are aware of the map, know how to access it, know how to use it, and are being consistent and accurate in their communications about the map.

NWS will use the information from the data collection about the local forecast products to revise these products and/or consider additional outreach or training on these products.

NWS/NHC will have ERG perform thematic analyses on the detailed interview notes to identify key findings and recommendations.

The data do not directly contribute to a GPRA measure.

NWS/NHC is not using any statistical methods to select participants from the population and will select all population members in the interviews.

2. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes an initial list of 20 individuals that will be obtained from the NWS/NHC (6 emergency managers, 8 government officials, and 6 media/broadcast meteorologists).

There is no way to predict how many storms will occur in 2015; therefore this request is for one or more events, as the survey could be repeated with the same types and number of respondents if additional events occur in different locations. Subsequent interviews will most likely be with different individual given that storms will likely occur in different locations. However, certain interviewees (e.g., FEMA officials, national TV broadcasters) could be interviewed more than once as they could be involved in multiple events. These interviews would not be duplicative since the storm surge flooding map and local forecast products will be different for every storm.

No statistical methods are being used in the initial participant selection.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical Method for Stratification and Sample Selection

The NWS is not using statistical methods for collecting these data.

Estimation Procedure and Accuracy

The NWS does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates in not meaningful to calculate

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection which may be repeated for additional weather events.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

ERG will contact all of interviewees to introduce the project, our role in the project, and request that an approximately one-hour interview be scheduled at a convenient time. NWS expects that 100 percent of the interviewees will schedule an interview with ERG and Dr. Morrow, given the importance and level of interest in this product and that these interviewees were in prior contact with NWS/NHC during or after the storm.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The NWS consulted with Eastern Research Group, Inc. (ERG) on the development of the interview guide and questions. ERG has significant experience assessing technical assistance provided by federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer satisfaction with services. The interview guide and question development process was informed through discussions with NHC and NWS staff.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the interview guide and discussion questions and implement the data collections. ERG’s project manager for this work is Linda Girardi (703-841-0501; [email protected]).

4