SPC Day1 Outlook supp and Part B questions

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) Storm Prediction Center (SPC) provides watch and forecast products for hazardous mesoscale weather phenomena, including severe thunderstorms and tornadoes. SPC’s Day 1 Outlook provides probabilistic and categorical information about the likelihood of thunderstorms, tornadoes, damaging thunderstorm winds, and damaging hail for the current day. The key customers of the Day 1 Outlook include state and local emergency management personnel, broadcast meteorologists, and other private sector meteorologists. The Day 1 Outlook is made available on the SPC’s website and is updated several times each day. In addition, the Day 1 Outlook is disseminated via Twitter and Facebook.

SPC is interested in obtaining feedback from key customers about whether and how they use the Day 1 Outlook and whether and how potential changes in the spatial or temporal resolution of the Outlook would change how they use it.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? Statistics? What suggestions did you get about improving the survey?

NOAA/SPC contracted with Abt Associates Inc. on the development of the focus group protocol and the interview guide. Abt Associates worked with Dr. Rebecca Morss (of the National Center for Atmospheric Research) who has conducted extensive research on the social impacts of weather forecasts and Dr. Robert Johnston (Clark University), a leading expert in conducting focus group research. The Abt Associates team reviewed existing research on risk communication, communication of weather forecasts and communications of other natural hazards to identify key topics for the focus groups. In addition, the Abt team worked with SPC and other NOAA personnel to identify key topics that will help the government improve the Day 1 Outlook.

The focus group protocol and the interview guide ask where participants obtain weather information, how they use weather information, how they use and interpret the current Outlook products, and how changes in spatial or temporal resolution would change how they use the Outlook. As part of the focus groups and interviews, we will show participants an example of how the Outlook might change. There is a different version of the protocol for emergency management personnel and for private sector meteorologists.

A draft protocol for each audience was developed and shared with NOAA personnel for review and comment. The protocols were revised based on feedback received.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

Focus groups will be conducted with state and local emergency planning and management personnel, and with regional/national level emergency response personnel. Individual or small group interviews will be conducted with private sector meteorologists; these will be conducted at an industry event, such as the annual meetings of the National Weather Association and the American Meteorological Society. The reason for the different method is that we anticipate a greater challenge coordinating with meteorologists; individual interviews will offer more flexibility in scheduling which will help increase participation among this audience.

Emergency management personnel and meteorologists will be recruited with outreach through existing NWS and Abt professional networks, particularly through contacts with local NWS Weather Forecast Offices (WFO). WFO personnel have regular contact with the emergency management community in their region and will be able to suggest individuals that should be invited to participate. In addition, we will work through our contacts at the U.S. Army Corps of Engineers, Federal Emergency Management Agency and other organizations to identify emergency management personnel that can be invited to participate.

The target cities for focus groups are: Dallas, TX; Minneapolis, MN; Wichita, KS; Birmingham, AL; and Washington, DC. Candidate cities were identified through discussion with SPC personnel and the final list was selected based on storm/tornado risk, size of the metro area and comparison of regional socio-economic characteristics.

WFO personnel who identify prospective focus group members or interviewees will ask those individuals to contact Abt Associates by email to express their interest in participating and to provide their phone numbers. Abt personnel will follow up via email with those that contact us to express interest in participating. The email will introduce the study, describe the purpose and format of the focus group/interview, point out OMB approval, describe confidentiality and offer a contact for questions. Similar information will be communicated in follow-up telephone calls, as well as details for the focus group (location, time, etc.). In addition, we will answer any questions from invited participants via email or phone.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

Focus group and interview data will be analyzed qualitatively to identify key themes of how user groups interpret and use the current Outlook and how they might interpret and use a revised Outlook.

The focus group and interview findings will be used to draft a communications plan that will help SPC maximize the usefulness and impact of its Day 1 Outlook. The plan will present guidelines for graphically presenting higher resolution Outlooks in a way that is understandable and actionable among SPC audiences. The Plan will document how research findings support the recommendations and it will highlight remaining questions or issues that may need to be resolved.

No statistical techniques will be used to generalize the results.

The data do not directly contribute to a GPRA measure.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The respondent universe includes emergency management personnel and private sector meteorologists in the target cities listed above. We will conduct outreach efforts to obtain the sample shown below.

User group	No. of Focus Groups	Participants per Focus Group	Total Target Sample
Local emergency response planners and managers	5	10	50
Private sector meteorologists, including broadcasters	N/A (interviews)	N/A	30
Regional and national emergency response planners and managers (e.g., FEMA)	1	10	10

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

• NWS is not using any statistical methodology for stratification and sample selection, or any estimation procedures.

• We do not expect any unusual problems requiring specialized sampling procedures.

• This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Potential participants will be identified through networking with WFOs and other organizations.

Once a priority list of participants is identified, WFO/USCG personnel will then reach out to request their participation in a focus group/interview and ask them to contact Abt Associates if they are interested in participating. WFO will contact invitees by email and will introduce the study, describe the purpose and format of the focus group (or interview), point out OMB approval, describe confidentiality, and offer a contact for questions. The email will ask the invitee to contact Abt Associates if they are interested in participating. WFO personnel will send one additional reminder via email to those invitees who do not respond. When an invitee declines to participate we will ask WFO personnel to invite another identified individual from that city or metro area so that we can target the desired sample size.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Abt Associates and its consultants have extensive experience developing and conducting focus groups and interviews to evaluate government programs and research public preferences and other topics. While not a formal test, the first two focus groups will be conducted in the Washington, D.C. area so that two of the lead researchers can collaborate in conducting the focus groups and in refining the protocol based on initial findings.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Abt Associates Inc. to design and conduct the focus groups and interviews. Abt’s project manager for this work is Mark Lorie (301-634-1728; [email protected]); an alternate contact is Jacky Haskell (301.347.5526, [email protected]).