Supplemental and Part B questions

NOAA National Weather Service

Weather Prediction Center (WPC) Survey

Part A and B DOC/NOAA Customer Survey Clearance

OMB Control No. 0648-0342

Expiration Date: 05/31/2018

NATIONAL WEATHER SERVICE (NWS) 2016: Identifying Key Partners/Users of Weather Prediction Center Products and Mapping Related User Decision-Making

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) Weather Prediction Center (WPC) seeks to gather feedback via a web-based survey to better understand its customers and how to meet their needs. The WPC is one of the NWS’s National Centers for Environmental Prediction (NCEP). Located in College Park, Maryland, the WPC provides forecast, guidance, and analysis products and services to support the daily public forecasting activities of the NWS. It also provides tailored support for other government agencies, such as the Federal Emergency Management Agency (FEMA), in emergency and special situations and serves as a backup to the National Hurricane Center. WPC products cover hydrometeorological conditions (from intense rain to locally or regionally heavy snowfall) in a one- to seven-day range.

While the WPC knows that its information serves both internal NWS audiences and external stakeholders, it seeks a deeper understanding of which specific entities and individuals come to its website for information, what WPC products/services they use, when they use these products, and how they use the products in their decision-making and communication. The WPC intends to use this information to better focus its services and product suite to meet the needs of its users.

2. Explain how this survey was developed. With whom did you consult regarding content during the development of this survey? Statistics? What suggestions did you get about improving the survey?

The NWS contracted with Eastern Research Group, Inc. (ERG) on the development of the survey. ERG has significant experience in conducting detailed interviews, focus groups, and surveys for federal agencies that focus on customer satisfaction and outcome attainment. To develop the survey, ERG worked with Dr. David Novak, the director of the WPC, as well as various WPC forecasters.

In addition, prior to conducting the survey, ERG conducted a series of interviews with known users of WPC information, including forecasters from NWS Weather Forecast Offices (WFOs) and River Forecast Centers (RFCs), as well as FEMA and DHS. These interviews revealed clear patterns of how partners use and receive products that nicely translate into a survey. For example, using products for general situational awareness versus incorporating into a forecast model was a key theme. Suggestions for improving the survey focused on showing product names and linking to the product on the web. The interviewees said that remembering names of products was a challenge leading them to suggest weblinks.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

The link to the web-based survey will be disseminated via the WPC website and its social media channels. Any user that comes to these platforms will have an opportunity to take the survey.

To date, the WPC has not analyzed traffic to its website via Google Analytics or other statistical means, so it is not known how many people visit the WPC site each day. The WPC does have a Twitter account with 32,400 followers. The WPC recently conducted a voluntary survey asking website users for their input on a new, experimental WPC homepage. From April 4, 2016, to August 26, 2016, the WPC collected 151 survey responses.

Based on these data, it is possible that as many as 10 percent of all WPC Twitter followers (3,240) will take the survey; however, given the response rate of 151 individuals (over the course of fourth months) for the prior survey, the number of respondents is more likely to be much lower. Given all of this information, we estimate the sample size at around 2,000. The WPC will actively promote the survey on its website and social media channels to increase the rate of response.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

NOAA/NWS/WPC will use the information resulting from this testing to help determine who is using WPC products, whether the WPC is meeting their needs, and if there are any suggestions for improving WPC services or products that should be taken under consideration. The survey will also help determine which WPC products are accessed the most frequently, as well as which products are accessed infrequently or not at all.

The NWS is not using any statistical methods to select participants from the population, and the data do not directly contribute to a GPRA measure.

2. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Potential respondents will be asked to participate in the survey via a web link featured on the WPC website and promoted on the WPC’s social media platforms. The survey will include a few demographic and job/experience questions to discern how the survey respondents are using WPC data. The projected response rate and breakdown of entities responding are expected to be similar to that of a prior survey that the WPC conducted in 2016 from April 4 to August 26 (see Figure 1). That survey was also a web-based one, distributed via WPC’s website and social media channels and was designed to gather feedback on a new, experimental website design.

Figure 1. Breakdown of Survey Respondents on WPC Experimental Homepage Redesign

Entity	Number of respondents
Business/Industry	7
Individual	80
Media	7
NOAA	14
Research Institute	1
State/Local Government	8
Student/Teacher	11
University Faculty	3
University Student	2
Other	13
Not Provided	5
Total	151

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical Method for Stratification and Sample Selection

The NWS is not using statistical methods for collecting these data.

Estimation Procedure and Accuracy

The NWS does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates in not meaningful to calculate.

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The WPC plans to increase the response rate by actively promoting the survey on its website and social media platforms.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The survey was reviewed by WPC leadership and forecasters prior to its launch. It was also pre-tested with several NWS/WFO/RFO staff who use WPC data to ensure the questions are clear and comprehensive.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the interview guide and discussion questions and implement the data collections. ERG’s project manager for this work is Gina Eosco (781-704-4458; [email protected]).

5