0648-COU Supporting Statement Part B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Surveys to Collect Data on Use of the NOAA National Weather Service

Cone of Uncertainty

OMB Control No. 0648-xxxx

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method used. Provide data on the number of entities (e.g., establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

This set of surveys covers four sectors in regions that experience coastal or inland impacts from hurricanes. Table 4 summarizes the estimated populations for each survey, the number of respondents that will be selected, the anticipated response rate, and the targeted respondent number. These estimates are based on conversations with the NWS Weather Forecasting Offices (WFOs) and the NOAA managers of the Weather Ready Nation (WRN) Ambassadors’ Program (see Section A.12, Table 1 for more details). NOAA is not using statistical methods to select the respondents; thus, the targeted number of respondents is the number we expect to respond to the survey.

As noted in Question 12 of Section A, the survey respondents can be broken into two broad categories:

• Respondents on NWS Weather Forecast Office (WFO) lists. The National Hurricane Center (NHC) has developed a list of 55 NWS WFOs in the geographic areas of interest (i.e., locations that experienced hurricanes within recent years) consisting of 30 inland and 25 coastal areas. Each WFO has developed its own partner list based on its Core Partners and guidance provided under NWS Policy Directive 10-24 for providing impact-based decision support services. Based on information from the WFOs, we have estimated that each list will contain approximately 60 names for use in our survey. The survey will involve collecting data from all names compiled through this process.

• Respondents on targeted lists. The NWS also has targeted lists of partners from the NHC (marine sector), which contains 232 marine partners’ names. In addition, the NWS maintains a list of names on its Weather-Ready Nation Ambassadors (WRN) initiative, which works to build partnerships to strengthen partner and community resilience to extreme weather and water events. As of January 24, 2019, there were 10,561 ambassadors in the program as a whole. Based on discussions with NOAA staff that manage the WRN initiative, we assume that one-tenth of the ambassadors (1,056) fall within the four sectors of interest and that those names are evenly divided in the four sectors of interest (264 names in each of the four sectors). In addition, there are 100 ambassadors in the WRN Aviation Ambassador program, which fall within the transportation sector.

We also expect that 1) the Office of the Federal Coordinator for Meteorology (OFCM) and the U.S. Departments of Energy and Transportation will share the survey link with their partners, and 2) that some respondents will share the link with a colleague or other member of their profession, but we have not made an attempt to estimate those numbers.

Table 4. Populations and Respondent Information

Survey/ Sub-Group	Target Population									Potential Number of Respondents	Response Rate	Expected Number of Survey Respondents
	Marine			Transportation		Energy/Utilities		Tourism
	NHC1	WRN2	WFO3	WRN4	WFO5	WRN6	WFO7	WRN8	WFO9
Federal government	27		250	91	212	66	212	66	212	1,136	30%	341
State/local government		132	250	182	425	66	213	66	213	1,547	30%	464
Private sector	205	132	250	91	213	132	425	132	425	2,005	30%	601
TOTALS	232	264	750	364	850	264	850	264	850	4,688	30%	1,406

¹ There are 232 names on these lists; 27 are in the federal sector. The rest are private sector.

² The 264 names on the WRN list in the marine sector are evenly split between state/local governments and the private sector.

³ The 750 WFO names in the marine sector are evenly split among federal government, state/local government, and the private sector.

⁴ About one-half of the ambassadors in the WRN Ambassadors program in the transportation sector are in state/local government, one-fourth are in the federal government, and one-fourth are in the private sector.

⁵ About one-half of the names on the WFO lists in the transportation sector are assumed to be in the state/local government sector, one-fourth are in the federal sector, and the remaining one-fourth are in the private sector.

⁶ About one-half of the ambassadors in the WRN Ambassadors program in the energy/utilities sector are in the private sector, one-fourth are in the federal sector, and one-fourth are in state/local government.

⁷ About one-half of the names on the WFO lists in the energy/utilities sector are in the private sector, one-fourth are in the federal sector, and the remaining one-fourth are in state/local government.

⁸ About one-half of the ambassadors in the WRN Ambassadors program in the tourism sector are in the private sector, one-fourth are in the federal sector, and the remaining one-fourth are in state/local government.

⁹ About one-half of the names on the WFO lists in the tourism sector are assumed to be in the private sector, one-fourth are in the federal sector, and the remaining one-fourth are in state/local government.

The NWS will conduct an online survey to collect the desired information. The NWS’s contractor, ERG, will develop the survey in its online survey platform (Qualtrics). The NWS will distribute the survey by providing a link to potential respondents. ERG will keep the survey open for at least two weeks, allowing as many responses as possible to accumulate in that timeframe. If the target is not met after two weeks, ERG will allow the survey to remain open until the 30 percent response target is met.

We are not employing statistical methods in this survey since we do not expect to need to extrapolate to the population.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Stratification, Sample Size, Estimation Procedure, and Precision and Accuracy

As we are not employing any statistical sampling methods in this survey, we do not extrapolate the survey result to the population under consideration. The only stratification being employed involves separating the respondents into the four sectors described in Table 1.

The NWS’s primary purpose for collecting these data is to better understand the breadth of use and understanding of the Cone of Uncertainty and to assess potential changes to the Cone based on the results of this survey. No information from this survey would be used to justify decisions to change the Cone. Any changes the Cone would require depends on additional research and assessment. This is described in more detail under section B.3 below.

Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Maximizing response rates

The NWS’s contractor, ERG, will employ the following measures to maximize response rates from the potential respondents:

• The WFOs have strong knowledge of the organizations and enterprises at risk from hurricanes in their jurisdictions. In turn, these groups have a great deal of trust in the WFOs. Each WFO will send a personalized survey invitation to entities within the sectors of interest on their lists. This will help raise the visibility of the survey.

• The personalized invitation for the survey will include details about the scope and purpose of the survey so that recipients can direct it appropriately within their organizations.

• ERG has developed a survey that minimizes the burden on respondents by using good survey design. This includes developing well-written questions and limiting the number of questions to the minimum necessary.

Dealing with nonresponse

• ERG will use multiple prompts to generate responses. ERG will send a pre-notification email to respondents about the survey, an email that asks the respondent to take the survey, and then two reminder emails.

• ERG will extend the survey open period if the target response is not met after two weeks. ERG will allow the survey to remain open until the 30 percent response target is met.

Adequacy for intended uses

The purpose of the survey is to gather feedback from individuals in four key sectors (marine, transportation, energy/utilities, and tourism) and among internal government meteorologists to discern the degree to which they use the Cone of Uncertainty to make operational decisions, as well as determine whether the product helps or hinders their decision-making. The survey supports Section 104 of the 2017 Weather Research and Forecasting Innovation Act (Pub. L. 115-25, H.R. 535), which calls for NOAA to improve hurricane forecasting and communication. NOAA has no prior data on these uses beside anecdotal information provided by stakeholders. The data will provide a strong basis for NOAA to better understand how key partners in important U.S. sectors use its information in preparing for hurricanes. No decisions are being made solely from the data being collected from these surveys. Rather, these data feed into a base of information that NOAA is collecting about how the Cone of Uncertainty and other hurricane forecast products are used and interpreted.

The results of the survey will be used primarily for internal deliberative purposes. NWS may also present the results at public meetings and/or conferences as needed.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The NWS’s contractor, ERG, will perform a pre-test of the survey prior to full implementation of the instrument. ERG will select approximately 10 to 20 respondents to be pre-testers from the set of respondents identified in Table 4. The pre-testers will be respondents who are known to WFOs and who are willing to assist the NWS in this effort. Thus, the pre-test portion of the total potential respondents is included as part of this request. ERG will make changes as needed based on the pre-test.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA, to design the survey instrument, develop the sampling approach, implement the survey, and analyze the resulting data collected. The survey design team included the following individuals:

Dr. Lou Nadeau, (781) 674-7316, [email protected]; Linda Girardi, (703) 841-0501, [email protected]; Dr. Betty Morrow, [email protected], (305) 812-2125; and Jeffrey Lazo, [email protected], (970) 707-4160.

5