Supplemental and Part B questeions

A. Supplemental Questions for DOC/NOAA Customer Survey Clearance

(OMB Control Number 0648-0342)

1.      Explain who will be conducting this survey. What program office will be

conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

The Analyze, Forecast and Support Office (AFSO) will be leading the development and will conduct the survey.  The purpose of this survey is to collect feedback on possible alternatives to our current hazard messaging system.  The customers will be the public, including external NWS stakeholders.  The surveys will be advertised on the NWS Homepage, via partner mailing lists and on various social media outlets.

2.      Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

This survey was developed by working with risk communication specialists from NOAA and the University Corporation for Atmospheric Research. The Haz Simp team received feedback from these experts on how to appropriately phrase the questions.

3.      Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

A link to the survey will be provided from the NWS Homepage as well as on the new HazSimp Project Website.  Here, the viewers will have the option to review examples of the consolidated forecasting products and answer the questions via Survey Monkey. The Haz Simp team has been working with the NWS Office of Communications on how to appropriately advertise the surveys on social media (e.g., Facebook, Twitter).  

**This is a web-based survey and the Haz Simp team understands that this is limited to only those who utilize the Web**

4.      Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

The feedback from the four hazard messaging alternatives will be presented to NWS leadership after a detailed analysis is completed. At this time, it will be decided which prototype (or a combination thereof) will move forward in the 2017 generalizable survey.  This work, alone, will not be used to premise any changes to products and services.

This survey is not intended to measure a GPRA measure.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.      Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

No structured selection/sampling method will be used, as this will be a close ended survey without any targeted advertising beyond the page itself. Thus, the people most likely to take the survey are regular NWS web service users and any close contacts they refer to the survey. We estimate up to 3000 total responses ranging from our external stakeholders (emergency managers, broadcasters, the media, and those who work at Universities/Institutions). The HazSimp team will utilize the same questions and will be testing multiple hazards throughout the year (Winter Weather, Flooding, Extreme Heat, and Severe Thunderstorms).

Of the 3000 we are estimating, we believe that 40% will be from NWS external stakeholders. The NWS stakeholders will be made aware of the demonstration through various Public Information Statements (PNS).

Based on previous experience with similar surveys, those who participate from the public are likely to be heavily female, middle-aged, with slightly higher incomes and average levels of education as compared to the US population as a whole. The public will be made aware of the demonstration though social media posts (Twitter, Facebook.) Without any overt solicitation, response rates cannot be calculated.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The HazSimp team will make the survey available via weblink to SurveyMonkey for all people who visit the NWS HazSimp website. Thus, there is no methodology for sample selection; the participants will self-select into the study. We do not expect our results will generalize to the US population. This survey, alone, will not premise any changes to products and services --- rather, it will inform later work that generalizes more broadly.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

In order to maximize response rates, the Haz Simp team has limited the amount of surveys the individual may complete. The individual will only be allowed to review one of the three and our current hazard messaging system; none of the proposed questions are considered “open-ended.” Partial and non-submitted surveys will not be factored into the data analysis.   

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The internal social scientists will conduct a limited participant test (fewer than ten) before the surveys go “live”.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Vankita Brown, NWS – 301.427.9338

Denna Geppi, NOAA CFO – 240.533.9039

Kim Klockow, UCAR/NOAA/OAR visiting scientist