Supplemental and Part B questions

A. Supplemental Questions for DOC/NOAA Customer Survey Clearance

(OMB Control Number 0648-0342)

1.      Explain who will be conducting this survey. What program office will be

conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

The Analyze, Forecast and Support Office (AFSO) will be leading the development and will conduct the survey.  The purpose of this survey is to collect feedback on two optional four-colored Watch, Warning, and Advisory (WWA) maps.  The customers will be the public, including external NWS stakeholders.  The surveys will be advertised on the NWS Homepage, via partner mailing lists and on various social media outlets.

2.      Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

This survey was developed by rewording previously approved Hazard Simplification (Haz Simp) Questions (OMB approved in 2015) and pre-approved OMB quantitative questions. The AFSO, Hazard Simplification (Haz Simp) team has been working with risk communication specialists from NOAA and the University Corporation for Atmospheric Research. The Haz Simp team received feedback from these experts on how to phrase the questions.

3.      Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

A link to the survey will be provided from the NWS Homepage.  Here, the viewers will have the option to review the two maps and answer the questions via Survey Monkey. The Haz Simp team has been working with the NWS Office of Communications on how to appropriately advertise the optional maps on social media (e.g., Facebook, Twitter).  

**This is a web-based survey and the Haz Simp team understands that this is limited to only those who utilize the Web**

4.      Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

The results will be used to inform NWS as to whether to operationalize a limited color map to depict weather and water hazards and, if so, how it should be done.  Basic comparative statistical analysis of customer satisfaction data, coupled with supporting  details provided in the open-ended questions, will The survey results will help the team analyze which aspects of the maps worked, didn’t work, and what areas require improvement. Since the maps would be displayed online, a web-based sample is sufficient to represent those who are likely to see them.  This work, alone, will not be used to premise any changes to products and services.  It will be used in conjunction with other analyses.

This survey is not intended to measure a GPRA measure.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.      Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

No structured selection/sampling method will be used, as this will be an open survey without any targeted advertising beyond the page itself. Thus, the people most likely to take the survey are regular NWS web service users and any close contacts they refer to the survey. We estimate up to 500 total responses ranging from our external stakeholders (emergency managers, broadcasters, the media, and those who work at Universities/Institutions). Of the 500 we are estimating, we believe that 40% will be from NWS external stakeholders. The NWS stakeholders will be made aware of the demonstration through various Public Information Statements (PNS).

Based on previous experience with similar surveys, those who participate from the public are likely to be heavily female, middle-aged, with slightly higher incomes and average levels of education as compared to the US population as a whole. The public will be made aware of the demonstration though social media posts (Twitter, Facebook.) Without any overt solicitation, response rates cannot be calculated.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

We will make the survey available via weblink to SurveyMonkey for all people who visit the NWS HazSimp website. Thus, there is no methodology for sample selection; the participants will self-select into the study. We do not expect our results will generalize to the US population. This survey, alone, will not premise any changes to products and services --- rather, it will inform later work that generalizes more broadly.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

In order to maximize response rates, the Haz Simp team will be limiting the amount of questions, and some of the questions will not be open ended.  This will cut down on the time it takes the user to complete the survey.  Partial and non-submitted surveys will not be factored into the data analysis.   

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The internal social scientists will be conducting a limited participant test (fewer than ten) before the maps/questions go “live” for survey completion.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Vankita Brown, NWS – 301.427.9338

Denna Geppi, NOAA CFO – 240.533.9039

Kim Klockow, UCAR/NOAA/OAR visiting scientist