0342 ss 042018 Part A revised2

SUPPORTING STATEMENT

NOAA CUSTOMER SURVEYS

OMB CONTROL NO. 0648-0342

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

This is a request for extension of a generic clearance for voluntary customer surveys to be conducted by NOAA program offices. In accordance with Executive Order 12862, the

National Partnership for Reinventing Government, and good management practices, NOAA offices seek to be able to continue to gather customer feedback on services and/or products, which can be used in planning for service/product modification and prioritization.

Under this generic clearance, individual offices would continue use of approved questionnaires and develop new questionnaires, as needed, by selecting subsets of the approved set of

collection questions and tailoring those specific questions to be meaningful for their

particular programs. These proposed questionnaires would then be submitted through a fast-track request for approval process. A proposed questionnaire would be submitted to the NOAA Clearance Officer. If the latter finds that the proposal appears to be consistent with the generic clearance, the proposal would be forwarded to NOAA’s OMB Desk Officer for fast-track review. The generic clearance will not be used to survey any bodies NOAA regulates unless precautions are taken to ensure that the respondents believe that they are not under any risk for responding or for the contents of their responses; e.g. in no survey of such a population will the names and addresses of respondents be required. Currently there are no such surveys being submitted for approval. In some instances there are options to provide contact information for voluntary follow-up.0

Two sets of survey questions (included as supplementary documents) are used for generation of program-level questionnaires:

1) “Quantitative Questions” seeks to obtain numerical ratings from respondents on their satisfaction with various aspects of the product or service they obtained – satisfaction with the quality of the product, the courtesy of the staff, the format of and documentation for data received, and similar standard types of questions. The offices using such questions are able to determine which aspects of their program need improvement, or have improved. The rating system is intended to aid the respondents in identifying their relative level of satisfaction in particular areas, and is not generally intended to be used to establish numerical performance goals or as part of any complex statistical analyses over time. The potential benefits of the latter are outweighed by the difficulties in ensuring that the data is unbiased and fully representational of customers.

2) “Qualitative Questions” are more focused on who is using the product and service, how it is being used, and the medium or format in which the respondent would like to see data provided. The respondent is also given an opportunity to make specific suggestions on what new products or services should be offered or on how existing products or services could be improved.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

The responses to the quantitative questionnaires will be used by the sponsoring program office to

determine the customers’ satisfaction with the level of service and products delivered,

and identifying perceived weaknesses in those products or services or gaps in services. Information such as this will be used to help direct program improvement efforts.

The uses of the qualitative questions are somewhat different. Rather than seeking information on the degree of customer satisfaction, the objectives are more complex. Questions 1, 5, 6, and 7 seek information on what product/service was received, suggestions about improving the product or its format, and suggestions for other products or services. This information will assist the program office in better identifying the needs of customers by providing more specific data. For instance, responses concerning formats will be used to help determine which products the users are most interested in seeing through the program Web site. Questions 2, 3, 4, and 9 seek information that will help the program office identify the types of users for specific products and how they use those products.

These NOAA line offices: National Ocean Service (NOS), National Marine Fisheries Service (NMFS), National Environmental Satellite, Data and Information Service (NESDIS), National Weather Service (NWS) and Oceanic and Atmospheric Research (OAR) have been expending significant effort to review, report on, and act on the information gathered from their surveys. Many programs have used the NOAA Web site Customer Survey, posted on their individual program area sites, soliciting responses specifically about those sites. Response information utilization includes a wide variety of modifications made to the content, scope and navigation of the program Web sites.

Some surveys solicit comments on how to maintain or improve access to program data. Others provide information about customer usage and their diversity, and allow notification to subsets of user respondents of program or data changes applicable to them, rather than broadcasting emails to the complete user universe. Still others gather feedback on experimental products, to be used in product modification as indicated.

The most recently approved and ongoing surveys for which NOAA is requesting renewal are listed in the table below and described in following pages, respectively. In 2017 and early 2018, there were 15 new surveys approved by OMB under this generic collection: one from NMFS, five from NOS, six from NWS, one from NESDIS, one from OAR and one from the Technology Partnership Office (over the 3-year period, 41 surveys were approved, in addition to the ongoing Web site surveys).

Survey Name	Responses*	Burden Hours*
NOAA Web site Customer Satisfaction (ongoing - administered through NOS, all line offices other than NWS, results sorted by and available to each participating program)	96,558 over 3 years	8,046 over 3 years
NWS - Experimental Products/Services (ongoing)	279,000 over 3 years	23,256 over 3 years
3. National Hurricane Center and NWS: The National Oceanographic and Atmospheric Administration’s National Weather Service and National Hurricane Center have been updating and improving their products, services, and information to effectively convey tropical cyclone risk and uncertainty, resulting in more resilient behaviors that reduce losses of life and property. The National Hurricane Center seeks public feedback on a new experiment product for the 2017 Hurricane Season, the arrival of sustained tropical storm force winds	126	84
NATIONAL WEATHER SERVICE (NWS) 2017: Identifying Key Partners/Users of Weather Prediction Center Products and Mapping Related User Decision-Making: NOAA’s National Weather Service (NWS) Weather Prediction Center (WPC) seeks to gather feedback via a web-based survey to better understand its customers and how to meet their needs. The WPC is one of the NWS’s National Centers for Environmental Prediction (NCEP). Located in College Park, Maryland, the WPC provides forecast, guidance, and analysis products and services to support the daily public forecasting activities of the NWS. It also provides tailored support for other government agencies, such as the Federal Emergency Management Agency (FEMA), in emergency and special situations and serves as a backup to the National Hurricane Center. WPC products cover hydrometeorological conditions (from intense rain to locally or regionally heavy snowfall) in a one- to seven-day range.	2,000	2000
NWS Hazard Simplification Surveys
a. Hazardous Weather Messaging Case Study Survey for NOAA’s National Weather Service Hazard Simplification Project. We estimate up to 3,000 total responses ranging from our external stakeholders (emergency managers, broadcasters, the media, and those who work at Universities/Institutions). The HazSimp team will utilize the same questions and will be testing multiple hazards over a 3-year period (Winter Weather, Flooding, Extreme Heat, and Severe Thunderstorms).	2,500	900
b. Public focus groups to assess fire weather services: Current NWS fire weather products and services were not originally intended for the public. Instead, they were created to alert fire and land management agencies. However, these products are commonly broadcast through public media outlets and displayed on web pages. Therefore, the NWS is seeking input on how the public uses NWS fire weather products and services, including what decisions or actions individuals take based on this information and what other information they may need that is currently not included in the NWS fire weather products and services.	192	80
c. Public Survey for NOAA’s Fire Weather Risk Messages This survey builds on a set of focus groups that was approved under this control number on 3/13/17 (Public Focus Groups to Assess Fire Weather Services).	900	225
d. Experimental Snow Products Focus Group: The NWS is experimenting with different methods and visualizations of showing probabilistic information, using different colors, categories and other visualizations. As such, the NWS is looking for input on how to optimize the design and communication of NWS experimental probabilistic snowfall accumulation products. This input is intended to feed into the larger NWS effort to communicate uncertainty and impacts across NWS service areas for the protection of life and property. To accomplish this, the NWS seeks to conduct focus groups with a purposive sample of WFO partners, including emergency managers (BMs), state and local officials (e.g., school superintendents, state departments of transportation, public works), and a few broadcast meteorologists. The purpose of the focus groups is to gather partner feedback on the visual design of the graphics and overall snowfall probability communication to see how it improves NWS partner decision making.	240	120
Transportation Weather: NWS WR developed the survey to help evaluate, qualitatively, the utility of a new web page that provides a one-stop shop for transportation weather information. We use Google Analytics for most of the quantitative evaluation of the web page. We consulted with the lead developer of the web page to ensure the most important questions were asked. A suggestion we received to improve the survey was to ask for any complaints that users have for the web page. We were unanimous in trying to make this survey short to ensure maximum participation.	100	17
Climate Outlook: The survey will be conducted by the University of Maryland’s Environmental Decision Support Science Lab at the Earth System Science Interdisciplinary Center for National Oceanic and Atmospheric Administration’s (NOAA) Climate Prediction Center (CPC). CPC provides medium and long term temperature and precipitation outlooks to visitors of the CPC webpage. The survey will be conducted under the Cooperative Institute for Climate and Satellites (CICS) grant “Identifying Users, Diagnosing Understandability Challenges, and Developing Prototype Solutions for NOAA Climate Prediction Center’s Temperature and Precipitation Outlooks (NA14NES4320003).” This study examines how to better communicate risk and uncertainty associated with the forecasts of NOAA National Weather Service CPC with the goal to inform the potential adjustments or revision of the CPC’s temperature and precipitation outlook visuals.	400	100
NOS Coast Survey Website Usability: The focus of this survey is to validate our assumptions about the usability of our new website design. Questions were developed in order to ensure that customer were able to quickly and easily access our service and products from our redesigned website.	1,000	167
NOS: Needs Assessment Survey for Monitoring Data Synthesis The purpose of this survey is to understand how long-term estuarine monitoring programs can better address regional and national coastal management needs. NOAA will use these survey results to help guide investments in monitoring data synthesis for research, education, and outreach programs.	100	50
NOS: National Marine Sanctuary-related surveys: a) The Office of the National Marine Sanctuaries’ (ONMS’s) Greater Farallones Association will be conducting this research. This survey will collect information about customer satisfaction associated with recreational activities that visitors engage in within GFNMS. The visitors (local and/or non-local) to the sanctuary have access to beaches and coastline, sanctuary visitor centers, or other sanctuary sponsored exhibits. This survey will help to assess staff’s effectiveness of messaging and marketing sanctuary activities and opportunities. b) Get Into Your Sanctuary Day: This survey’s intent is to assess visitors’ (customer) satisfaction associated with services provided as part of the National Get into your Sanctuary (GIYS) Day effort. These services include educational programs, access to recreational activities, and informational events and exhibits, as well as the promotional materials used to alert potential customers about the events themselves. The customers are those who visit the sanctuary, sanctuary visitor centers or other sanctuary sponsored exhibits on GIYS Day. These services are provided to users of the resources on site. c) Partnership Survey: this survey’s intent is to assess partners’ satisfaction associated with services and products provided by ONMS to and in conjunction with our partners. These services include educational programs, informational events and exhibits, staff support and staff time. In the context of this survey our partners are the customers. ONMS collaborates with other organizations to further our educational goals and objectives. Partners include non-profit organizations, zoos and aquaria, universities and schools. Use of this data will help to improve the quality of partnerships and the outputs of those partnerships (such as educational products/services, interactions with the public and improvement to sanctuary resources	750 720 150	50 48 13
NESDIS: Geostationary Operational Environmental Satellite (GOES): One survey will be conducted for the GOES Rebroadcast (GRB) user group of the GRB radio frequency (RF) link, the other will be conducted with the GOES Data Collection System (DCS) user group. NESDIS provides secure and timely access to global environmental data and information from satellites and other sources to promote and protect the Nation's security, environment, economy, and quality of life. The services are provided through internet access and direct readout ground stations that can receive the satellite RF signals.	175	15
The Technology Partnership Office oversees two programs that help foster product development and technology innovation and transfer: NOAA’s Small Business Innovation Research (SBIR) program and the Technology Transfer Program. The SBIR program encourages small business innovation by providing federal research and development (R&D) funding critical to small businesses that develop products with the high probability of commercialization and align with NOAA’s mission. NOAA’s Technology Transfer Program encourages the collaboration and transfer of information through the use of Cooperative Research and Development Agreements (CRADAs), which can speed the commercialization of NOAA-developed technology and provide other collaborator benefits, such as protecting research results from public disclosure for a five-year period of time. NOAA’s TPO is interested in the extent to which the two programs are meeting the needs of project partners, their satisfaction with the programs, and the results that are being generated by the programs in terms of meeting the overall program objectives.	126	42
Total for All of 2015-18, proxy for 2018-21 (from most recent Notice of Action + 3x annual Web site surveys in first two rows)	385,037	46,752
Additional burden estimated for new surveys in 2018-21	15,000	5,000
TOTALS	400,037	51,752
Annualized Totals	133,345	17,250

*Other than the ongoing Web site surveys (first two rows) these are one-time surveys

Ongoing Surveys:

The NOAA Web site Customer Satisfaction survey is administered by the NOS and implemented on 18 separate Web sites, for programs under NOS, NMFS, NESDIS and OAR. Data is collected into a single database, separated by the individual Web site on which it is

implemented. An administrator is able to view survey results for each Web site and act upon these results as appropriate.

National Weather Service

Web site Experimental Products/Services Surveys

NWS makes extensive use of the survey to collect user feedback on proposed changes, additions, or terminations of Official and Experimental Products/Services. Under NOAA’s Partnership Policy, we are required to collect user feedback on changes to environmental information and services. For proposed new products and services, the survey responses have provided invaluable feedback from private and commercial users, as well as government partners, to local, Regional, and National decision makers. Survey allows for fairness and openness in proposed changes, and assists the decision makers in determining what actions and services are appropriate for NWS to provide. All survey responses are carefully evaluated and considered in determining the appropriate action. For proposed termination of services/products, the surveys have provided user input, allowing NWS to ensure data requirements were met while still being able to consolidate Web services. Since June 1, 2015, there have been approximately 300,000 Customer Survey responses for Official and Experimental Products/Services.

(Also listed in the table of recently approved ICs) Hazardous Weather Messaging Case Study Survey for NOAA’s National Weather Service Hazard Simplification Project. We estimate up to 7,400 total responses ranging from our external stakeholders (emergency managers, broadcasters, the media, and those who work at Universities/Institutions). The HazSimp team will utilize the same questions and will be testing multiple hazards over a 3-year period (Winter Weather, Flooding, Extreme Heat, and Severe Thunderstorms).

Hazard Simplification Institutionalization Study Survey: NOAA’s National Weather Service (NWS) issues watch, warning, and advisory (WWA) products to help communities prepare for and respond to hazardous weather in order to protect people’s lives and property. These various WWA products (e.g., flash flood watch, tornado warning, heat advisory) are communicated to the public through websites, smart phones, television programs, radio broadcasts, and NOAA Weather radio. NWS customers include weather professionals, transportation and aviation officials, emergency management personnel, public works officials, broadcast meteorologists and other media, the public, and others. The NWS seeks to conduct a survey to gather feedback from organizations that use hazardous weather warning information to discern the degree to which any specific types of watches, warnings, and advisories (or the actual terms “watch,” “warning,” and “advisory”) are embedded in their decision-making, laws, policies, operating procedures, technologies, by-laws, or other activities or processes. The goal of the data gathering is to gauge the level of institutionalization of these products and terms in different industry sectors as the NWS contemplates potential improvements to the present WWA system.

National Marine Fisheries Service

The Marine Fisheries Advisory Committee (MAFAC) and its Climate Resilience Task Force (CRTF) will be conducting an ongoing survey to assist it in its work to provide advice to NOAA Fisheries. MAFAC is a Federal Advisory Committee and has been asked to provide recommendations to NOAA Fisheries on topics related to addressing climate change impacts on fisheries, including regarding effective communication strategies for key fisheries audiences and stakeholders. NOAA provides information on past, current, and possible future climate-related changes to fishermen, communities, cooperatives, managers, and Fishery Management Council members to help them make decisions and adapt to the changing climate. This information is provided through a variety of methods including websites, scientific reports, and presentations. The ongoing survey will identify ways to increase access, delivery, and use of this information to empower local actions and strategic decision-making that is adaptive to change. It will also strengthen the value proposition of NOAA to provide climate science. The Communications working group determined that a survey of those who are likely to use this information is an essential step in formulating these recommendations. Estimated annual hours: 251.

National Ocean Service (NOS)

NOS Coast Survey Website Usability: See Table of recently approved ICs.

NOAA will retain control over the information and safeguard it from improper access, modification, and destruction, consistent with NOAA standards for confidentiality, privacy, and electronic information. See response to Question 10 of this Supporting Statement for more information on confidentiality and privacy. The information collection is designed to yield data that meet all applicable information quality guidelines. Prior to dissemination, the information will be subjected to quality control measures and a pre-dissemination review pursuant to Section 515 of Public Law 106-554.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

Currently, most surveys are conducted via email. Some surveys are mailed to their customer lists. Web site customer satisfaction and some product satisfaction surveys are posted on the applicable Web sites, with monitoring to eliminate most, if not all, frivolous responses.

4. Describe efforts to identify duplication.

A team with representatives from all of NOAA’s major organizations helped to develop the

questions and identify any current efforts. While there may be other customer surveys planned

that will be the subject of separate clearance requests, NOAA is confident that the procedures in

place ensure that no current or future survey will duplicate any other similar survey within the

program area involved. The NOAA PRA Clearance Officer alerts those planning a survey, through this OMB Control No. or others, about similar/overlapping surveys being planned or conducted.

Three years ago, DOC implemented an overlapping generic information collection which includes customer surveys (OMB Control No. 0690-0030). NOAA uses this vehicle for surveys

for which less robust results are needed, as very little description/documentation is needed. The vast majority of NOAA’s customer surveys are still conducted under o0648-0342.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

While small businesses will be respondents to some of the surveys, the burden on any respondent

is expected to be minimal. Response to all surveys will continue to be voluntary.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If these surveys were not conducted, the program offices have significantly less information for determining which areas of their programs should be modified, and how they might be modified, to provide better service to the public. The frequency of surveys will vary. Some will be conducted once a year, while others will be ongoing. The ongoing approach is deemed especially useful when asking questions about specific products and formats, rather than about general satisfaction with a program. This more frequent feedback may allow the program office to get helpful information from respondents at the time a product is received or a Home Page is used, rather than later as part of an annual survey.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

Respondents who choose to complete surveys on the Web may be responding in less than thirty calendar days from when they receive the request. In those surveys where a questionnaire is attached to each product delivery, a person who frequently orders products will receive more than one request quarterly, but all responses are voluntary.

8. Provide information on the PRA Federal Register Notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

A Federal Register Notice soliciting public comments was published on November 13, 2017 (82 FR 52279). No comments were received.

Comments were received from two individuals who took National Ocean Service Web site surveys. Neither had any issues with the burden estimate or clarity of instructions. Regarding usefulness of the information, on commented that she planned to use it for educational purposes, as well as working on local ocean clean-up projects. The other suggested more photographs, and making a referenced publication more easily available.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

No payment or gift will be given to any respondent.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

There will be no assurance of confidentiality; rarely, we ask for contact information for follow-up, but provide a Privacy Act Statement on the form in that case.

Privacy Act System of Records Notice: COMMERCE/NOAA-11, Contact Information for Members of the Public Requesting or Providing Information Related to NOAA's Mission.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

No sensitive questions will be asked.

12. Provide an estimate in hours of the burden of the collection of information.

The three-year total for 2015-18 was 385,037 responses and 46,752 hours. Allowing for a slightly greater number of surveys in the next three years, we are estimating that we will have totals of 400,037 responses and 51,752 hours over the next three years. Annualized responses will be 133,345 and hours, 17,250.

It is possible that there may be more than one response per respondent per year; this information is not tracked, but each entry on Web-based surveys has a time stamp.  More than one entry per respondent can be reasonably expected as new information and products appear. However, entries in close succession, which generally could be construed as frivolous, are eliminated.

13. Provide an estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection (excluding the value of the burden hours in Question 12 above).

There will be no cost to respondents. For mailed surveys, envelopes with pre-paid postage will be supplied.

14. Provide estimates of annualized cost to the Federal government.

All surveys will be conducted and analyzed in-house as part of program planning and thus there is no additional cost beyond regular staff time.

15. Explain the reasons for any program changes or adjustments.

Adjustments:

The burden for the last three years was slightly less than expected, but in case of a greater number of surveys in the next three years, we added an estimated 15,000 responses and 5,000 hours for the 2018 extension.

	2015 estimates	2015 -2018 actuals	2018 estimates
Responses over 3 yr	546,594	385,037	400,037
Hours over 3 yr	50,436	46,752	51,752

16. For collections whose results will be published, outline the plans for tabulation and publication.

Aggregated results will be posted on the applicable Web sites as needed, to share with users as part of information to be imparted about planned program or product changes.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

All surveys will display the OMB expiration date.

18. Explain each exception to the certification statement.

No exceptions are requested.

12