Supplemental and Part B Questions

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) will be conducting the survey. Integrated Water Resources Science and Services (IWRSS) is supported by a consortium of federal agencies with complementary missions in water science, observation, management and prediction: The U.S. Geological Survey (USGS), U.S. Army Corps of Engineers (USACE), and NWS. The objective of IWRSS is to design, develop, and implement a national water modelling and information services framework to infuse new hydrologic science into current water resources management, develop hydrologic techniques and decision support applications for operational use, and provide advanced hydrologic services to address growing stakeholder needs. Toward this end, IWRSS applies a cross-cutting, multi-disciplinary systems approach to address complex water problems collaboratively.

Planned IWRSS services include: a) high spatial and temporal resolution “summit to sea” analyses and forecasts for full spectrum of water budget parameters; b) short through extended range river forecasts which quantify uncertainty; c) static flood inundation map libraries and real-time flood forecast inundation mapping services which show the areal extent and depth of flood waters; d) linking river forecasts, and associated flood inundation maps, to potential socioeconomic impacts; and e) integrating the access to geospatial water resources information from multiple federal agencies through a single portal.

IWRSS stakeholders include government and private sector water resources decision makers whose interests include hydropower, emergency management, reservoir management, watershed management, agriculture, fish and wildlife, river commerce, municipal and industrial water supply, recreation, and water quality.

The IWRSS agencies are currently developing a plan to demonstrate new IWRSS capabilities in the Potomac, Delaware, Susquehanna, and Hudson River basins. The customers of this initial demonstration plan include river basin commissions and their water resources management stakeholders. Pending successful demonstration and further stakeholder engagement, the IWRSS partner agencies plan to implement new services nationally.

To allow for broad input from the river basin stakeholders, NWS is seeking approval to conduct a survey of IWRSS stakeholders in each of the four river basins in order to inform the design, development and implementation of IWRSS. Specifically, the objectives of the survey are: a) validate existing and identify new gaps in water resource services, on how well the program is currently meeting needs for water resources information, b) identify priority issues for decision making, c) identify gaps in information needed for decision making, and d) obtain stakeholder perceptions of the importance and economic value of needed information.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? Statistics? What suggestions did you get about improving the survey?

NWS consulted with Eastern Research Group, Inc. (ERG) on the development of the survey instrument. ERG has significant experience assessing technical assistance provided by Federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer satisfaction with services. The survey development process was informed by reviewing relevant background materials on the availability of water resources information in the U.S., documentation of previous stakeholder engagements on this topic, and previous surveys of user needs, preferences, and satisfaction. The development of the survey questions was also informed by a stakeholder meeting in the Delaware River Basin.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

As a key initial step in the stakeholder engagement process, NWS, with ERG’S support, is compiling a list of 100 stakeholders in each of the four river basins (the Potomac, Delaware, Susquehanna, and Hudson River basins). The river basin commissions will be the primary source of the stakeholder lists. The river basin commissions are federal, interstate agencies responsible for coordinating water resources management for the watershed across political boundaries. Each stakeholder will be categorized into one of more of the 11 sectors identified by the IWRSS partner agencies: (1) water quality, (2) fish and wildlife, (3) emergency management, (4) reservoir management, (5) watershed management, (6) agriculture, (7) hydropower, (8) other energy extraction (e.g., hydraulic fracking), (9) river commerce, (10) municipal and industrial water supply, and (11) recreation.

No statistical methods are being used in the survey; all individuals in the stakeholder listing will receive a survey. NWS expects that 80% to 90% of the recipients will respond to the survey because these tend to be individuals who rely on water resources information to inform decision making as a part of their profession. These decisions include routine decisions that can have significant financial value and event-driven (e.g., flood-related) decisions which are critically dependant on accurate and timely water resources information.

NWS will perform the survey using a Web-based survey instrument. NWS has chosen a Web-based instrument for two reasons. First, the water resources information we are asking about are primarily provided to the customers via the internet in the form of GIS, formatted data sets, graphs, and other visualization tools. For this reason, the survey will also be made available to interested members of the public via a Web link posted on the river basin commissions’ Web sites. Second, the program has significant interaction with stakeholders over the internet (e.g., via email, on Web forums, previous surveys, etc.) and so a Web-based survey represents a logical mode to implement the survey.

NWS expects that a high response rate is achievable; however, NWS and ERG will continue to follow good survey practices to ensure high participation, including the following:

• ERG will send the potential respondents a pre-notification email to inform them of the upcoming survey.

• ERG will send the email with the survey link 3-4 days after the pre-notification email.

• ERG will send 2 reminders to non-responders one and two weeks following the email with the survey link.

Additionally, a Web link to the survey will be placed on the Web site of each river basin commission to encourage participation from other interested customers, such as private citizens, who do not appear on the list of stakeholders for the basin.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

NWS is not using any statistical methods to select from the population and will select all population members in the survey.

NWS will use the data collected under this effort to prioritize water resources information needs in the four river basins based on the importance of the information to decision making and economic value of providing the information. Additionally, the prioritized information needs will be used to inform the design of a demonstration project plan for the river basins.

NWS will have ERG summarize the results on information needs within and across river basins to look for trends and similarities between river basins and sectors. Assuming enough variation in the data exists, NWS will have ERG perform a set of multivariate analyses within each river basin to link respondents’ answers on decision making priorities to information needs to economic value of the information.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes at least 400 individuals who are listed as stakeholders by each of the river basin commissions (i.e., approximately 100 individuals per river basin), plus a number of interested individuals who regularly access the Web sites of the river basin commissions. Each river basin commission has compiled (or is currently compiling) a list of stakeholders. The survey will not sample from the list of stakeholders, rather the entire list of will receive a survey. NWS expects that between 80 to 90 percent of the stakeholders contacted will respond to the survey, since these tend to be individuals that have a strong interest in information about their river basin and rely on NWS and the other IWRSS partner agencies for water resources information. The survey will also be open to interested individuals visiting the river basin commissions’ Web sites.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

An email invitation to participate in the survey with a Web link to the survey instrument will be sent to all individuals in the list of river basin commission stakeholders in the four river basins, and a Web link to the survey will be placed on the Web site of each river basin commission with a brief announcement explaining its purpose. No sampling will be performed in selecting the sample, so NWS has not developed a statistical methodology for stratification and sample selection. Additionally, no estimation procedure or degree of accuracy is needed since no sampling procedures are being employed.

This will be a one-time data collection. There is no need for periodic collections.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

NWS expects that a high response rate is achievable; however, NWS and ERG will continue to follow good survey practices, including the following:

• ERG will send the potential respondents a pre-notification email to inform them of the upcoming survey.

• ERG will send the email with the survey link 3-4 days after the pre-notification email.

• ERG will send 2 reminders to non-responders one and two weeks following the email with the survey link.

As discussed above, ERG, on NWS’s behalf, will administer the survey to the full population of river basin commission stakeholders; no statistical methods will be used.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

NWS contracted with ERG to develop the survey instrument. ERG has developed a number of surveys related to assessing customer satisfaction with assistance. Additionally, as part of ERG’s work, ERG is coordinating stakeholder meetings in each of the four river basins (one of which was completed during the survey drafting process). In the process of coordinating those meetings ERG also held meetings with each river basin commission to discuss information priorities in the river basin.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

NWS has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the survey instrument and implement the survey. ERG’s project manager for this work is Arleen O’Donnell (781-674-7220; [email protected]); ERG’s task lead for survey development is Dr. Lou Nadeau (781-674-7316; [email protected]).

Stakeholder Engagement to Demonstrate IWRSS for RBC in the Mid-Atlantic 4