CRCP Survey Ecosystem Research and Monitoring Information supp and Part B

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s Coral Reef Conservation Program (CRCP) will be conducting the survey. The CRCP is a partnership between the NOAA line offices engaged in work on coral reef issues in order to support “effective management and sound science to preserve, sustain, and restore valuable coral reef ecosystems for future generations.”¹ CRCP helps fulfill NOAA’s responsibilities under the Coral Reef Conservation Act (CRCA) and Presidential Executive Order 13089 on Coral Reef Protection. The CRCP works toward this mission by providing resources to coral reef managers, scientists, and NOAA partners in the U.S. states, territories, and remote island areas.

CRCP services focus on data and information products pertaining to coral reefs, including: mapping data and products; in-water physical or chemical environmental data; remote sensing derived data, forecasts and predictions; transport and connectivity data and modelling; water quality data; information and organismal responses to stressors; fishery species life history and ecology data; fish/fishery species population demographics data; socioeconomic and/or fisheries dependent information; coral physiology or genomics; coral demographic data; and, information for assessment of recovery/restoration actions.

CRCP customers include individuals involved in the management of coral reefs with an emphasis on individuals in U.S. states, territories, and remote island areas. This can include individuals who are involved in management-related decisions-making as well as scientists and other stakeholders who have an interest in protecting coral reefs.

For the most part, CRCP provides products and other data to its customers through online sources such as the Coral Reef Information System web site (http://www.coris.noaa.gov/) and the main CRCP web site (http://coralreef.noaa.gov/).

CRCP is currently undergoing a science evaluation to focus on: (1) how well the information generated by CRCP research and monitoring supports CRCP Goals and Objectives 2010-2015; (2) the relevance and quality of the information for coral reef conservation and protection; (3) the effectiveness and efficiency of translating and disseminating the information so that it is useful and used for management and conservation; and (4) the need for any adjustments to science priorities to address gaps and/or emergent management needs. This survey effort will contribute to the larger CRCP science evaluation by collecting information on: a) use of CRCP data and information, b) the extent to which the information meets customer needs, c) the accessibility of the information, d) satisfaction with the documentation and format of the information, e) information gaps, and f) overall perceptions of satisfaction with the information provided by CRCP. The information will be used to improve the program and develop new information and data products to meet customer needs.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

CRCP consulted with Eastern Research Group, Inc. (ERG) on the development of the survey instrument. ERG has significant experience assessing customer satisfaction with information and data products provided by federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer perceptions of the products. The survey development process was informed by discussions between CRCP and ERG on the information and data provided by the program and the needs of the program evaluation effort, and review of other surveys of customer needs, preferences, and satisfaction.

Initially CRCP was going to focus the survey on individual data and information products, however, the sheer number of individual products (>100) would create excessive respondent burden and would not result in meaningful results for evaluation. ERG and CRCP developed an approach to ask respondents about categories of information, thereby reducing respondent burden, while still providing opportunities for respondents to provide product specific feedback.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

CRCP will perform the survey using a web-based instrument. CRCP has chosen a web-based instrument for two reasons. First, the coral reef data and information we are asking about is primarily provided to the customers via the internet. Second, the program has significant interaction with customers over the internet (e.g., via email, on web forums, etc.) and so a web-based survey represents a logical mode for implementing the survey. Additionally, customers are widely dispersed geographically, making other survey modes (e.g., phone, mail) less efficient.

No statistical methods are being used in the survey; all individuals in CRCP’s list of customers (i.e., coral reef managers that use CRCP products and information) will receive a survey. CRCP and ERG expect a high response rate to this survey since the potential respondents are individuals who rely on coral reef data and information to inform decision making as part of their professions and are all familiar with CRCP data and products. Nevertheless, CRCP and ERG will continue to follow good survey practices to ensure high participation, including the following:

• CRCP will “advertise” the survey well in advance to ensure the potential respondents are aware a survey is coming.

• A CRCP senior manager will send the potential respondents a pre-notification email to inform them of the upcoming survey and to stress the importance of providing a response.

• ERG will send the email with the survey link 3-4 days after the pre-notification email.

• ERG will send 1 reminder to non-responders one week following the email with the survey link.

• ERG will perform follow-up phone calls with non-responders to persuade them to take the survey. The phone calls will be performed by staff with some degree of recognition in the field of coral reef management. The respondent will have the option of performing the survey online, or completing it over the phone at the time of the call.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

CRCP is not using any statistical methods to select from the population and will select all population members in the survey. The data collected under this effort will be used as an input into CRCP larger program evaluation effort, to identify data and information improvements, and to plan for future data and information products to meet customer needs. CRCP will have ERG summarize the results based on information category, jurisdiction, occupation, and customer role or discipline to identify information needs.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes approximately 150 individuals who access information from the CRCP, see Table 1. CRCP is currently compiling a list of regular contacts to use for this purpose. The survey will not sample from the list of customers; the entire list will receive a survey.

Table 1. Summary of potential respondent universe, response rate, and expected number of respondents

Description	Potential Universe	Response Rate	Actual Respondents
Coral Reef Managers	150	75%	113

CRCP expects that 75 percent of the recipients will respond to the survey since these tend to be individuals that have a strong interest in the information about coral reefs and rely on CRCP for this information. Additionally, CRCP will perform follow-up phone calls to non-responders to urge them to complete the survey.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Following a prenotification email, an email invitation to participate in the survey with a web link to the survey instrument will be sent to all individuals in the list of customers. No sampling will be performed so CRCP has not developed a statistical methodology for stratification and sample selection. Additionally, no estimate procedure or degree of accuracy is needed, as no sampling procedures are being employed.

This will be a one-time data collection, there is no need for periodic collections.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

CRCP expects that a high response rate is achievable; however, CRCP and ERG will continue to follow good survey practices to ensure high participation, including the following:

• CRCP will “advertise” the survey well in advance to ensure the potential respondents are aware a survey is coming.

• A CRCP senior manager will send the potential respondents a pre-notification email to inform them of the upcoming survey and to stress the importance of providing a response.

• ERG will send the email with the survey link 3-4 days after the pre-notification email.

• ERG will send 1 reminder to non-responders one week following the email with the survey link.

• ERG will perform follow-up phone calls with non-responders to persuade them to take the survey. The phone calls will be performed by staff with some degree of recognition in the field of coral reef management. The respondent will have the option of performing the survey online, or completing it over the phone at the time of the call.

As discussed above, ERG, on CRCP’s behalf, will administer the survey to the full population of customers; no statistical methods will be used.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

NWS contracted with ERG to develop the survey instrument. ERG has developed a number of surveys related to assessing customer satisfaction with data and information products. ERG developed questions for this survey based on prior NOAA data collections that were successful.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

CRCP has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the survey instrument and implement the survey. CRCP’s lead for this project is Marie Bundy (240-533-0820; [email protected]). ERG’s project manager for this work is Lou Nadeau (781-674-7316; [email protected]).

1 Http://coralreef.noaa.gov/aboutcrcp/whoweare/

NOAA Coral Reef Conservation Program Customer Survey 4