Supplementary and Part B questions

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

The Gulf of Mexico (GoMX) Climate Community of Practice (CoP) was initiated in 2009 by the NOAA Gulf of Mexico Regional Collaboration Team with funding from the Mississippi-Alabama Sea Grant Consortium,¹ to act as a conduit for addressing climate adaptation strategies in the region. The climate CoP supports a broad group of participants across the region, including extension,² outreach and education professionals, scientists, federal and state agencies, non-governmental organizations, industry and communities. To date, the CoP has held three annual workshops (with a fourth planned for April 2013) and quarterly webinars, and developed both a website featuring climate adaptation information resources (http://masgc.org/climate/cop/index.html) and a professional networking site to connect participants (http://stormsmart.org/).

The NOAA Coastal Services Center (CSC) is seeking approval to collect information on how well the Climate CoP has made progress towards its objectives, enabled new networks among participants, provided benefit and satisfaction to participants, and to identify strengths and opportunities for improvement to meet the needs of participants. The assessment will cover the workshops, webinars, and online resources provided by the CoP to participants from its inception in 2009 to present.

The CSC is seeking approval on a survey of GoMX Climate CoP participants; the survey of 409 participants will collect information on the use, usefulness of, and satisfaction with the CoP activities and resources, as well as overall satisfaction with the program. The survey will also collect information on a number of objectives specified in the CoP logic model related to community membership, support of networking, and effectiveness of information sharing.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

CSC consulted with Eastern Research Group, Inc. (ERG) on the development of the survey instrument. ERG has significant experience assessing technical assistance provided by Federal agencies through detailed interviews, focus groups, and surveys that focus on customer satisfaction, progress toward objectives, and outcome attainment.

CSC contracted with ERG to develop the survey instrument; the development process was informed by reviewing relevant CoP documents, workshop and webinar agendas and evaluations, CoP websites, and 12 interviews with key CoP participants. The interviews with key participants were performed by ERG to better understand the CoP, how it works, and how it has evolved since the first workshop. The interview topics included:

• The extent to which the CoP workshops, webinars, and online resources are useful to participants.

• How the CoP workshops, webinars, and online resources benefit participants.

• The extent to which the CoP is meeting its objectives and how useful the CoP activities are for this.

• Opportunities for improvement.

Based on these interviews, ERG determined that the survey instrument should: 1) collect information about each activity or resource the participant has used, their perceptions of the CoP’s progress on the objectives; 2) provide multiple opportunities for the respondent to provide feedback, suggestions, and new ideas through open-ended questions; 3) additionally, the survey will collect information about the respondent’s occupation and sector in order to help differentiate between the unique perspectives of the diverse groups involved in the CoP.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

CSC and MS-AL Sea Grant have maintained a list of participants in CoP workshops and webinars and their contact information (i.e., name, title, email address) and will perform the survey using a web-based survey instrument. CSC has chosen a web-based instrument for two reasons: 1) the quarterly webinars and online resources provided by the CoP are web-based, and 2) the CoP has significant interaction with and between participants over the internet (e.g., via email updates, networking web site, webinars) and so a web-based survey represents a logical mode to implement this survey.

No statistical methods are being used in the survey; all CoP participants in the participant listing will receive a survey. CSC expects that between 80 to 90 percent of the recipients will respond to the survey since these tend to be individuals that CSC has worked with through the CoP since 2009, as well as related programs in the Gulf of Mexico region (e.g., the NOAA Coastal Storms Program) since 2007.

Despite the expectation that a high response rate is achievable, CSC and ERG will follow good survey practices, including the following:

• CSC will send the potential respondents a pre-notification email to inform them of the upcoming survey.

• CSC will send the email with the survey link 3-4 days after the pre-notification email.

• CSC will send 2 reminders to non-responders one and two weeks following the email with the survey link.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

CSC will use the data collected under this data collection to identify opportunities for improvement of the activities and resources provided by the CoP and determine progress towards the CoP logic model objectives. Additionally, the survey results will be presented to the CoP participants directly in order to encourage discussion of the results and develop ideas for future improvements.

The CSC will have ERG develop simple tabulations and cross tabulations of the survey results in order to summarize key trends and, where possible, link respondents’ answers on progress toward objectives and satisfaction to use and awareness of CoP activities and resources. CSC is not using any statistical methods to select from the population and will select all population members in the survey. This data does not directly contribute to a GPRA measure.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes 409 individuals who have participated in CoP webinars, workshops or online networking. The survey will not sample the population; the entire list of participants will receive a survey. CSC expects that between 80 to 90 percent of the recipients will respond to the survey since these tend to be individuals that CSC has worked with through the CoP since 2009, as well as related programs in the Gulf of Mexico region (e.g., the NOAA Coastal Storms Program) since 2007.

Group	Population	Number to Be Selected for Survey	Expected Response Rate	Expected Surveys Completed
CoP Webinar, Workshops, and Online Networking Participants	409	409	80-90%	348

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

An email invitation to participate in the survey with a web link to the survey instrument will be sent to all individuals in the list of CoP participants No sampling will be performed in selecting the sample, so NOAA has not developed a statistical methodology for stratification and sample selection. Additionally, no estimation procedure or degree of accuracy is needed since no sampling procedures are being employed.

This will be a one-time data collection. There is no need for periodic collections.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Despite the expectation that a high response rate is achievable, CSC and ERG will follow good survey practices, including the following:

• CSC will send the potential respondents a pre-notification email to inform them of the upcoming survey.

• CSC will send the email with the survey link 3-4 days after the pre-notification email.

• CSC will send 2 reminders to non-responders one and two weeks following the email with the survey link.

As discussed above, CSC will administer the survey to the full population of CoP participants; no statistical methods will be used.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

CSC contracted with ERG to develop the survey instrument. ERG has developed a number of surveys related to assessing customer satisfaction with assistance. Additionally, as part of ERG’s work, ERG performed 12 interviews with CoP participants (nine non-federal, three federal); those interviews were used to develop and refine the survey instrument being used in this survey.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

CSC has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the survey instrument and implement the survey. ERG’s project manager for this work is Dr. Lou Nadeau (781-674-7316; [email protected]), ERG’s task lead for this work is Lauren Jankovic (781-674-7261; [email protected]).

1 One of 32 university-based programs administered through the NOAA National Sea Grant Office.

2 Extension agents work with outreach coordinators and educators to link university resources and expertise with local communities and users.

Formative Evaluation of GoMX Climate CoP: Participant Survey 4