Supplemental and Part B questions

Supplemental Questions for DOC/NOAA Customer Survey Clearance
(OMB Control Number 0648-0342)
National Ocean Service/Coastal Services Center - Training Evaluation
1. Explain who will be conducting this survey. What program office will be conducting the
survey? What services does this program provide? Who are the customers? How are these
services provided to the customer?
This information collection will be conducted by staff at the National Oceanic and Atmospheric
Administration (NOAA) Coastal Services Center (Center) to assess whether past participants of
Center trainings have been applying the knowledge and skills gained through these trainings.
Specifically, Sacheen Tavares-Leighton (Program Training Specialist) and Chris Ellis (Social
Scientist) will guide this information collection.
The Center serves the needs of coastal and natural resource management programs and
professionals (e.g., state natural resource management agencies and staff, conservation
organization staff) through development and delivery of data and information products,
decision-support tools, professional development training, and technical assistance on a variety
of topics. Data and information products and decision support tools are delivered per customer
requests via online systems (e.g., clearing house, direct download). Professional development
training is offered in three distinct areas: geospatial technology, coastal issues, and process
skills. Delivery methods include face-to-face instructor-led training held at the Center and local
host sites, self-guided web-based training, and instructor-led web-based training. Technical
assistance is conducted via telephone, electronically, or remotely (on-site) depending on the
needs of the specific customer and the specific technical assistance topic.
2. Explain how this survey was developed. With whom did you consult during the
development of this survey on content? Statistics? What suggestions did you get about
improving the survey?
This survey instrument was developed by reviewing the list of cleared questions for the Generic
Clearance, OMB Control No. 0648-0342, and through consultation from Center staff with
expertise in the areas of survey research design, and training evaluation. A list of draft
questions was discussed with staff members responsible for/involved in training and
evaluation. Suggestions included keeping the list of questions as short as possible, re-wording
objectives, inquiry on tangible actions, and changing the scale from a 5-point scale to a 4-point
scale in order to be consistent with current, standardized, Center-wide performance measures.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer
than all customers will be surveyed)? What percentage of customers asked to take the
survey will respond? What actions are planned to increase the response rate? (Web-based
surveys are not an acceptable method of sampling a broad population. Web-based surveys
must be limited to services provided by Web.)
The survey will be created using the SurveyMonkey software program and administered via an
emailed link, which will direct past participants (those who have received Center training within
the past 6-12 months) to the SurveyMonkey website. Because this is a call for feedback on
improving these courses, all of these participants should have the opportunity to respond. The
number of responses and content will be recorded. It will be possible to calculate an actual
response rate for reporting purposes. The number of questions will be limited to eight in an
effort to achieve a higher response rate relative to more lengthy surveys, and based on the
program’s informational needs. The total estimated sample size is 81. Based on previous
interactions with trainees, and numerous repeat customers, the expected response rate is
expected to be high (approximately 75%).
4. Describe how the results of this survey will be analyzed and used. If the customer
population is sampled, what statistical techniques will be used to generalize the results to
the entire customer population? Is this survey intended to measure a GPRA performance
measure? (If so, please include an excerpt from the appropriate document.)
Analysis of survey data will be undertaken through basic descriptive statistics only (e.g.,
percent, mean scores). This information collection seeks to assess whether past participants of
Center trainings have been applying the knowledge and skills gained through these trainings,
and will be used to inform improvements to Center training. The respondent universe will
include all past participants (within 6-12 months) of these courses. Data from this information
collection will be aggregated with other data for GPRA reporting:
GPRA measure #1: Percentage of tools, technologies, and information services that are used by NOAA
partners/customers to improve ecosystem-based management
GPRA measure #2: Percentage of US coastal states and territories demonstrating 20% or more annual
improvement in resilience capacity to weather and climate hazards (%/yr)

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of entities
(e.g. establishments, State and local governmental units, households, or persons) in the
universe and the corresponding sample are to be provided in tabular form. The tabulation
must also include expected response rates for the collection as a whole. If the collection has
been conducted before, provide the actual response rate achieved.
Training participants for the courses being evaluated are land use planners, floodplain
managers, and Geographic Information System (GIS) technical staff. This includes state Coastal
Zone Management programs, state and local governments, local land trust organizations, other
non-profits and federal agencies. Following is the breakdown of trainees of interest.
Population
Federal government
Local government
State Coastal Zone Management Program
National Estuarine Research Reserve
National Estuary Program
Non-profit/NGO
Private Sector
Academia
Tribal
Other

% of Training Participants
(total 100%)
2
34
23
2
2
9
6
2
1
19

This survey will be distributed via email, and will inquire on application of skills and knowledge
gained through these trainings. The estimated time necessary for each respondent to complete
the questionnaire is 20 minutes, based on trials with a small (less than ten) pilot sample. Total
estimated public burden associated with this information collection is 27 hours (81 X 20
minutes). The computer program will keep track of the total number of complete responses.
2. Describe the procedures for the collection, including: the statistical methodology for
stratification and sample selection; the estimation procedure; the degree of accuracy needed
for the purpose described in the justification; any unusual problems requiring specialized
sampling procedures; and any use of periodic (less frequent than annual) data collection
cycles to reduce burden.
The survey will be created using SurveyMonkey and administered via an emailed link, which will
direct respondents to the SurveyMonkey website.

Completed surveys received via Survey Monkey will be downloaded to a password-protected
work space at the Coastal Services Center, accessible only by staff particular to this project.
Respondents will be instructed not to provide identifying information on the survey (names,
social security numbers, dates of birth, etc.), and any identifying information placed on surveys
will be removed.
3. Describe the methods used to maximize response rates and to deal with nonresponse. The
accuracy and reliability of the information collected must be shown to be adequate for the
intended uses. For collections based on sampling, a special justification must be provided if
they will not yield "reliable" data that can be generalized to the universe studied.
The intent of this information collection is to assess customer feedback on the utility of Center
training courses in order to inform improvements to the Center courses and associated
products and services. In order to improve response rates for this information collection, the
survey has been made as brief as possible (only eight questions). Nonresponse testing will be a
challenge in that no identifying information will be collected that will allow for follow-up
activities. However, the demographic information collected on organization would give some
indication of which organizations were less likely to respond, based on the known percentages
of most groups. The intended approach will yield a representative sample of the respondent
universe, based on a small respondent universe whose members have had high levels of past
engagement with the Center and its customers – and the information gained will be extremely
valuable in making course improvements. This survey allows equal and independent
opportunity for all users to provide feedback on Center training resources.
4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as
effective means to refine collections, but if ten or more test respondents are involved OMB
must give prior approval.
Draft versions of this survey were circulated for review and comment to nine former training
participants. Reviewers were asked to offer feedback on the length, appropriateness and clarity
of questions, content, or other aspects to improve the questionnaire. Comments from
reviewers were helpful and resulted in design, and content changes to clarify questions and
simplify instructions.
5. Provide the name and telephone number of individuals consulted on the statistical aspects
of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s)
who will actually collect and/or analyze the information for the agency.
The implementation of the information collection and data analysis will be completed by Dr.
Chris Ellis at the NOAA Coastal Services Center, available by telephone at (843) 740-1195 or by
email at [email protected].