CZMA External Reviews ss Part A. rev

SUPPORTING STATEMENT
COASTAL ZONE MANAGEMENT ACT EXTERNAL EVALUATION
OMB CONTROL NUMBER 0648-xxxx
A.

JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.
This request is for a new information collection.
As part of its continuing efforts to monitor and improve performance of programs authorized by
the Coastal Zone Management Act of 1972, as amended, NOAA has contracted for an
independent external review of CZMA programs. This review was commissioned to provide a
time-series comparison of how NOAA has addressed recommendations from previous program
reviews and in preparation for a re-assessment of CZMA Programs through the Office of
Management and Budget’s Program Assessment and Rating Tool (PART). OMB assessed the
CZMA Programs in 2003. The PART requires that each program have an independent external
review conducted at least every five years to monitor and assess the program’s performance.
In conducting this review, the contractor needs to interview with representatives from state, local
and non-governmental organizations that work for or with state Coastal Zone Management
Programs or National Estuarine Research Reserves (NERRS).
2. Explain how, by whom, how frequently, and for what purpose the information will be
used. If the information collected will be disseminated to the public or used to support
information that will be disseminated to the public, then explain how the collection
complies with all applicable Information Quality Guidelines.
Based on the discussion in Section I above and the data available, the contractor, SRA, is
proposing an evaluation that focuses on collecting and assessing complementary information to:
1) determine how CZ programs are perceived by parties external to their programs; and 2) solicit
information about models and approaches that are “working well” with the intent of sharing
these success stories broadly. Considering previous foci on assessments of state CZ managers,
this evaluation aims at a truly external audience – those with which these programs interact at a
state and regional level.
A. Overall Methodology
This approach relies on telephone interviews that assess how parties outside the State Coastal
Zone Management Programs/NERRS view the impact and value of those programs. Further, it
attempts to assess what factor(s) (e.g., approaches, foci, programs) are associated with perceived
effectiveness. Because programs are diverse in their approaches, structures and activities, we
hope this evaluation will provide meaningful information in the context of potential
reauthorization and improve the federal-state partnership. It is also the intent to share “what
works” information among the programs to improve best practices and foster information sharing
on effective and innovative approaches.

1

The data for this evaluation will be provided mainly through telephone interviews with a variety
of external parties at the local level associated with either the state coastal zone management
program or a NERRS (avg. 4 per program) associated with a given program or reserve.
Interviews with Local Collaborators for State Coastal Zone Management Programs
Since resources are limited and all the State Coastal Zone Management Programs and NERRS
cannot be studied in detail, we recommend stratifying the sample of state programs by structure
type (e.g., networked vs. policy-focused) in order to ask whether or not a particular type of
structure is correlated with perceived effectiveness. It is anticipated that we will choose 6
programs of the “network” type and 6 of the “policy type”. For each state program we intend to
interview key people in important interest groups (e.g., state government, local government, nonprofit sector) for an average of 4 interviews per program.
Total Interviews Anticipated = ~48
We will work with NOAA CZ experts to determine useful criteria for stratifying the State
Coastal Zone Management Programs, but will include factors such as:
• Independence: Program primarily imbedded in a single agency vs. stand alone.
• Funding: A few sources of funding vs. many sources of funding.
• Governance: Board vs. agency lead.
Interviews with Local Collaborators for National Estuarine Research Reserve Programs
NERRS programs are dissimilar from State Coastal Zone programs in that their program
structure type is not so clearly divided into categories. For that reason, and the fact that the
overall number is smaller, we will choose 6 NERRS with an eye toward size and geographic
distribution, ensuring both small and large reserves as well as one reserve on the East Coast,
West Coast, Gulf Coast, and Great Lakes.
Total Interviews Anticipated = ~18
Interviews with National Level Collaborators
In addition to interviewing people with continuous, direct experience with a particular State
Coastal Zone Management Program or NERRS, we are interested in collecting information from
programs that interact on a national level with these programs (e.g., other federal agencies, state
associations, local government associations, coastal experts/academicians). The purpose of these
interviews is to assess perceived effectiveness of the State Coastal Zone Management
Programs/NERRS in the aggregate including:
1. factors that contribute to effectiveness;
2. important roles that programs play; and
3. changes in perceived effectiveness/role over time.
Through our background materials review, convening process and interviews with State CZ and
NERRS personnel, we will identify 12 national level collaborators for further discussions.
Through our background material review, discussion with collaborators regarding their partners
we will identify individuals who:

2

• have previous significant experience working with CZ programs;
• are leaders in current government organizations and nonprofits with coastal zone
management focus;
• are leaders in current government organizations and nonprofits likely to interact with CZ
programs; and
• are academics who focus on coastal zone issues.
Suggestions for this group of interviews includes:
• Coastal States Organization
• National League of Cities
• National Governor’s Association
• Association of State Floodplain Managers
• EPA’s Office of Water (National Estuary Program/Wetlands Office)
• USACE Programs (Beach Nourishment, 404 Program, Floodrisk Management)
• Restore America’s Estuaries
• Christophe Toulou (academic)
• Timothy Beatley (academic)
Total Interviews Anticipated = ~12
B. Interview Questions
Introduction & Background
Past assessments of NOAA's Coastal Zone and NERRS programs found that both exhibited
deficiencies mainly in the areas of strategic planning, program results and accountability. In
many cases, these deficiencies also were tied to lack of performance measurement. These
evaluative efforts (e.g., 1997 GAO Audit, Hershman, et.al. 1998) also reported that lack of
program measures/indicators hampered their efforts to assess the impact (and therefore
effectiveness) of coastal zone state programs.
We are conducting an evaluation that focuses on collecting and assessing complementary
information to: 1) determine how CZ and NERRS programs are perceived by parties external to
their programs; and 2) solicit information about models and approaches that are “working well”
with the intent of sharing these success stories broadly. Considering previous foci on
assessments of CZ and NERRS managers, this evaluation aims at a truly external audience –
those with which these programs interact at a state and regional level. We've contacted you to
assist us in this effort.
There will be some common questions for the State Coastal Zone Management Programs and the
National Estuarine Research Reserves, but others tailored toward their unique programmatic
goals and organic legislative intent.

3

Major Study Questions
Below is an initial list of the major study questions. These are not the specific questions we will
use as interview questions; rather, the information we are trying to gather.
Part One: Questions Common to Each Group
• What are the strongest drivers/influences for decisions made related to coastal resources in
the states?
o Who are the most influential parties?
o To what degree has change been influenced by CZ/NERRS programs?
• What role do State Coastal Zone/NERRS management programs play in assisting in coastal
decisions at the state/regional level (e.g., decision-maker, data provider, data translator,
expert)?
• Do you view your State Coastal Zone/NERRS program as effective? If so, why?
o What are the factors you believe make the State Coastal Zone Management/NERRS
program effective? (e.g., organizational, institutional, financial, data provider,
coordinator/facilitator)?
• What do State Coastal Zone/NERRS programs do that is unique? What is their niche?
• If State Coastal Zone Management Programs did not exist, what would be the impact in the
coastal zone?
o What influence do they actually have on final decisions?
• What is the most helpful role that State Coastal Zone Management programs/NERRS provide
to state/regional decision-makers?
• How has the role of State Coastal Zone/NERRS programs changed over time?
• How has the influence of State Coastal Zone/NERRS programs changed over time?
Future Looking Questions
• What are the most exciting types of work being conducted at the CZ and NERRS programs?
Why are these projects/approaches particularly helpful?
• How do/should CZ programs and NERRS identify and weigh future challenges? How do
they prepare to address them?
• How much connectivity between CZ programs and NERRS exist at a state/regional level?
How has this been helpful or harmful?
• How are/should CZ programs and NERRS position themselves to understand and effect time
sensitive resource decisions?

Part Two: Assessment of the Influence of State Coastal Zone Management Programs
In addition to the above questions, the following questions will be the basis of interviews with
external stakeholders that interact with State Coastal Zone Management Programs.
• How important a role does the State Coastal Zone Management Program play in protecting or
enhancing natural resources in the coastal zone and adjacent waters? (using 1-5 Likert scale)
o Are there activities, programs, efforts are particularly effective?

4

•
•
•
•
•
•

How important a role does the State Coastal Zone Management Program play in enhancing
public awareness of estuarine areas/issues/topics? (1-5 Likert scale)
o Are there activities, programs, efforts that are particularly effective?
How important a role does the State Coastal Zone Management Program play in reducing
harmful coastal development? (1-5 Likert scale).
o Are there activities, programs, efforts that are particularly effective?
How important a role does the State Coastal Zone Management Program play in supporting
compatible economic development? (1-5 Likert scale)
o Are there activities, programs, efforts that are particularly effective?
How important a role does the State Coastal Zone Management Program play in increasing
public access for recreation? (1-5 Likert scale)
o Are there activities, programs, efforts that are particularly effective?
How important a role does the State Coastal Zone Management Program play in assisting in
the coordination of decision-making at a local, state, federal level?
o Are there activities, programs, efforts that are particularly effective?
Is there a difference in decisions in the coastal zone between states that have approved State
Coastal Zone Management Programs as those that do not? If so, how do they differ?

Part Three: Assessment of the Influence of National Estuarine Research Reserve Programs
In addition to Part One, the following questions will be the basis of interviews with external
stakeholders that interact with National Estuarine Research Reserve Programs.
• How important a role does the NERRS play in enhancing public awareness of estuarine
areas? (1-5 Likert Scale)
o Are there activities, programs, efforts that are particularly effective?
• How important a role does the NERR play in identifying and establishing priorities among
coastal management issues? (1-5 Likert Scale)
o Are there activities, programs, efforts that are particularly effective?
• Are there gaps in understanding, information, etc. that could/should be provided by the
NERR that would be particularly useful to you?
The contractor will use the information collected through the interviews to analyze, report and
make recommendations on the effectiveness of the CZMA programs and to assess status of the
deficiencies identified in the 2003 PART from OMB. NOAA will, in turn, use the contractor’s
evaluation report and recommendations to report on the CZMA’s performance/effectiveness,
particularly as evidence for an OMB PART assessment. NOAA will also use this information
for continuous improvement of the program.
NOAA will retain control over the information collected and safeguard it from improper access,
modification, and destruction, consistent with NOAA standards for confidentiality, privacy, and
electronic information. The evaluation report delivered to NOAA, which will result in part from
these interviews, will be available to the public. See response to Question 10 of this Supporting
Statement for more information on confidentiality and privacy. The information collection is
designed to yield data that meet all applicable information quality guidelines. Although the
information collected is not expected to be disseminated directly to the public, results may be
used in scientific, management, technical or general informational publications. Should NOAA’s

5

Ocean Service decide to disseminate the information, it will be subject to the quality control
measures and pre-dissemination review pursuant to Section 515 of Public Law 106-554.
3. Describe whether, and to what extent, the collection of information involves the use of
automated, electronic, mechanical, or other technological techniques or other forms of
information technology.
All interviews will be conducted via telephone.
4. Describe efforts to identify duplication.
NOAA is the only agency charged with administering the CZMA. The type of information
collected under this request is similar in nature to the Government Accountability Office’s report
(GAO-08-1045); however, the GAO focused on the coastal zone management programs, not the
research reserves. This evaluation has a much broader scope than that of the GAO study and will
include interviews not limited to coastal zone management program managers.
5. If the collection of information involves small businesses or other small entities, describe
the methods used to minimize burden.
This collection does not involve small businesses and is not likely to involve other small entities.
Most partners of the CZMA programs consist of state, local, regional or inter-state government
agencies or entities, or in some cases, non-governmental organizations, which in most cases
would not qualify as small entities.
6. Describe the consequences to the Federal program or policy activities if the collection is
not conducted or is conducted less frequently.
The agency would not be able to report periodically on its performance or effectiveness
implementing the CZMA if program partners and customers could not be interviewed or
surveyed periodically.
7. Explain any special circumstances that require the collection to be conducted in a
manner inconsistent with OMB guidelines.
Not applicable.
8. Provide information on the PRA Federal Register Notice that solicited public comments
on the information collection prior to this submission. Summarize the public comments
received in response to that notice and describe the actions taken by the agency in response
to those comments. Describe the efforts to consult with persons outside the agency to
obtain their views on the availability of data, frequency of collection, the clarity of
instructions and recordkeeping, disclosure, or reporting format (if any), and on the data
elements to be recorded, disclosed, or reported.
A Federal Register Notice published on 9 June 2009 (74 FR 27279) solicited public comment on
this collection request. None was received.

6

9. Explain any decisions to provide payments or gifts to respondents, other than
remuneration of contractors or grantees.
Not applicable.
10. Describe any assurance of confidentiality provided to respondents and the basis for
assurance in statute, regulation, or agency policy.
As stated on the interview forms, the contents of the final report from the external evaluation are
considered a matter of public record. Interview notes would be considered pre-decisional and/or
private under the Freedom of Information Act.
11. Provide additional justification for any questions of a sensitive nature, such as sexual
behavior and attitudes, religious beliefs, and other matters that are commonly considered
private.
No sensitive questions are asked.
12. Provide an estimate in hours of the burden of the collection of information.
The total one-time burden is estimated as 156 hours. The number of interviews to be conducted
will not exceed 78. Each interview is expected to last less than 2 hours.
13. Provide an estimate of the total annual cost burden to the respondents or recordkeepers resulting from the collection (excluding the value of the burden hours in Question
12 above).
There is no cost burden to the respondents because the contractor will conduct interviews with
respondents by telephone as part of the contract.
14. Provide estimates of annualized cost to the Federal government.
The cost in staff time for NOAA to coordinate, review and disseminate the report is estimated as
10 hours at $56/hour, for a total annual cost of approximately $560. The cost of the contract to
conduct interviews and complete analysis and report is $92,000.
15. Explain the reasons for any program changes or adjustments.
Not applicable. This is a new collection.
16. For collections whose results will be published, outline the plans for tabulation and
publication.
NOAA may summarize information contained within the information collection in order to
provide internal assessments of program performance or resource allocation, progress reports, or
accomplishments, or information required by the Congress or agencies of the federal government
outside of NOAA for oversight.
7

17. If seeking approval to not display the expiration date for OMB approval of the
information collection, explain the reasons why display would be inappropriate.
Not applicable.
18. Explain each exception to the certification statement.
Not applicable.

8