Final Report from the Blue Ribbon Panel Conducting an External Review of the U.S. Department of Agriculture Conservation Effects Assessment Project

C
E
A
P

FINAL REPORT
FROM THE BLUE RIBBON PANEL CONDUCTING AN
EXTERNAL REVIEW OF THE U.S. DEPARTMENT OF AGRICULTURE

CONSERVATION EFFECTS
ASSESSMENT PROJECT

A Soil and Water Conservation Society Project

FINAL REPORT
FROM THE BLUE RIBBON PANEL CONDUCTING AN
EXTERNAL REVIEW OF THE U.S. DEPARTMENT OF AGRICULTURE

CONSERVATION EFFECTS
ASSESSMENT PROJECT

A Soil and Water Conservation Society Project

About SWCS
The Soil and Water Conservation Society (SWCS) is a nonprofit scientific and educational
organization that serves as an advocate for natural resource professionals and for sciencebased conservation policy. Our mission is to foster the science and art of soil, water, and
environmental management on working land—the land used to produce food, fiber, and
other services that improve the quality of life people experience in rural and urban
communities. We work to discover, develop, implement, and constantly improve ways
to use land that sustains its productive capacity and enhances the environment at the
same time.
SWCS has about 7,000 members around the world. They include researchers, administrators, planners, policymakers, teachers, students, farmers, and ranchers. Nearly
every academic discipline and many different conservation institutions are represented
within the membership.
Member benefits include the widely respected Journal of Soil and Water Conservation,
representation in policy circles, opportunities for leadership and networking, and discounts on books and conference registrations.
SWCS chapters throughout the United States, Canada, and the Caribbean Basin conduct a variety of activities at local, state, and provincial levels and on university campuses. These 75 chapters represent the grassroots element of the organization. Each
chapter elects its own officers, organizes conservation forums, and formulates local
recommendations on conservation and environmental issues.

Soil and Water Conservation Society
945 Southwest Ankeny Road, Ankeny, Iowa 5 0 023
P: ( 515) 289- 2331 • F: ( 515)289- 1227 • www.swcs.org

2

SOIL AND WATER CONSERVATION SOCIETY

Table of Contents
EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Panel Strongly Endorses Purpose of CEAP
CEAP Must Change Direction to Achieve Purpose
CEAP Must Inform Strategic Resource Management
Blueprint
Build The Science Base

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
CEAP Fact Sheet Sidebar

CHANGE DIRECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Solving Problems versus Estimating Effects
Monitoring versus Simulation or Extrapolation
Strategic Resource Management
Opportunities to Facilitate Change in Direction
Soil and Water Resource Conservation Act of 1977 Sidebar

STRATEGIC RESOURCE MANAGEMENT BLUEPRINT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Components of a Strategic Resource Management System
Components Included in Current CEAP Plans
Components of a Strategic Resource Management System Table
Components Missing from USDA Plans

BUILD THE SCIENCE BASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Scientific Priorities
Capacity to Deliver Science

CAUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Transparency
Program Neutral
CEAP: FINAL REPORT TO USDA

3

EXECUTIVE SUMMARY
he U.S. Department of Agriculture (USDA) has
asked the Soil and Water Conservation Society
(SWCS) to help design and implement the
Conservation Effects Assessment Project (CEAP). One of
SWCS’s roles is to facilitate an external, policy-level
review of CEAP. The external review has two primary
purposes: (1) to seek, analyze, and synthesize input from
future users of information generated by CEAP as a
means of helping USDA understand how best to design
CEAP and package its outputs, and (2) to recommend
new approaches or refinements of planned approaches
that will enhance the capacity to produce comprehensive national assessments in 2006 and beyond.The project is intended to help make CEAP more useful,
responsive, and credible and help assure that CEAP’s
products will have wide utility for policymakers, program managers, and the conservation community.
SWCS has assembled a blue-ribbon panel of academics and leaders of nongovernmental organizations, state
agencies, and tribes.The panel was constructed to represent the communities who will use, interpret, and shape
opinion regarding the meaning and value of outputs
from CEAP. The panel is conducting its work through
meetings, teleconferences, and outreach to user groups.
The panel is focusing it’s work in three areas: (1) understanding the expectations and needs of eventual users of
CEAP outputs, (2) scrutinizing detailed plans for CEAP
and interacting with those USDA staff members responsible for the national assessment and watershed research
projects, and (3) reviewing preliminary outputs generated by CEAP in 2005. Federal agencies involved in the
design and implementation of CEAP have assigned a staff
liaison(s) to the panel to serve as an information source
and point of contact.
The blue ribbon panel met for the first time January
12-14, 2005, in Washington, D.C. to discuss the detailed
CEAP study plans USDA had provided and to interact
with federal staff through intensive briefings and discussion sessions. In March 2005, the panel issued a report
of its preliminary findings that summarized the panel’s
initial reactions to USDA’s overall plans for building and
implementing CEAP. In their preliminary findings the

T

4

SOIL AND WATER CONSERVATION SOCIETY

panel strongly and unanimously endorsed the purpose
of CEAP, but recommended an immediate change in
direction and emphasis for the project to ensure
CEAP’s purpose is achieved.
The panel met for the second time in May 2005 in
response to a request from USDA to help identify
opportunities to use CEAP to inform the 2007 farm
bill conservation title debate. This request from USDA
was the panel’s first recommendation made in its preliminary findings to change CEAP’s direction was for
USDA to redirect CEAP in the short-term to strategically inform the 2007 farm bill.The panel issued its second report in September 2005 outlining what they
considered the most promising opportunities for CEAP
to inform the 2007 farm bill conservation debate.
Copies of both reports can be found on the SWCS
website at www.swcs.org.
In this final report, the panel takes up the second
major recommendation the panel made in its preliminary
findings for changing CEAP’s direction: “USDA must
also change its stated long term goal for CEAP to ensure
the program is built primarily to look to the future—to
enable rigorous and science-based evaluation of options
to improve conservation efforts in the future.”
Panel Strongly Endorses Purpose of CEAP
CEAP’s stated purpose is to help policymakers and program managers implement existing, and design new,
conservation programs to more effectively and efficiently meet the goals of Congress and the
Administration. The panel unanimously endorses this
purpose. A coherent and science-based assessment and
evaluation system is urgently needed to ensure conservation programs cost-effectively produce the gains in environmental quality taxpayers and agricultural producers
expect. The panel commends USDA for taking on this
critical and difficult task. The panel hopes its work will
help USDA and its partners succeed in their efforts.The
panel, however, recommends important changes in direction for CEAP to ensure it achieves its purpose.

CEAP Must Change Direction to Achieve Purpose

CEAP Must Inform Strategic Resource Management

CEAP must change direction to become the coherent,
science-based assessment and evaluation system policymakers, program managers, and the conservation community urgently needs.The panel found that CEAP was
conceived largely as a way to supply annual, quantitative, and generalized estimates of the effects of conservation practices needed to support a suite of programspecific performance measures. The panel concluded
this vision of CEAP is too limited and is likely to produce misleading information—even in the long term
and after large investments of resources to quantify
annual, program-by-program performance.

CEAP should be built to answer the question “What
should we do next year?” rather than “What did we do
last year?”The panel recommends two critical roles for
CEAP: (1) CEAP should become an integral part of a
larger, collaborative, and ongoing system to inform and
adapt strategic resource management, and (2) CEAP
should define and test the science-base for adaptive
management of conservation programs.
The panel is also acutely aware that it is recommending a significant change in CEAP and recommending a
strategic resource management system be built at a time
when agency budgets are tight and likely to get tighter.
The panel identified six opportunities to facilitate the
change in direction it is recommending, and to reduce
the cost and increase the efficiency with which such as
system can be built: (1) USDA must achieve a broader
consensus on the purpose and future direction for
CEAP, (2) CEAP must expand and strengthen collaboration, (3) Congress should update and reauthorize the
Soil and Water Resources Conservation Act of 1977
(RCA), (4) the strategic resource management system
should initially focus on a few critical environmental
goals, (5) the system should look more to regional,
rather than national-level assessments, and (6) more
weight should be given to strategic components when
evaluating program performance.

Solving Problems versus Estimating Effects
The panel found little value in even the best estimate of
the environment effects of a conservation program
unless that estimate could be (1) compared to established environmental goals and (2) linked to the ecological and economic context in which the estimated
effect occurs. As one panel member put it: “What does
it mean if we are told EQIP (Environmental Quality
Incentives Program) reduced nitrate losses from farm
fields by 80 million tons? Was that enough, should we
have done more, how much more?”
Monitoring versus Simulation or Extrapolation

Blueprint
Uncertainties and error introduced by broad practice
definitions, missing quantitative links between variability in practice application(s) and environmental effects,
and the difficulty of simulating real world interactions
among conservation practices in process models, will
seriously impair the scientific credibility of simulated,
quantitative estimates of environmental effects being
produced by conservation programs. Simulations and
extrapolations cannot—and must not—substitute for
on-the-ground monitoring and inventory systems
designed to determine if anticipated conservation and
environmental benefits are being achieved.
The panel is encouraged by changes USDA has
made in its short- and long-term plans for CEAP during the course of the panel’s deliberation. The panel
commends USDA for making such changes and urges
USDA to take additional steps to ensure the investment
in CEAP produces the credible, science-based assessment system so critical to the future of USDA conservation programs.

The strategic resource management system envisioned
by the panel must be able to accomplish six tasks: (1)
construct and update the conservation baseline, (2) set
meaningful goals, (3) evaluate alternative strategies, (4)
monitor program implementation, (5) monitor environmental benefits, and (6) reevaluate strategies.
Components Included in Current CEAP Plans
The panel is encouraged that USDA’s current plans for
CEAP could produce several important components of
this strategic resource management system. The CEAP
Cropland-CRP national assessment simulation capabilities will enable large-scale estimation of the baseline
effects of the conservation effort represented by USDA
programs. It will allow for more rigorous assessment
and reevaluation of strategies for employing staff and
programs to meet goals—at least at large regional and
national scales and in the short-term for cropland only.
CEAP watershed studies will help refine and validate

CEAP: FINAL REPORT TO USDA

5

the methods used to simulate effects of conservation
practices and programs.The panel recommends greater
priority be given in CEAP watershed study plans to
building the capacity to conduct regional assessments of
the environmental benefits of conservation activities.
Performance reporting systems, already in place in
NRCS and the Farm Service Agency (FSA), will provide the nuts and bolts information about program
implementation needed to document the current level
of conservation effort supported by USDA. Efforts to
more precisely geo-reference program implementation
data are essential and should be the highest priority for
enhancing current performance reporting systems.
Components Missing from USDA Plans
The panel is concerned that several critical components
of a strategic resource management system are not
included in USDA’s current plans for CEAP.
Monitoring. The most important and troubling missing piece is the absence of plans for on-the-ground
monitoring of change in the environmental indicators
and outcomes conservation programs and activities are
intended to improve. The panel recommends that
Congress mandate that at least one percent of the funding for each authorized program—about $40 million of
the $4 billion taxpayers are investing in conservation—
be set aside to support monitoring and evaluation of
those programs.
Conservation Needs and Priorities. The panel did not see
any evidence for rigorous and comprehensive identification and assessment of the extent and magnitude of environmental and resource management problems that are
not being met through current conservation efforts.The
panel strongly recommends that clear links be forged
between CEAP staff, USDA strategic planners, and staff
responsible for the RCA process. Linked staff should be
charged with producing a coordinated plan for data collection, resource inventory, and resource assessment activities that will produce credible assessments of unmet
needs and priorities. The Natural Resources Inventory
(NRI) system, in particular, should be revisited to determine if and how the system could be revamped to produce statistically valid estimates of the extent and geographic distribution of conservation needs.
The assessment of needs and priorities must be
developed in collaboration with other federal and state
conservation agencies, nongovernmental organizations,
and the private sector. Meaningful and cost-effective
assessment can only be completed if they are based on
a collaborative effort between federal entities such as
USDA, the Environmental Protection Agency (EPA),
the National Oceanic and Atmospheric Administration

6

SOIL AND WATER CONSERVATION SOCIETY

(NOAA), the U.S. Fish and Wildlife Service (USFWS),
the U.S. Geological Survey (USGS); state conservation,
natural resource, and environmental protection agencies, nongovernmental conservation organizations, and
the private sector.
Beyond USDA Programs. Many private and public
sector entities contribute to the nation’s conservation
effort on working land. The CEAP national survey on
cropland will collect some information on such efforts,
but it is not clear to the panel what plans are in place to
regularly update and enrich information available about
non-USDA driven conservation on working land. The
panel recommends a collaborative effort among USDA
and its partners to design and implement a cost-effective system to collect and consolidate information
about the level of conservation effort supported by
other federal, state, and local units of government, nongovernmental organizations, and the private sector.
Build The Science Base
Building the science-base for strategic resource management on working land must be a primary purpose
for CEAP.The panel applauds USDA’s commitment to
using CEAP to help in building this science base. The
panel is particularly enthusiastic about the unique value
of the watershed study component of CEAP. CEAP
should support a broad, collaborative effort to create a
set of “representative” watersheds or other salient geographic units in various agricultural settings on a
regional basis.The network of watershed studies should
be constructed in a comparative fashion to answer three
broad, inter-related, and critical questions: (1) What are
the most effective interventions—practices, alternative
farming systems, landscape or hydrologic restoration—
to achieve specific environmental benefits?, (2) Where
should such interventions be placed on the landscape to
most effectively achieve specific environmental benefits?, and (3) How can we design cost-effective and costfeasible systems to monitor changes in core environmental indicators?
Panel members are acutely aware of the importance
of the technical services infrastructure—research, education, and technical assistance—as the foundation of
the nation’s conservation effort on working land. Even
the best science base will produce few results unless
technically proficient people and technical tools are
available to translate that science into on-the-ground
changes in farm, ranch, and watershed/geographic unit
management. Moreover, rapid scientific advances are
improving our ability to understand and manage the
spatial and temporal variability in agricultural land-

scapes in ways that could dramatically improve the performance of conservation programs. Taking advantage
of these opportunities, however, will require better
information and people trained to use that information
to more precisely apply conservation treatments at field,
farm, ranch, watershed, and other salient geographic
scales. An in-depth assessment of the strengths and
weaknesses of the current technical services infrastructure should be among the first priorities undertaken by
a strategic resource management system.

CEAP: FINAL REPORT TO USDA

7

INTRODUCTION
he U.S. Department of Agriculture (USDA) asked
the Soil and Water Conservation Society (SWCS)
to help design and implement the Conservation
Effects Assessment Project (CEAP). One of SWCS’s roles
was to facilitate an external, policy-level review of CEAP.
The external review had two primary purposes: (1) to
seek, analyze, and synthesize input from future users of
information generated by CEAP as a means of helping
USDA understand how best to design CEAP and package its outputs, and (2) to recommend new approaches or
refinements of planned approaches that will enhance the
capacity to produce comprehensive national assessments.
The project is intended to help make CEAP more useful,
responsive, and credible and help assure that CEAP’s
products will have wide utility for policymakers, program
managers, and the conservation community (See “CEAP
Fact Sheet” sidebar on page 9).
SWCS assembled a blue-ribbon panel of academics
and leaders of nongovernmental organizations, state
agencies, and tribes.The panel was constructed to represent the communities who will use, interpret, and shape
opinion regarding the meaning and value of outputs
from CEAP. The panel conducted its work through
meetings, teleconferences, and outreach to user groups.
The panel focused its work in three areas: (1) understanding the expectations and needs of eventual users of
CEAP outputs, (2) scrutinizing detailed plans for CEAP
and interacting with USDA staff members responsible for
CEAP national assessment and watershed research projects, and (3) exploring the relationship of CEAP to
agency reporting systems and similar assessment efforts
undertaken by other federal and state agencies. Federal
agencies involved in the design and implementation of
CEAP assigned a staff liaison(s) to the panel to serve as an
information source and point of contact.
The blue ribbon panel met for the first time January
12-14, 2005, in Washington, D.C. to discuss the detailed
CEAP study plans USDA had provided and to interact
with federal staff through intensive briefings and discussion sessions.The panel focused its deliberations on what
results CEAP should produce rather than how CEAP
should produce those results.
CEAP has two components: (1) a national-scale

T

8

SOIL AND WATER CONSERVATION SOCIETY

effort—largely based on the NRI sampling frame—to
estimate the environmental benefits produced by the
conservation practices applied by participants in USDA
conservation programs, and (2) a set of watershed
research projects to test and enhance the methods used in
the national assessment. The Natural Resources
Conservation Service (NRCS) and the Agricultural
Research Service (ARS) are leading a team of federal
agencies to design, build, and implement CEAP.
In March 2005, the panel issued a report of its preliminary findings that summarized the panel’s initial
reactions to USDA’s overall plans for building and
implementing CEAP. In their preliminary findings, the
panel strongly and unanimously endorsed the purpose
of CEAP, but recommended an immediate change in
direction and emphasis for the project to ensure
CEAP’s purpose is achieved.
The panel met for the second time in May 2005 in
response to a request from USDA to help identify
opportunities to use CEAP as a way to inform the 2007
farm bill conservation title debate. This request from
USDA was the panel’s first recommendation made in its
preliminary findings.The panel issued its second report
in September 2005 outlining what they considered the
most promising opportunities for CEAP to inform the
2007 farm bill conservation debate.
Copies of both reports can be found on the SWCS
website at www.swcs.org.
In this final report, the panel takes up the second
major recommendation the panel made in its preliminary
findings for changing CEAP’s direction: “USDA must
also change its stated long term goal for CEAP to ensure
the program is built primarily to look to the future—to
enable rigorous and science-based evaluation of options
to improve conservation efforts in the future.”

CEAP FACT SHEET

Purpose
The Conservation Effects Assessment Project (CEAP)
began in 2003 as a multi-agency effort to quantify the
environmental benefits of conservation practices used
by private landowners participating in selected U.S.
Department of Agriculture’s (USDA) conservation programs. Project findings and results will be used to
report progress on the environmental effects of these
programs, aid discussions on conservation policy
development, guide conservation program implementation, and ultimately, help farmers and ranchers
make informed conservation choices.

vation practices for national and regional reporting and
(2) Assess the potential for existing conservation programs and future alternatives to meet the Nation’s environmental and conservation goals.
Currently, there are four active components within the National Assessment:
• Cropland component
• Wildlife component
• Wetlands component
• Grazing lands (pastureland and rangeland)
component
CEAP Watershed Studies

Scope
CEAP will assess the benefits of conservation practices
associated with the following USDA conservation programs in the 2002 Farm Bill: Environmental Quality
Incentive Program (EQIP), Conservation Reserve
Program (CRP), Conservation Security Program (CSP),
Wetland Reserve Program (WRP), Wildlife Habitat
Incentives Program (WHIP), Natural Resources
Conservation Service (NRCS) Conservation Technical
Assistance Program, and Grassland Reserve Program
(GRP). Conservation practices that will be assessed
include conservation buffers; erosion control; wetlands conservation and restoration; establishment of
wildlife habitat; and management of grazing land,
tillage, irrigation water, nutrients, and pests.
CEAP will focus on developing approaches,
methodologies, and databases to produce scientifically credible estimates of environmental benefits of conservation. Initially, the project will study water quality,
soil quality and water conservation on cropland,
including enrolled CRP land. Eventually, CEAP will
assess benefits to water quality, soil quality, water
conservation and wildlife habitat on cropland, grazing
land, and wetlands. Work is underway to develop suitable and affordable analytical approaches for these
other land uses and natural resource concerns.
CEAP National Assessment
The National Assessment component of the
Conservation Effects Assessment Project (CEAP) will
provide scientifically credible estimates of the environmental benefits obtained from USDA conservation programs. The National Assessment component has two
goals: (1) Provide NRCS and the conservation community with quantitative estimates of the benefits of conser-

Small watershed case studies are being conducted
to complement the national assessment. These
studies will provide in-depth assessments of water
quality and other benefits at a finer scale than is
possible for the National Assessment. Presently
there are three CEAP watershed categories:
USDA
• ARS Benchmark Watersheds. These
Agricultural Research Service watershed studies will
provide information needed to verify the accuracy of
models used in the National Assessment. Twelve
watersheds were selected with a focus on water and
soil quality and water conservation as primary
resource concerns on rain-fed agricultural land.
• Special Emphasis Watersheds. These have been
established by NRCS to address specific resource
concerns, such as manure management from animal feeding operation and water use and conservation on irrigated cropland. Nine special emphasis
watersheds were established in 2004, with work
designated for completion in 2006 and 2007.
• Competitive Grants Watersheds. These watersheds established by the Cooperative State
Research, Education and Extension Service, will
focus on understanding on how best to schedule
and locate conservation efforts within a watershed in order to achieve locally defined water
quality goals. Four project grants were awarded
in 2004. Four additional project grants were
awarded in September 2005.
CEAP home page http://www.nrcs.usda.gov/technical/NRI/ceap.

CEAP: FINAL REPORT TO USDA

9

CHANGE DIRECTION
EAP’s stated purpose is to help policymakers
and program managers implement existing, and
design new, conservation programs to more
effectively and efficiently meet the goals of Congress and
the Administration.The panel unanimously endorsed this
purpose in its preliminary findings. The panel’s work in
the months since that report was released reaffirmed the
panel’s conviction that a coherent and science-based
assessment and evaluation system is urgently needed to
ensure conservation programs produce the gains in environmental quality, taxpayers and agricultural producers
expect as cost-effectively as possible.The panel commends
USDA for taking on this important task.
The panel’s work since its first meeting also reconfirmed the panel’s conviction that CEAP must change
direction to become the desired, science-based assessment and evaluation system. Indeed, the panel’s concern
that USDA must revisit its long-term direction for
CEAP has grown more urgent given growing pressures
on conservation policy and programs from deficit
reduction, international trade agreements, indicators of
environmental degradation, and missed opportunities to
realize the promise of increased funding and authorities
provided in the conservation title of the 2002 farm bill.

C

Solving Problems versus Estimating Effects
The long-term goal for CEAP, presented to the panel at
its first meeting and embedded in detailed CEAP study
plans, was to produce annual, national-level estimates of
the environmental effects of individual conservation programs. Such estimates were to be used to populate a
diverse set of individual program performance measures
that were to be the basis for evaluating the effectiveness
of programs. The panel, however, questions the wisdom
of relying on such performance measures given the serious scientific limitations of performance measures based
on generalized estimates of environmental effects derived
by simulating or extrapolating from data on the kind and
extent of practices conservation programs funded.
The panel found little value in even the best estimate
of the environment effect of a conservation program

10

SOIL AND WATER CONSERVATION SOCIETY

unless that estimate could be compared to established
environmental goals. Knowing what was accomplished,
in other words, is not useful unless we also know what
should or could have been accomplished. As one panel
member put it:“What does it mean if we are told EQIP
(Environmental Quality Incentives Program) reduced
nitrate losses from farm fields by 80 million tons? Was
that enough, should we have done more, how much
more?”The panel stressed that simulating or extrapolating the environmental effect of conservation practices
must not be confused with evaluating the extent to
which conservation programs are solving environmental problems. Quantifying effects and evaluating effectiveness are related, but very different.
Generalized performance measures, such as reduction
in nitrate losses from farm fields must be linked to the
locations where the reductions occur, the sensitivity of
the affected resources, and the threshold levels of such
performance measures that must be achieved to produce
measurable, on-the-ground environmental improvement.Where environmental effects are produced can be
as or even more important than the magnitude of the
effect. Small reductions in pollutants or increases in critical habitat that occur in highly sensitive areas or that
affect high value resources may produce greater benefits
than large reductions in pollutants or increases in habitat
that occur in less sensitive areas or affect less valuable
resources. Performance measures based on generalized
estimates of environmental effects of conservation programs must be presented in the ecological and environmental context in which the estimated effects occur and
must be related to quantitative goals for change in that
performance measure. Performance measures based on
estimates of the environmental effects of conservation
programs that are not linked to the relevant context and
goals may mislead more than inform analyses of program
performance and effectiveness.

Monitoring versus Simulation or Extrapolation

Strategic Resource Management

The panel also concluded that unavoidable technical
problems—even in the long term and after large investments of resources in data collection and modeling—
will seriously impair the scientific credibility of simulated or extrapolated estimates of environmental benefits
produced by conservation programs. Uncertainties and
error introduced in such estimates by broad practice
definitions, missing quantitative links between practice
application(s) and environmental effects, variability in
practice implementation, and the difficulty of simulating real world interactions among conservation practices in process models, will seriously impair the scientific credibility of simulated, quantitative estimates of
environmental benefits produced by conservation programs. Even large investments in additional data collection and modeling may produce results that are still
subject to serious question regarding their scientific
credibility. Using such estimates to evaluate than effectiveness of programs and the strategies used to employ
those programs, may mislead rather the inform work to
enhance the nation’s conservation effort.
The scientific problems and potential for error are
much larger in efforts to estimate environmental effects
through simple extrapolations from information about
the practices programs have funded producers to adopt.
The panel was briefed about—and troubled by—plans
to undertake such extrapolations using generalized sets
of conservation practices, generalized geographic units,
and standardized estimations of the environmental
effects of those practice sets. Such extrapolations suffer
from the same uncertainties and errors outlined in the
previous paragraph that plague simulation efforts. In
addition, such extrapolations must assume that effects
can be linearly aggregated to larger scales, that is, that
environmental benefits increase in direct proportion to
the amount of acres treated and/or number of practices
employed. Although intuitively attractive, the assumption that the effects of practices can be “added up” is
refuted by scientific studies of practice effects at field
and larger geographic scales. Estimates of environmental effects derived from such extrapolations from practice information must be treated with caution. The
value of such estimates for informing efforts to enhance
the efficiency and effectiveness of conservation programs is questionable.
The panel concluded that simulations and extrapolations cannot—and must not—substitute for on-theground monitoring and inventory systems designed to
determine if anticipated conservation and environmental benefits are being achieved.The absence of plans for
such monitoring is the most troubling gap in the long
term blueprint for CEAP discussed in the next section.

As stated earlier, the panel found that CEAP was conceived largely as a way to supply annual, quantitative,
and generalized estimates of the effects of conservation
practices needed to support a suite of program-specific
performance measures.The panel concluded this vision
of CEAP is too limited and is likely to produce misleading information—even in the long term and after
large investments of resources to quantify annual, program-by-program performance.
CEAP should, be built to answer the question “What
should we do next year?” rather than “What did we do
last year?” The planned National Resources Inventory
(NRI) simulation capabilities and watershed studies are
much better suited to, and more properly used to assess
and evaluate alternative strategies than to attempt to
quantify the effect of past practice implementation.
Sensitivity analyses can, and must be used to transparently address the technical uncertainties and errors involved
in such forward looking simulations. Substantial investments in data collection and modeling capabilities are
anticipated to build CEAP. Making specific recommendation for how and where to make those investments is
beyond the panel’s scope and expertise. In some cases, the
data collection and modeling capabilities needed to assess
future options can also be used to estimate past performance. When such “win-win” situations are not apparent,
the panel urges USDA to give first priority to investments
that will build a system capable of producing rigorous
assessments of options for implementing conservation
programs in the future.
The panel recommends the primary purpose of CEAP
should be informing and enabling a strategic approach to
program evaluation and resource management.The panel
recommends two critical roles for CEAP.
1. CEAP should become an integral part of a larger,
collaborative, and ongoing system to inform and
adapt strategic resource management.
2. CEAP should define and test the science-base for
adaptive management of conservation programs.
The larger strategic assessment system must (1)
determine needs based on a core set of indicators of
resource and environmental conditions, (2) set goals
based on the severity of problems and the magnitude of
opportunities tempered by fiscal or technical constraints, (3) develop strategies for deploying people and
resources to most effectively achieve goals, (4) evaluate
progress, and (5) adjust goals and strategies based on
ongoing evaluation of the extent to which problems are
being solved and opportunities exploited. The system
should enable and inform adaptive management to help
policymakers and program managers implement exist-

CEAP: FINAL REPORT TO USDA

11

ing conservation programs and design new conservation programs to more effectively and efficiently meet
the goals of Congress and the Administration.
Opportunities to Facilitate Change in Direction
The panel understands the change in direction it is recommending entails a significant departure from the priorities and values embedded in the original plans for
CEAP that were communicated to the panel through
study plans and briefings.The panel is also acutely aware
that it is recommending a strategic resource management system be built at a time when agency budgets are
tight and likely to get tighter.
The panel identified six opportunities to facilitate
the change in direction it is recommending and to
reduce the cost and increase the efficiency with which
such a system can be built:
1. USDA must achieve a broader consensus on the
purpose and future direction for CEAP.
2. CEAP must expand and strengthen collaboration.
3. Congress should update and reauthorize the Soil
and Water Resources Conservation Act of 1977
(RCA).
4. Focus the system on a few critical and explicitly
stated national and regional environmental goals.
5. Look more to regional-level versus national-level
assessments.
6. OMB should give more weight to strategic components when evaluating program performance.
Aggressively pursuing these opportunities, would,
the panel believes, increase the likelihood that the
strategic resource management system they recommend
could be built even in the face of tight budgets.
Consensus on CEAP Future Direction
Since the panel began its work, USDA has made important changes in its plans for CEAP. In particular, the
panel applauds USDA’s commitment to redirecting
CEAP resources—in the short-term—to informing the
debate over the conservation provisions of the 2007
farm bill. CEAP is a multi-agency effort. Lead agencies
include USDA’s Natural Resources Conservation
Service (NRCS),Agricultural Research Service (ARS),
Cooperative State Research, Education, and Extension
Service (CSREES), Farm Service Agency (FSA), and
National Agricultural Statistics Service (NASS).
Collaborating agencies include USDA’s Economic
Research Service (ERS), Forest Service (FS), Office of
Risk Assessment and Cost-Benefit Analysis (ORACBA),
the U.S. Army Corps of Engineers (ACE), the U.S.
Environmental Protection Agency (EPA), the U.S. Fish

12

SOIL AND WATER CONSERVATION SOCIETY

and Wildlife Service (FWS), and the U.S. Geological
Survey (USGS). The panel’s briefings from, and interaction with USDA and other federal staff, however, suggests
that staff—including top agency leadership—have different visions of the ultimate role CEAP should or could
play in their agencies assessment, monitoring, evaluation,
and resource management systems.The panel believes the
opportunity presented by CEAP to enhance the credibility and meaning of conservation information provided to
policymakers, program managers, and the conservation
community is large. Disparate visions of the ultimate role
for CEAP will reduce the likelihood that this opportunity is realized.The panel urges USDA to work in collaboration with its partners to produce a common vision for
where CEAP is headed and the investments needed by
USDA and its partners to make that vision a reality.The
panel sincerely hopes that its work, and this report, will
help in that effort.
Collaboration
All of the panel’s recommendations will be easier to
implement if USDA aggressively pursues collaboration
with other agencies and organizations at federal,
regional, state, and local levels. In many cases, collaboration is the only way the panel’s recommendations can
be implemented.
The panel recommends collaborative efforts be organized around geographically-specified outcomes such as
soil quality, water quality, water conservation, wildlife
habitat, air quality, and other environmental outcomes.
Diverse teams—inside and outside USDA—should be
employed to build capacity to do strategic resource management regarding each geographically-specified outcome. Collaborative teams involving expert staff from
multiple federal and state resource agencies and nongovernmental organizations are the only feasible way to
produce the most important strategic components the
panel thinks are currently missing: (1) strategies for “smart
monitoring,” (2) assessment of unmet needs and opportunities, and (3) the evaluation of alternative goals and
strategies. Progress will be much faster and cost less if
driven by effective and ongoing collaboration. Indeed,
given the overlapping and multiple agendas of federal,
regional, state, and local units of government and nongovernmental organizations involved with managing soil,
water, air and wildlife, collaboration is the only hope for
strategic resource management on working land.

Reauthorize RCA
The Soil and Water Resource Conservation Act of 1977
(RCA, P.L. 95-192, 16 U.S.C. 2001-2005) calls for the
Secretary of Agriculture to (1) conduct ongoing
appraisals of the soil, water, and related resources of the
United States, (2) develop and periodically update a
program to conserve, protect, and enhance soil, water,
and related resources “consistent with the roles and program responsibilities of other Federal agencies and State
and local governments,” and (3) provide reports to
Congress and the public outlining the results of its
appraisals and the content of it national plan for conservation [See sidebar on page 14.]. The Act identifies six
topics on which data should be included in the appraisal, and ten topics that should be addressed in the program, but does not limit either to the identified topics.
In short, RCA envisioned an ongoing, collaborative
approach to resource assessment, program evaluation,
and evaluation of alternatives that includes many of the
components of the strategic resource management system the panel is recommending. The RCA Act, for
example requires that the assessment and planning
process provide: (1) data on the costs and benefits of
alternative soil and water conservation practices, (2) an
evaluation of the effectiveness of conservation programs
and the progress in meeting conservation objectives,
and (3) an evaluation of alternative ways to provide
conservation. Current law requires that appraisals be
completed by the end of calendar years 1979, 1986,
1995, and 2005, and that the national program be completed by the end of calendar year 1979, and updated by
the end of 1987, 1997, and 2007 to fund these products,
and no products after 2007.The law does not authorize
a specific appropriation level to fund the effort.
The panel thinks the RCA process could and should
serve as an umbrella to support the multi-agency collaborative effort needed to assemble data from CEAP, agency
Government Performance and Results Act (GPRA)
strategic planning, agency performance reporting systems,
and related assessment efforts being conducted by CEAP
lead and collaborating agencies. The RCA umbrella
appears to be a particularly useful way to realize the full
potential of CEAP in informing the debate over the conservation provisions of the 2007 farm bill.
More importantly, the panel strongly recommends
that the RCA be permanently reauthorized in the 2007
farm bill, and legislative language be included to: (1)
authorize sufficient appropriations to support ongoing
assessment, evaluation, and planning activities, (2)
update the purposes and focus of appraisals and planning activities to address the environmental agenda now
confronting owners and managers of the nation’s privately owned land, and (3) strengthen provisions for

collaboration. Such action should provide new authority and funding to create the strategic resource management system the panel recommends and that is critical
to ensuring USDA conservation programs produce the
environmental benefits taxpayers should expect from
their $4 billion investment in conservation.
Goals
The panel also recommends building CEAP and the strategic resource management system around a handful of critical and explicitly stated national and regional environmental goals. The panel concluded that building CEAP—or a
larger strategic resource management system—to address
the performance of every program producing every environmental benefit everywhere in the nation is a massive and
expensive task that is fraught with problematic assumptions.
The panel strongly recommends focusing CEAP and the
strategic resource management system a around a few environmental and natural resource objectives that are of critical
regional or national significance and that are expected to
drive conservation efforts over the coming decades. The
panel hopes that such a geographically-focused approach—
built around goals and regional assessments—could be built
faster and at less expense and could produce more credible
and useful results in the short term.
Regionalization
The panel recommends USDA look more to regionallevel rather than national-level assessments and reporting of results to reduce the cost and increase the utility
and credibility of CEAP and related strategic resource
management activities. Panel members are concerned
that the demands of national-level reporting planned
for CEAP national assessment activities [cropland-CRP,
grazing land, wetlands, wildlife] will obscure opportunities to produce more focused and comprehensive
regional-level assessments. Moreover, regional-level
assessments would likely provide a more specific context for environmental problems, more meaningful
interpretations of CEAP simulations and assessments,
and more effective strategies for employing USDA people and programs to achieve environmental goals.
Representative regional assessments should be properly designed and executed to produce results that are
applicable at larger, even national scales, while reducing
the overall cost of CEAP and related activities.
Regional assessments should, to the maximum extent
possible, be linked to on-the-ground environmental
improvement and resource conservation projects in
which USDA conservation programs play an important
role.The value of investments in on-the-ground monitoring at the project scale—strongly recommended by

CEAP: FINAL REPORT TO USDA

13

the panel earlier in this report—would be multiplied if
project-scale monitoring and evaluation was tied directly to regional- and national assessment. Producing a
national assessment by aggregating valid regional assessments to the national scale is a more credible and meaningful approach than generalizing from the national to
regional scales. Such a system could also draw on past
analyses of trends in resource condition and environmental quality. As important, such an approach would
allow integrated assessment of the aggregate contribution of multiple programs to environmental quality.
Emphasize Strategic Components when Evaluating Programs
As noted earlier, the panel was struck by the emphasis
placed on tracking change in generalized estimates of the
environmental effects of programs as the primary indicator of program performance or results. Much less attention
appeared to be paid to the strategic components of program implementation: the analyses used to set goals, the
rigor employed to evaluate alternative strategies for
employing programs to achieve those goals, the systems
built to monitor progress on-the-ground, and the commitment to reevaluate and modify strategies based on the
results of that monitoring. The unavoidable technical

problems in simulating or extrapolating generalized environmental effects from information about practices
implemented should caution against over-reliance on
such measures in program evaluation. In addition, performance measures tend to be atomistic and highly generalized geographically. Changes in what appear to be
quantitative performance measures may be telling us
more about the assumptions and methods used to simulate or extrapolate those changes than about what programs are actually accomplishing on the ground. The
panel recommends that much more attention be paid to
on-the-ground monitoring and the strategic components
of program management when attempting to assess the
performance of those programs.
The Office of Management and Budget (OMB) uses
the Program Assessment and Rating Tool (PART) to
evaluate the effectiveness of federal programs. PART is
intended to evaluate the performance of programs by
asking questions about (1) program purpose and design,
(2) strategic planning, (3) program management, and (4)
program results and accountability. Answers to those
questions are evaluated and aggregated into a PART
“score” that rates programs as effective, moderately
effective, adequate, ineffective, or results not demonstrated. One of the primary drivers of CEAP, the panel

SOIL AND WATER RESOURCE CONSERVATION ACT OF 1977: SUMMARY OF PROVISIONS
Attainment of policies and purposes
The Secretary shall promote the attainment of the policies
and purposes expressed in this chapter by:
1) appraising on a continuing basis the soil, water, and
related resources of the Nation;
2) developing and updating periodically a program for
furthering the conservation, protection, and
enhancement of the soil, water, and related
resources of the Nation consistent with the roles and
program responsibilities of other Federal agencies
and State and local governments; and
3) providing to Congress and the public, through
reports, the information developed pursuant to paragraphs (1) and (2) of this subsection, and by providing Congress with an annual evaluation report as
provided in section 2006 of this title.

14

2) data on the capability and limitations of those
resources for meeting current and projected
demands on the resource base;
3) data on the changes that have occurred in the status
and condition of those resources resulting from various past uses, including the impact of farming technologies, techniques, and practices;
4) data on current Federal and State laws, policies, programs, rights, regulations, ownerships, and their
trends and other considerations relating to the use,
development, and conservation of soil, water, and
related resources;
5) data on the costs and benefits of alternative soil and
water conservation practices; and
6) data on alternative irrigation techniques regarding their
costs, benefits, and impact on soil and water conservation, crop production, and environmental factors.

Continuing appraisal of soil, water, and related resources

Public participation

In recognition of the importance of and need for obtaining
and maintaining information on the current status of soil,
water, and related resources, the Secretary is authorized
and directed to carry out a continuing appraisal of the
soil, water, and related resources of the Nation. The
appraisal shall include, but not be limited to—
1) data on the quality and quantity of soil, water, and
related resources, including fish and wildlife habitats;

The appraisal shall be made in cooperation with conservation districts, State soil and water conservation agencies, and other appropriate citizen groups, and local and
State agencies under such procedures as the Secretary
may prescribe to insure public participation.

SOIL AND WATER CONSERVATION SOCIETY

was told by agency liaisons, is providing information
needed to get “a good PART score.”
OMB is near the end of the first five-year cycle of
using PART to evaluate federal programs. The panel
understands the OMB may revisit and re-evaluate the
PART process at the end of that five-year cycle. The
panel recommends OMB give more weight to, and
more scrutiny of, the strategic components of program
implementation in any plans for a new cycle of PART
evaluations or as they take a second look at programs
that successfully completed the PART evaluation in the
first cycle. Answers to the first three questions posed by
PART should be used to focus more attention on the
extent to which program design, implementation, and
management enables a rigorous, adaptive management
approach.The panel outlines the six key components of
a strategic resource management system to enable adaptive management in a following section of this report.
Future iterations of the PART process should evaluate
the extent to which these six components are part of
the approach used to implement programs.
A great deal of attention and weight is currently
given to answers to the fourth question—program
results and accountability. CEAP was clearly intended
to simulate the effect of conservation practice applica-

Soil and water conservation program
The Secretary is hereby authorized and directed to develop
in cooperation with and participation by the public through
conservation districts, State and national organizations and
agencies, and other appropriate means, a national soil and
water conservation program (hereinafter called the “program”) to be used as a guide in carrying out the activities of
the Secretary which assist landowners and land users, at
their request, in furthering soil and water conservation on
the private and non-Federal lands of the Nation. The program
shall also include but not be limited to:
1) analysis of the Nation’s soil, water, and related
resource problems;
2) analysis of existing Federal, State, and local government authorities and adjustments needed;
3) an evaluation of the effectiveness of the soil and
water conservation ongoing programs and the overall progress being achieved by Federal, State, and
local programs and the landowners and land users
in meeting the soil and water conservation objectives of this chapter;
4) identification and evaluation of alternative methods
for the conservation, protection, environmental
improvement, and enhancement of soil and water
resources, in the context of alternative time frames,
and a recommendation of the preferred alternatives
and the extent to which they are being implemented;

tion as a way to provide quantitative estimates of program effects to demonstrate and track program results.
The panel’s concerns about the meaning of generalized
indicators of practice/program effects were discussed
earlier in this report.The panel recommends that future
iterations of the PART process should focus more
attention on the extent to which such estimates of
effects can be directly compared to specified goals for
the simulated effects and on the extent to which such
performance measures are linked to ecological and economic context in which the estimated effects occur.
The PART process also allows for combined evaluations of multiple programs in a single PART analysis.The
panel recommends OMB explore options to conduct outcome-specific rather than program-specific evaluation of
performance. For example, rather than evaluate the performance of EQIP in producing change in multiple environmental performance indicators, evaluations would
assess the cumulative effect of multiple USDA conservation programs on water quality, or habitat, or air quality, or
other national or regional goals. Such an approach could
and should reward efforts to integrate implementation of
multiple programs and strategically focus resources from
multiple programs to more effectively achieve designated
environmental goals.

5) investigation and analysis of the practicability, desirability, and feasibility of collecting organic waste materials, including manure, crop and food wastes, industrial organic waste, municipal sewage sludge, logging
and wood-manufacturing residues, and any other
organic refuse, composting, or similarly treating such
materials, transporting and placing such materials
onto the land to improve soil tilth and fertility
6) analysis of the Federal and non-Federal inputs required
to implement the program;
7) analysis of costs and benefits of alternative soil and
water conservation practices; and
8) investigation and analysis of alternative irrigation techniques regarding their costs, benefits, and impact on
soil and water conservation, crop production, and
environmental factors.
Utilization of available information and data
In the implementation of this chapter, the Secretary shall utilize
information and data available from other Federal, State, and
local governments, and private organizations and coordinate
his actions to avoid unnecessary duplication and overlap of
planning and program efforts.
Termination of program
The provisions of this chapter shall terminate on
December 31, 2008.
CEAP: FINAL REPORT TO USDA

15

STRATEGIC RESOURCE
MANAGEMENT BLUEPRINT
he panel discussed at length the components
that must be part of a strategic resource management system. The panel then evaluated the
extent to which CEAP currently provides those components or is anticipated to provide those components.
The panel also identified missing pieces—components
of a strategic resource management system that are not
part of CEAP plans and that were not part of briefings
or background material provided to the panel.

T

Components of a Strategic Resource Management
System
The strategic resource management system envisioned
by the panel must be able to accomplish six tasks:
1. Construct and update the conservation baseline:
describe, quantify, and regularly update—at salient
spatial scales—the magnitude and extent of environmental problems as well as opportunities to
advance conservation efforts on the nation’s
working land.
2. Set meaningful goals: evaluate the feasibility and
desirability—at salient spatial scales—of alternative goals for environmental improvement on
working land.
3. Evaluate alternative strategies: rigorously evaluate the
relative effectiveness of and benefits produced by
alternative strategies for deploying people and
programs to produce the changes in practices and
farming systems where they are most needed to
achieve established goals.
4. Monitor program implementation: assemble and assess
program implementation data to determine if
people and resources have been employed as per
the selected strategy.
5. Monitor environmental benefits: assemble and assess
program implementation and natural resource
inventory and monitoring data to assess whether
progress is being made toward established goals
and if that progress is sufficient to hit established
timelines.

16

SOIL AND WATER CONSERVATION SOCIETY

6. Reevaluate strategies: rigorously reevaluate strategies
based on information generated through monitoring of program implementation and environmental benefits and analysis of the reasons anticipated benefits were or were not achieved.
CEAP, as currently defined and presented to the panel
includes the following components: (1) CEAP National
Assessments—Cropland-CRP, Wildlife, Wetlands, and
Grazing Land, and (2) Watershed Studies—ARS benchmark, CSREES, and NRCS watershed studies. Current
plans for CEAP include elements that are well positioned to become part of a strategic resource management system. Critical components of such a system,
however, appear to be missing or were not included in
the scope of the panel’s inquiry.The table on the opposite page summarizes how the planned components of
CEAP—as currently envisioned—could contribute to a
comprehensive strategic resource management system.

Strategic Resource Management
System Components

CEAP National
Assessments

CEAP
Watershed
Studies
Construct and Update Conservation Baseline

Simulated environmental effects at
national or regional scale of current
conservation effort.

X

Document current level of conservation
effort supported by USDA conservation
programs.

?

Document current level of conservation
effort supported by other federal and
nonfederal activities.

?

Performance
Reporting
Systems

?

X

Assess extent and magnitude of unmet
environmental problems and
opportunities.
Set Meaningful Goals
Evaluate desirability and feasibility of
alternative goals.
Evaluate Alternative Strategies
Simulate the effect of alternative
strategies.

X

?

X

?

Compare simulated strategies to
monitoring data.
Evaluate the relative effectiveness of
and benefits produced by
alternative strategies.

Monitor Program Implementation
Determine if people and resources
have been employed consistent
with selected strategies.

X

Monitor Environmental Benefits
Monitor change in environmental outcomes
which programs are intended to achieve.
Reevaluate Strategies
Simulate the effect of alternative
strategies.
X

?

Compare simulated strategies
to monitoring data.
Evaluate the relative effectiveness
of and benefits produced
by alternative strategies.
Key:
X
?

X

?

Component included in current CEAP plans.
Priority given to component in current CEAP plans is unclear.
Component missing from CEAP plans.

CEAP: FINAL REPORT TO USDA

17

Components Included in Current CEAP Plans
The panel is encouraged that USDA’s current plans for
CEAP could produce several important components of
a strategic resource management system.
The CEAP Cropland-CRP national assessment simulation capabilities will enable large-scale estimation of
the baseline effects of the conservation effort represented by USDA programs. That simulation capability will
also allow for more rigorous assessment and reevaluation of strategies for employing staff and programs to
meet goals—at least at large regional and national scales
and in the short-term for cropland only. Current CEAP
plans call for augmenting these capabilities through
planned activities in the wetlands and wildlife components of the national assessment and building the capacity to estimate effects of conservation activities on grazing land. The CEAP national assessment simulation
capability should be a critical beginning contribution to
a strategic resource management system for the nation’s
working land.
CEAP watershed studies will help refine and validate
the methods used to simulate effects of conservation
practices and programs. Building the science base to support such simulations will be an important contribution
to the strategic resource management system. CEAP
watershed study plans also, however, indicate that building the capacity to conduct regional assessments of the
environmental benefits of conservation activities is one of
the potential outcomes of the watershed study component. It is unclear to the panel, however, what priority is
being accorded to building this regional assessment
capacity and how CEAP scientists will scale-up from
watershed analyses to the regional level. Regional assessments, tied to more specific environmental goals in more
specific geographic settings, will produce more credible
and useful conservation baselines and evaluations of alternative strategies. Moving to a more regional approach to
strategic resource management is one of the promising
opportunities and starting point the panel has identified
for building a strategic resource management system
within the constraints of limited budgets and staff. The
panel recommends USDA and its partners to give greater
emphasis on, and priority to, building such outcome and
region-specific assessment capacity in their overall strategy for CEAP watershed studies.
Performance reporting systems, already in place in
NRCS and FSA, will provide the nuts and bolts information about program implementation needed to document the current level of conservation effort supported by USDA. Efforts to more precisely geo-reference
program implementation data are essential and should
be the highest priority for enhancing current performance reporting systems. Agency staff indicated to the
18

SOIL AND WATER CONSERVATION SOCIETY

panel that such efforts are already underway. The panel
applauds these efforts and encourages USDA to complete this task as soon as possible.
Components Missing from USDA Plans
The panel is concerned that several critical components
of a strategic resource management system are not
included in USDA’s current plans for CEAP. In some
cases, the panel has reason to hope that the missing
pieces will, or are, being provided by other agency
activities beyond the scope of CEAP—and beyond the
charge to the panel. In other cases, the panel did not see
any evidence that provision was being, or would be,
made to provide critical components of a strategic
resource management system.
On-the-Ground Monitoring and Inventory Systems
The most important and troubling missing piece is the
absence of plans for on-the-ground monitoring of
change in the environmental indicators and outcomes
conservation programs and activities are intended to
improve. Plans for tracking the environmental effects of
programs rely largely, if not entirely, on micro-simulations of or extrapolations from practice information.
The panel has stressed its concerns about the credibility and utility of such estimates, even if large investments
are made to improve their accuracy. There is an urgent
need for USDA to revisit and strengthen its plans for onthe-ground monitoring and inventory systems as the primary source of information to evaluate progress toward
goals and the environmental benefits of conservation programs. That monitoring system must be built around a
core set of environmental/ecological indicators tied to the
primary environmental outcomes sought from USDA
conservation programs. Monitoring, for example, should
be employed to determine whether investments in irrigation efficiency or other water conservation measures
are resulting in improved in-stream flows, recharge of
aquifers, or water supply reservoirs. On CRP acres, the
quality and habitat value of cover should be monitoring
to determine if wildlife goals are being met. Many more
examples could be listed. In all cases, monitoring systems must be designed at the right scale and around the
right core indicators.The objective of all such monitoring efforts must be to determine the extent to which
the environmental benefits people and programs were
employed to achieve are in fact being achieved.
Simulations and extrapolations cannot substitute for
on-the-ground monitoring and inventory systems
designed to determine if anticipated conservation and
environmental benefits are being achieved. Such monitoring and inventory systems should be—in fact must

be—accomplished through a collaboration of USDA and
its federal, state, and local partners. No individual agency
or entity has the personnel and resources required to
design and implement such monitoring and inventory
systems. Large investments in monitoring systems are
currently made by multiple federal, state, and local units
of government and nongovernmental organizations in
the public and private sector. CEAP must harness investments in monitoring already being made by other federal, state, and local entities to the maximum extent possible to provide the on-the-ground monitoring information needed for credible assessment of the environmental
benefits of conservation programs.
At a minimum, USDA must carefully compare the
results of its own estimates of the effects of its programs
to the results of its own and other monitoring programs
that track changes in some or all of the environmental
indicators USDA conservation programs are expected
to improve. USDA must be able to explain discrepancies between its estimates of effects and any measured
changes in environmental outcomes or indicators.
Moreover, USDA must be able to clearly and effectively communicate the source and meaning of those discrepancies to policymakers, program managers, and the
conservation community. Unexplained discrepancies
will erode the credibility of both USDA’s estimates of
the effects of its programs and the decisions made based
on those estimates.
USDA and its partner agencies and organizations
have designed and implemented their current monitoring programs to meet specific objectives central to the
missions of each particular agency and organization.
Unfortunately, the collective information gathered
through these programs may not be easily transformed
to meet the needs of USDA to document the environmental effects of its conservation programs. Even with
outstanding interagency cooperation, current monitoring conducted by USDA and its partners may not be
enough. It is likely that additional funds and staff
resources will be needed to support monitoring and
inventory systems. The panel recommends that
Congress mandate that at least one percent of the funding for each authorized program be set aside to support
monitoring and evaluation of those programs. The
funds set-aside for monitoring and evaluation should be
pooled to allow cross-cutting evaluations of the contribution multiple programs are making to a single environmental outcome.
On-the-ground monitoring need not be overwhelming complex and expensive. The key to “smart”
monitoring is carefully selecting indicators that can be
used to tell the story of what is happening in the environment. These indicators can be used to document
both the state of the environment and the way that it is

responding to the conservation actions that are being
implemented. For example, one goal of nutrient management is to provide enough nutrients for good plant
growth, but not so much that the excess leaches into
groundwater or runs off into adjacent streams and lakes.
Soil phosphorus concentrations or index values can
indicate both the nutritional status of the soil and the
potential for losses that can negatively affect water quality. Likewise, one goal of conservations program to
establish riparian buffers is to improve wildlife habitat,
by increasing vegetative cover. Percent cover by woody
vegetation could provide an indicator of the habitat
available to riparian shrub and forest birds, and the
effectiveness of conservation programs that support
buffer creation.
There are many examples of partnership approaches
for managing the costs of on-the-ground monitoring.
These include (1) water quality monitoring programs
implemented in partnership with watershed organizations and other citizen groups, (2) vegetation and bird
habitat surveys implemented in partnership with the
conservation and wildlife organizations such as The
Nature Conservancy, and (3) biotic surveys conducted
with a variety of partners such as the Riverwatch network. Similarly, soils data collected by extension agents
and other agricultural partners could be compiled and
supplemented with targeted sampling to provide a statistically valid overall picture of nutrient conditions in a
watershed. These data could be used both to identify
areas that might receive particular emphasis in new
conservation efforts and document trends in nutrient
status and nutrient losses to the environment over time
as conservation programs are implemented.
The panel strongly recommends using—at a minimum—$40 million of the $4 billion taxpayers are
investing annually in conservation programs to determine if taxpayers are getting the conservation and environmental benefits they are paying for. In the words of
one panel member “nobody builds a $4 billion building
without an architect.” Moreover, the panel recommends
the Administration strongly support such a mandatory
monitoring and evaluation fund and ensure such funding, once authorized, is included in the President’s
annual budget proposal to Congress.
Unmet Needs and Meaningful Goals
The panel did not see any evidence that provision was
being made for rigorous and comprehensive identification and assessment of the extent and magnitude of
environmental and resource management problems that
are not being met through current conservation efforts.
Such information is the foundation of a strategic
resource management system and the basis for setting

CEAP: FINAL REPORT TO USDA

19

meaningful goals. The panel is aware of tentative plans
to use the CEAP cropland-CRP conservation baseline
to simulate the extent of such needs by comparing estimates of environmental effects under current treatment
to the environmental effects that would be expected
under a “full treatment scenario.”This is a step forward,
but such simulations will not substitute for more comprehensive and on-the-ground efforts to identify unmet
needs. The ability to set goals and evaluate alternative
strategies will be seriously constrained in the absence of
solid information about unmet environmental problems
and opportunities.
The panel is aware that strategic planning activities
mandated under the Government Performance and
Results Act (GPRA) and the national conservation
planning process mandated under the Resource
Conservation Act (RCA), must include rigorous assessment of unmet needs and opportunities and must set or
evaluate meaningful goals in order to meet the mandates of those statutes. The panel made no effort to
review, let alone evaluate these strategic planning activities. Such an effort was well beyond the charge to the
panel.The panel hopes and expects these efforts will, in
fact, produce the rigorous assessment of unmet needs so
essential to a strategic resource management system.
The panel strongly recommends that clear links be
forged between CEAP staff, USDA strategic planners,
and staff responsible for the RCA process.Without such
links, many of the benefits of a strategic resource management system will be lost. Linked staff should be
charged with producing a coordinated plan for data
collection, resource inventory, and resource assessment
activities that will produce credible assessments of
unmet needs.The Natural Resources Inventory (NRI)
system, in particular, should be revisited to determine if
and how the system could be revamped to produce statistically valid estimates of the extent and geographic
distribution of conservation needs.
Beyond USDA Conservation Programs
Many private and public sector entities contribute to
the nation’s conservation effort on working land.
Meaningful assessment of the effects of those efforts must
be part of a strategic resource management system. The
CEAP national survey on cropland will collect some
information on such efforts, but it is not clear to the
panel what plans are in place to regularly update and
enrich information available about non-USDA driven
conservation on working land.The panel recommends a
collaborative effort among USDA and its partners to
design and implement a cost-effective system to collect
and consolidate information about the level of conservation effort supported by other federal, state, and local

20

SOIL AND WATER CONSERVATION SOCIETY

units of government, nongovernmental organizations,
and the private sector. Such consolidated information
would provide a much better basis for strategic decisions
about how USDA people and resources could best be
deployed to meet conservation goals.

BUILD THE SCIENCE BASE
uilding the science-base for strategic resource
management on working land must be a primary
purpose for CEAP. That purpose is currently
overshadowed by more short-term objectives, but in
the long-term, building the science base should become
among the most important contributions of CEAP to a
strategic resource management system. Long after the
assessment reports generated by CEAP are out of date,
the research findings and CEAP’s contribution to scientific understanding and technical know-how will continue to contribute to improving the nation’s conservation effort on working land.
The panel applauds USDA’s commitment to building the science base through CEAP. USDA Agricultural
Research Service estimates that $18 million of their
internal funds are directed to CEAP each year. USDA’s
Cooperative State Research, Education, and Extension
Service (CSREES) is providing $2 million and NRCS
$1 million annually to support additional watershed
studies. The NRCS investment in national assessment
work is also contributing to the knowledge base regarding the environmental effects of conservation practices.
USDA is also working with The Wildlife Society and
the Soil and Water Conservation Society to document,
through literature reviews, the state of the scientific
knowledge regarding the effect of conservation practices on environmental outcomes. In addition, a workshop entitled, “Managing Agricultural Landscapes for
Environmental Quality,” is planned for October 2006 to
assemble scientific expertise regarding quantification
and evaluation of environmental effects of conservation
efforts at watershed or other salient geographic scales.
Efforts such as these to build the science base are
critical to long-term success of CEAP and to strategic
resource management. The scientific momentum
CEAP has generated to date must be maintained well
beyond the 2007 farm bill if the full benefits of investments to date are to be realized.

B

Scientific Priorities
Scientific efforts currently underway under the aegis of
CEAP appear to focus on two main objectives: (1) validating models used in national assessments to simulate
the effect of application of conservation practices on
selected environmental outcomes, and (2) refining estimates of effects at regional and/or watershed scales.The
panel believes these are important objectives for CEAP
in the short-term.
The panel is particularly enthusiastic about the
unique value of the watershed study component of
CEAP. Studies conducted at watershed or other salient
geographic scales, could and should become a rich
source of information to improve the effectiveness of
conservation programs. Environmental outcomes—soil
quality, water quality, water conservation, air quality, and
fish and wildlife habitat—are produced at scales larger
than the individual field, farm, or ranch. Strategic management of conservation programs to secure those benefits requires knowledge of where intervention is needed in the watershed as much as what treatment is needed. Watershed studies are uniquely suited to refine our
ability to precisely apply conservation treatments to
ensure the most cost-effective results at the watershed
or other salient geographic scales. The panel strongly
recommends the watershed studies be conducted in a
fashion that will allow comparisons between studies in
regard to methods, measures, and outcome indicators.
The primary objective of the watershed study component of CEAP should be to build the capacity to
effectively direct conservation efforts within designated
watersheds or other salient geographic units to achieve
designated environmental benefits in the most costeffective and sustainable way. The key scientific issues
watershed/geographic unit studies could be designed to
address are many, exciting, and detailed below. In short,
these multiple issues all lead to answers to three broad,
inter-related, and critical questions:
1. What are the most effective interventions—practices, alternative farming systems, landscape or
hydrologic restoration—to achieve specific environmental benefits?

CEAP: FINAL REPORT TO USDA

21

2. Where should such interventions be placed on the
landscape to most effectively achieve specific
environmental benefits?
3. How can we design cost-effective and cost-feasible systems to monitor changes in core environmental indicators?
CEAP should support a USDA-wide agenda of scientific activities designed to determine, in the words of
one panel member, “What are we doing that works;
what are we doing that doesn’t work; and how do we
do things better?” That agenda could and should
expand the watershed studies component of CEAP
over time to (1) increase the land uses and geographic
settings covered by such studies, and (2) to expand the
scope of watershed studies to include effects on aquatic and terrestrial wildlife.The panel recommends additional USDA-CSREES funds, particularly funds from
the National Research Initiative be made available to
scientists and institutions to conduct collaborative
research associated with CEAP activities and objectives.
The panel urges USDA to take advantage of all
opportunities—including additional funding—to create
a set of “representative” watersheds or other salient geographic units in various agricultural settings on a
regional basis.The network of watershed studies should
be constructed through collaboration with multiple
partners in the federal, state, and local governments,
research and education institutions, nongovernmental
organizations, and the private sector and include studies
already underway by other federal, state, and local government agencies and other entities in the private commercial and nonprofit sectors. Such a system of watershed/geographic unit studies should create a solid scientific foundation for strategic resource management
and provide a basis for more credible assessments of the
effect of conservation programs at larger regional or
national scales.
The issues such an expanded network of current and
new watershed/geographic unit studies and related scientific activities undertaken under CEAP should
address include:
• Variable source area hydrology; identifying critical
source areas within watersheds for nonpoint
source pollutants.
• The contribution of geomorphic instability
(stream channel adjustment to changes in hydrologic regime) to resource degradation and effective approaches to managing those effects.
• Wetland functions and values at ecosystem and
watershed scales, including contributions to
healthy fish and wildlife habitats and populations.

22

SOIL AND WATER CONSERVATION SOCIETY

• Quantifying and predicting net quantitative benefits of conservation efforts at watershed and other
salient geographic scales.
• Best options for targeting conservation efforts
within watersheds to enhance effectiveness.
• Integration of fish and wildlife habitat conservation as a critical and contributing component of
improving watershed health and the subsequent
watershed benefits derived from those actions.
• The quantifiable contributions of applied conservation efforts to improve the quality and health of
fisheries habitats and aquatic resources.
• Quantitative benefits of terrestrial wildlife
conservation.
• Effective and reliable quantitative measures of
progress at watershed and other salient
geographic scales.
• Strategies for innovative approaches to monitoring changes in resource and environmental outcomes that are cost-effective and feasible.
• Information needs to enhance or augment adaptive management at watershed/geographic unit
scales.
• Elucidation of the economic and social factors
that drive application of conservation at farm,
ranch, watershed, and other salient geographic
scales.
An active and well-focused research agenda is essential to building a strategic resource management system
and for ensuring the scientific credibility and legitimacy of CEAP efforts.
Capacity to Deliver Science
Panel members repeatedly discussed the status of the
technical services infrastructure—research, education,
and technical assistance—during their deliberations.
Panel members are acutely aware of the importance of
this infrastructure as the foundation of the nation’s conservation effort on working land. Even the best science
base will produce few results unless technically proficient people and technical tools are available to translate
that science into on-the-ground changes in farm,
ranch, and watershed/geographic unit management.
The best and most rigorously developed strategy for
deploying staff and programs will only be as good as the
capability to execute that strategy.
Moreover, rapid scientific advances are improving our
ability to understand and manage the spatial and temporal variability in agricultural landscapes in ways that could
dramatically improve the performance of conservation

programs.Taking advantage of these opportunities, however, will require better information and people trained
to use that information to more precisely apply conservation treatments at field, farm, ranch, watershed, and
other salient geographic scales. We will miss a major
opportunity for cost-effective conservation if our technical infrastructure is too weak to take advantages of the
new opportunities science is creating.
An in-depth assessment of the strengths and weaknesses of the current technical services infrastructure
should be among the first priorities undertaken by a
strategic resource management system. Panel members
realize such an in-depth assessment is largely outside the
scope of CEAP and outside the scope of the panel’s
charge. But if CEAP simulations consider the implications of program implementation at higher levels of
effort or through more strategic implementation, they
will be assuming that that the technical services needed
to secure those higher levels of performance are available. Clearly, those assumptions warrant additional
scrutiny. Moreover, panel members believe technical
assistance and advice is often a more cost-effective
means to enhance environmental benefits from conservation on working land than financial assistance, particularly when directed at implementation of management-intensive, annual conservation systems. The most
important task USDA should give to its strategic
resource management system is a rigorous assessment of
alternatives for strengthening the technical services
infrastructure that drives the nation’s conservation
effort on working land.

CEAP: FINAL REPORT TO USDA

23

CAUTIONS
n its preliminary findings, the panel identified four
potential “soft-spots” that, if not addressed, could
derail CEAP and efforts to build a strategic resource
management system.Those soft spots included:

I

1. Transparency of data, methods, and estimates.
2. Scientific and technical limits on ability to
quantify effects.
3. Simulation versus monitoring.
4. Program neutral versus program specific
assessments.
In this report, the panel has already reconfirmed and
reiterated its concerns about the scientific and technical
limits on ability to quantify effects and its concerns
about over-reliance on simulation and under reliance
on monitoring.The panel would like to also reaffirm its
concerns about transparency and program neutrality.
Transparency
Any effort to simulate or extrapolate the environmental
effects of conservation programs based on the number
and kind of practices those programs fund, faces important hurdles. Estimates of environmental effects must rely
on our ability to infer and/or simulate multiple steps
between practice implementation and ultimate environmental effects. Every step involves assumptions and has
the potential to introduce error in the final estimate.
The panel understands that assumptions and potential error cannot be eliminated from CEAP or agency
performance reporting systems. Instead, the panel
strongly recommends that USDA ensure that CEAP
and performance reporting methods and data be made
as transparent as possible so that users of the information understand the assumptions and potential sources
of error that are imbedded in the assessment and
accounting information generated. Source data and
modeling protocols should be shared widely within the
conservation and scientific communities to maximize
transparency and, therefore, the credibility of results in
the short term. USDA must identify the most sensitive
assumptions and largest sources of error in its results and

24

SOIL AND WATER CONSERVATION SOCIETY

explain the implications of that sensitivity analysis for
users of CEAP and performance reporting system outputs. In the long term, the panel concluded that one of
the most valuable outcomes of the overall CEAP effort
may well be identification of gaps in knowledge and data
collection that allow pinpointing “what we know and
what we don’t know.” Such knowledge can be used to
identify “tractable problems” in our accounting and
assessment systems that can be fixed at a reasonable cost.
Program Neutral
The unit of analysis originally envisioned for CEAP is
the environmental performance of an individual conservation program. Indeed, the original detailed study
plans for CEAP indicate USDA intended to report
national estimates of the environmental effects produced each year by the following programs:
• Environmental Quality Incentives Program (EQIP).
• Conservation Reserve Program (CRP).
• Wetlands Reserve Program (WRP).
• Wildlife Habitat Incentives Program (WHIP).
• Conservation Technical Assistance (CTA).
• Conservation Security Program (CSP).
• Grassland Reserve Program (GRP).
The panel remains concerned that focusing on individual programs will produce fragmented assessments
and impede strategic resource management. As one
panel member put it,“a focus on program may tell us a
lot about that particular program, but it might not tell
us much about how effectively environmental problems
are being solved.” CEAP and the strategic resource
management system, of which it is an integral part,
should be built around a handful of goals and be conducted at geographic scales appropriate to each goal.
The system should stress integrated assessment of the
aggregate contribution of multiple programs to environmental quality.

This report is part of a larger effort—supported by a cooperative agreement with the
U.S. Department of Agriculture (USDA) Agricultural Research Service—to assist with the
design and implementation of the Conservation Effects Assessment Project (CEAP). The
Soil and Water Conservation Society (SWCS) facilitated an external, policy-level review
of CEAP. We relied on input and advice from the CEAP Blue Ribbon Panel, whose members are listed below.
We are indebted to the agency liaisons and presenters who shared their time and
expertise during the panels’ meetings. The findings and conclusions, however, are solely
the responsibility of SWCS.
CEAP Blue Ribbon Panel
Sandra Batie: Professor, Michigan State
University-Department of Agricultural
Economics.

Jennifer Mock: Agriculture Conservation
Policy Analyst, International Association of Fish
and Wildlife Agencies.

Otto Doering: Professor, Purdue UniversityDepartment of Agricultural Economics.

Peter Nowak: Professor, University of
Wisconsin-Madison-Department of Rural
Sociology and Department of Environmental
Studies.

Ronald Hammerschimdt: Director, Division of
Environment, Kansas Department of Health and
Environment.
Krysta Harden: Chief Executive Officer,
National Association of Conservation Districts.
Jay Hardwick: Farmer, National Cotton
Council.
Ferd Hoefner: Policy Director, Sustainable
Agriculture Coalition.

Ross Racine: Executive Director, Intertribal
Agricultural Council.
Tim Searchinger: Attorney, Ecosystem
Restoration Program, Environmental Defense.
Jeff Vonk: Director, Iowa Department of
Natural Resources and SWCS Board of
Directors Ex-Officio Liaison.

Charlie Ingram: Director, Legislative and
Regulatory Affairs, National Association of State
Departments of Agriculture.

Mary Watzin: Director, Rubenstein Ecosystem
Science Laboratory, University of VermontRubenstein School of Environment and Natural
Resources.

Joe Martin: Director of Congressional Affairs,
American Farm Bureau Federation.

Jeffrey Zinn: Specialist in Natural Resources
Policy, Congressional Research Service.

Tamara McCann Thies: Director for
Environmental Issues, National Cattlemen’s Beef
Association.

®

Soil and Water Conservation Society
945 SW Ankeny Rd.
Ankeny, IA 50023
Phone: 515-289-2331
Fax: 515-289-1227
www.swcs.org

Copyright © 2006 by the Soil and Water Conservation Society
All rights reserved
Manufactured in the United States of America
Printed on recycled paper

C
E
A
P