Supporting Statement

SUPPORTING STATEMENT

Retrospective Cost Study

October 1, 2010

(1) Title of the Information Collection

Exploratory Analysis of Ex Ante Regulatory Cost Estimates

(2) Short Characterization/Abstract

The U.S. EPA conducts Regulatory Impact Analyses (RIAs) to estimate the benefits and costs of complying with environmental regulations. Even though a number of cost concepts exist, social cost is the theoretically correct measure to use for benefit-cost analysis. Social cost represents the total burden that a regulation will impose on the economy and is represented by the sum of all opportunity costs incurred as a result of the regulation. However, social costs can be very burdensome to calculate. In practice, EPA typically uses engineering cost estimates to predict the costs of complying with regulations. Studies have shown that cost estimates (both total and per unit) in EPA RIAs often differ from the ex-post costs of regulations. To date, EPA has never analyzed in a systematic way why the actual costs of complying with a regulation differ from the estimates of compliance costs predicted in the RIAs.

The existing studies that examine the accuracy of ex-ante cost estimates generally compare ex-post costs found in studies by other government agencies, academic researchers, and trade organizations on an individual regulation with the regulatory agency’s ex-ante cost estimate (Harrington, Morgenstern, and Nelson, 2000; Anderson and Sherwood, 2002; Office of Management and Budget, 2005; Harrington, 2006). Harrington et al. (2000) compares the ex-ante direct costs to ex-post assessments of select EPA and OSHA regulations. In general they find that total costs are overestimated more often than underestimated – of the 28 regulations the authors examined, ex-ante costs are higher for 14 of the rules while only three rules have lower ex-ante costs. However, their findings are different for unit costs. For the thirteen EPA regulations Harrington et al. (2000) assessed, unit costs are often accurate. When they do differ, the frequency of over and under-estimate of unit costs is about evenly distributed.

One potentially important reason for the difference between ex-ante and ex-post costs identified in the literature is unanticipated technological innovation that occurs between the time the rule is promulgated and when the rule is implemented. The existing literature offers other hypotheses for why ex-ante and ex-post costs may differ. Some of these hypotheses include:

• More abatement options available to impacted industry than regulator knows about

• Legal risk greater for underestimate of cost (e.g., establishing “economic achievability” using overestimates of cost ensures affordability at lower estimates

• Rule flexibility not fully accounted for in predictions of firm behavior

• Legislative requirement that technology is available

• Incomplete monitoring and/or enforcement which in turn allows non-compliance

• More barriers to modeled abatement option than regulator knows about

• “Agency-aggrandizing behavior”

• Innovation and learning are difficult to forecast

• Strategic behavior from those providing data to regulators on inputs

• Jointness in compliance not accounted for

• Structure of the analyzed rule does not match structure of the actual rule

• Dated estimates of cost, emissions, existing adoption of pollution controls, and other inputs

• Inaccuracies in the forecasts of the baseline (e.g., fuel prices)

One of the limitations of the existing literature is that there are not a sufficient number of ex-post analyses to provide general support for any hypothesis. Furthermore, with limited ex-post analyses, it is difficult to determine which of the competing hypotheses has the largest impact. In addition, most studies do not examine all of the reasons costs may be over or under-estimated - they typically focus on only one or two reasons. A further limitation of existing studies is that they tend to focus on much older rules and the few case studies that focus on more recent rules tend to concentrate on incentive-based policies.

EPA’s National Center for Environmental Economics (NCEE) is evaluating the degree to which ex-ante costs and ex-post costs of regulations differ and are exploring, through a pilot project, possible reasons why the costs diverge. One of the goals of the project is to determine if there are any systematic biases in EPA’s ex-ante cost estimates. If systematic biases do exist, NCEE intends to explore the sources of these biases. While the literature also compares ex-ante estimates of benefits with ex-post estimates, this project will focus exclusively on costs.

NCEE has selected five rules that have targeted different media to serve as pilot case studies. These rules were selected based on effective implementation date, source category and industries covered and analytical approach or methodology used to estimate costs. The rules selected for the pilot study are:

• National Primary Drinking Water Regulation for Arsenic (published 1/22/2001)

• Integrated NESHAP and Effluent Guidelines for Pulp and Paper (Cluster rule) (published 4/15/1998)

• NSPS for Nitrogen Oxide Emissions from Fossil-Fuel Fired Steam Generating Units (published 9/16/98)

• Locomotive Emission Standards (Regulatory Support Document was issued 1998)

• Methyl Bromide Critical Use Nomination for Preplant Soil Use for Strawberries Grown for Fruit in Open Fields on Plastic Tarps.

For each rule, NCEE will attempt to determine what affected entities did to comply with the selected regulations and how much it cost them to comply. Various methodologies exist for collecting ex-post cost information, ranging from using publicly available data sources, reaching out to experts on industry compliance, reaching out to associations and trade groups, and conducting site visits to facilities, to administering a comprehensive industry survey.

Recognizing that conducting comprehensive industry surveys is time and resource intensive, NCEE will first explore the viability of the other methodologies for collecting the ex post cost information. While simultaneously investigating the feasibility of using publicly available information, NCEE will explore the feasibility of collecting additional information through other means, namely through interviews with industry compliance experts and trade associations. At issue is whether these other sources will be able to produce the cost information with enough detail to be useful in this exercise. With input from the relevant program offices within EPA, NCEE will develop detailed ex-post cost templates based on the regulatory impact analyses for each of the selected rules that will be tested using protocol interviews to gauge their usefulness. .

NCEE is providing two sample question templates for the National Primary Drinking Water Regulation for Arsenic. Because of the preliminary nature of the research and the uncertain outcomes of the focus groups, researchers may deviate from these sample templates.

With contractor support, a number of potential participant organizations will be identified for each rule including trade associations, state authorities, and engineering firms using publicly available information (including relevant Agency documents). Individuals at these respective organizations will be recruited by the contractor by phone to gauge their interest and willingness to participate in protocol interviews. Should they agree to participate, the contractor will share the appropriate template with them via mail (either by USPS or by email).

Two templates are anticipated for each rule: one a more detailed set of questions and another more general set of questions. Examples of the kinds of questions we anticipate for the two types of templates are provided for a sample rule (arsenic in drinking water). Even though each participant will be given both sets of questions, each will be asked to complete as much of the detailed template as possible. In the event the detailed questionnaire proves to be too burdensome for participants to complete, the participants will be asked to complete the list of general questions. A follow-up interview will be conducted to identify any difficulties participants faced in completing the detailed template and to identify problems with the way in which questions were posed. Similarly, participants will be asked to react to the list of general questions and identify problematic questions.

Results from this exercise will inform the design of future templates, providing information as to the appropriate level of detail for different types of respondents. The results will also help us identify viable sample populations for future efforts.

(3) Need for the Collection

The goal of NCEE’s Retrospective Cost Study (RCS) is to evaluate the degree to which ex-ante costs and ex-post costs of regulations differ, to explore possible reasons why the costs diverge, and to determine if there are any systematic biases in EPA’s ex-ante cost estimates and potential sources of these biases. Although a number of studies exist that compare ex ante costs with ex post costs, none have systematically explored the reasons for the divergence. Ultimately, the results of the RCS will be used to identify areas for modeling improvement in ex-ante cost estimates and could inform future revisions to the Guidelines for Preparing Economic Analyses.

This pilot project is an important first step in determining the most fruitful means of collecting ex post cost information with enough detail to eventually (once enough case studies are completed) identify the determinants of the differences in ex ante and ex post costs. Specifically, the collection proposed under the generic ICR will help establish viable methodologies for collecting ex post cost information.

(4) Non-duplication

To the best of our knowledge this study is unique and does not duplicate other efforts. As mentioned above in sections (2) and (3), while other studies do compare ex ante cost estimates to ex post costs, no other study attempts to identify in a systematic way the source of the difference between the ex-ante cost estimate and the ex-post costs.. Specifically, this collection will allow us to explore methodologies for collecting data beyond what is publicly available. These more detailed data will be necessary to identify the source of the variation.

(5) Consultations

This is a new collection so no periodic consultations have been conducted related to this effort.

This collection, however, is of interest to other Agencies including the Office of Management and Budget, [Federal Communication Commission, National Economic Council]. NCEE has made concerted efforts to keep interested parties informed of the progress of this project and to solicit feedback and will continue to do so going forward.

(6) Peer Review Plans

Ex-post Cost templates developed for each selected rule will be subject to routine internal review by the relevant program offices. External peer review is beyond the scope of this initial effort, but would be conducted prior to any comprehensive field study.

(7) Confidentiality

The survey instrument will fully conform to federal regulations – specifically the Privacy Act of 1974 (5 U.S.C. 552a), the Hawkins-Stafford Amendments of 1988 (P.L 100-297), and the Computer Security Act of 1987. Each prospective respondent will be informed that their participation in the exercise is voluntary. While responses may be associated with a type of organization, the identities of the individuals will be kept confidential to the extent permitted by law by the investigators and not associated with their responses in any report.

(8) Sensitive Questions

There are no questions included in the survey materials on sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private or sensitive in materials.

(9) Respondents

Respondents will vary by rule but may include trade organizations, state authorities, and experts on industry compliance. We expect to engage in this kind of collection for each selected pilot rule to explore potential differences by media.

(10) Collection Schedule

The proposed timeline for the data collection is as follows.

Task:	Date:
Contact potential respondents	Spring 2011
Share detailed templates with respondents	Spring 2011
Interview individual respondents and evaluate the usefulness of the template.	Spring/Summer 2011

Please note that the goal of this exercise is to inform the design of the ex-post cost templates and to determine whether and how the templates will need to vary in detail and design according to the type of sample population.

(11) Respondent Burden

We have budgeted for contact with at most 15 entities per rule (including state organizations, trade associations, and experts on industry compliance), assuming participation by, at most, 5 of each kind of organization per rule. .Each participating organization will be asked to complete as much of the ex-post cost template as possible. We expect that each template will take approximately 2 hours to complete. While we recognize several people within an organization may be asked to complete various portions of the template depending on their expertise, we assume that only one person will be needed to complete each section. Follow-up interviews with the primary contact within each participating organization are estimated to take approximately 1 hour.

With these assumptions in mind, we calculate the expected burden as follows:

15 entities x 5 rules x 3 hours = 225 hours.

REFERENCES

Anderson, John, and Todd Sherwood. 2002. Comparison of EPA and other estimates of mobile source rule costs to actual price changes. Society of Automotive Engineers (SAE) paper 2002-01-1980. Presented at the SAE Government Industry Meeting. May 14, 2002, Washington, DC.

Harrington, Winston, Richard D. Morgenstern, and Peter Nelson. 2000. On the accuracy of regulatory cost estimates. Journal of Policy Analysis and Management 19 (2): 297–322.

Harrington, Winston, 2006. Grading Estimates of the Benefits and Costs of Federal Regulation: A Review of Reviews. Resources for the Future Discussion Paper 06-39

OMB. 2005. Validating Regulatory Analysis: 2005 Report to Congress on the Costs and Benefits of Federal Regulations and Unfunded Mandates on State, Local, and Tribal Entities. Available at http://www.whitehouse.gov/omb/inforeg/regpol-reports _congress.html

6