2533ssb04

– PART B OF THE SUPPORTING STATEMENT –

1 SURVEY OBJECTIVES, KEY VARIABLES, AND OTHER PRELIMINARIES

1(a) Survey Objectives

The primary objective of the statistical methods applied in this information collection request is for EPA to survey anaerobic digestion (AD) facilities processing food waste in the United States. It is important for us to collect this data because there is little publicly available information about the number, location, and operational characteristics of AD facilities that accept food waste in the United States. EPA recognizes that better data is needed to support efforts to divert organic wastes from landfills. The reports are useful to stakeholders engaged in efforts to reduce food loss and waste. These stakeholders include: state, local and tribal governments, non-profits interested in diverting food waste from landfills, the biogas industry, renewable energy and wastewater treatment researchers, and universities.

1(b) Key Variables

Key variables will change slightly depending on the type of facility answering the question, however the majority of the questions remain the same. Variables include:

• If the AD facility is considered stand-alone, on-farm, or water resource recovery facility (WRRF)

• Processing Capacity: refers to the maximum amount of food waste feedstock an anaerobic digester can accept per unit time

• Operational Dates: the dates that the AD facilities surveyed became operational

• Food Waste Processed: refers to the actual amount of food waste that is processed by an AD facility for the specific year of the survey

• Non-food Waste Processed: refers to the amount of non-food waste processed via AD. Non-food waste feedstocks include things like mixed yard waste, crop residues, manure, wastewater solids, etc.

• Feedstock Types: refers to the type of food waste and non-food waste that a facility processes. Examples include beverage processing industry waste, food processing industry waste, fats oils and greases (FOG), manure, etc.

• Feedstock Sources: refers to the locations where digesters receive their feedstock. Examples include food/beverage processors, industrial, restaurants and food service, grocery stores/supermarkets, etc.

• Tipping Fees: refers to how facilities receive income based on their feedstocks received.

• Pre-processing: refers to activities occurred prior to processing. Examples include screening, de-packaging/de-bagging, grinding, mixing, etc.

• Operational Specifications: refers to the nature of the digesters and how they are classified as. It can include the temperature range and if the digester is wet or dry.

• Biogas Production: refers to the amount of biogas that is produced by a facility in a given year.

• Biogas Uses: refers to the ways in which a facility uses their biogas. Examples include producing head and electricity, fueling boilers and furnace, producing mechanical power, compressing to vehicle fuels.

• Gas Cleaning Systems: refers to the quality of the biogas and what times of constituents are removed by facility. Examples include sulfur, moisture, siloxanes, etc.

• Solid Digestate Uses: refers to what is done with the solid digestate that is generated at the AD facility. Examples include the digestate being de-watered and land applied, landfilled, composted, etc.

• Liquid Digestate Uses: refers to how facilities manage their liquid digestate that is generated at the AD facility. Examples include the digestate being recirculated through the digester, reused as fertilizer, etc.

We use this data in a report series that we will be publishing on the EPA AD website: https://www.epa.gov/anaerobic-digestion. This report is used by our stakeholders to increase their understanding of the scope and nature of anaerobic digestion in the United States.

1(c) Statistical Approach

We are selecting every facility that we know of that accepts food waste and falls under the stand-alone, on-farm, or WRRF category of anaerobic digesters. We are not selecting facilities that do not accept food waste. We are collecting data from every facility because the key variables differ per facility. Some of these facilities accept a large amount of food waste and some accept a smaller amount. Some are industry dedicated while some are multi-source digesters. We are making sure to clearly outline this in the report, as well as state how the results can be interpreted for the audience. For example, we state in the report that do not do any extrapolation or try to address those unknown facilities or facilities that do not respond to the survey.

Furthermore, we do not think that additional facilities that might not be surveyed are systematically different to the surveys that we are currently getting data from. In the past, when we have added new facilities to the list, we did not see anything that made the facilities seem fundamentally different than the current list. We are constantly communicating with stakeholders to ensure, however, that we are capturing facilities the meet our criteria for the survey.

1(d) Feasibility

EPA anticipates that the survey (the statistical sample) objectives are achievable given the existing time and resource constraints. We have dedicated staff and are looking into contractor support as well to assist in the data analysis to ensure the timeliness of report publication. The facilities have the data and we have a plan in place to encourage participation and send appropriate reminders for this survey.

• High survey response rates for stand-alone (82%- 2018 & 74%-2019) and WRRF (90%-2018 & 94%- 2019) digesters during previous data collection efforts gives EPA confidence that equivalent or better participation rates can be achieved in future data collection efforts. The response rate for on-farm digesters is lower (35%- 2018 & 27%- 2019), but we are planning on doing more intensive outreach to these facilities and will be working with our partners in EPA’s AgSTAR program to ensure higher response rates in the future.

• The only foreseeable obstacles that respondents might face is with unit error. Occasionally a respondent will select the wrong units (e.g. gallons instead of tons) when completing the survey. However, the QA process we’re implementing has allowed us to identify the unit error and correct it.

• We have available funds to complete the survey as designed. We may use available funds to enlist the support of a contractor to assist in assembling the reported data. We do not anticipate making any major design changes, so we are not anticipating changes affecting the survey results.

• Following the schedule that we currently have with collecting data in the fall and early spring, then producing a report at the end of summer, the survey results will be ready in time to serve our program’s decision-making needs.

• The survey results will support future programmatic policy and waste reduction efforts.

2 SURVEY DESIGN

2(a) Target Population and Coverage

Anaerobic digestion facilities take in organic feedstock to generate biogas and a solid and liquid digestate. For this project, we are only sampling anaerobic digestion facilities that process food waste. Those facilities that process food waste are the target population for this project. We are not including facilities that do not process food waste because our program focuses on the sustainable management of food. Also, sampling the total number of AD facilities that do not process food waste would exhaust our available resources. It is therefore preferable for the Agency to only look at this specific subset of AD facilities. The AD industry appears to be growing over time, and one of the goals of this project is to track that growth and display it.

Stand-Alone Digesters:

Most stand-alone digesters accept and process feedstocks from one or more sources for a tipping fee. Stand-alone digesters can be operated as organics recycling businesses, community-based operations or built by a municipality to manage residential food waste.

Other stand-alone digesters are built to process industry specific wastes. This is common in the food and beverage industry. These digesters are co-located at processing plants and are designed to process a certain kind of material.  These units typically do not accept other feedstocks from offsite sources. 

The primary feedstock processed in stand-alone digesters is food waste. However, digesters built to process food waste can also co-digest other organic materials, such as yard waste, manures and wastewater solids. Demand for stand-alone digesters is increasing to address the increase of diversion of food waste from landfills. 

On-Farm Digesters:

On-farm digesters can bring great value to farmers and the surrounding community. Digesters can help farmers manage nutrients, reduce odors, and generate additional farm revenue. Dairy, swine and poultry are the primary animal types for farms with digesters. On-farm digesters can also accept outside food waste as a feedstock. 

The type of digester used on farms is typically determined by the manure management practices in place and type of animal manure that is fed into the digester. Biogas collected from the anaerobic digester systems is often used to generate electricity, to fuel boilers or furnaces, or to provide combined heat and power.

WRRF Digesters:

Plant operators, industry associations, research institutions, and state and federal agencies are now using the term “Water Resource Recovery Facility (WRRF)” to refer to Wastewater Treatment Facilities (also known as Publicly Owned Treatment Works or POTWs). The term “Water Resources Recovery Facility” acknowledges their ability to produce clean water, recover nutrients (such as phosphorus and nitrogen), and reduce our dependence on fossil fuel through the production and use of renewable energy.

The use of anaerobic digestion at WRRFs in the United States dates back to the early 1900’s.  Over the last century, there have been both advancements and setbacks in the development of the technology.  Anaerobic digestion is both a biological process and an engineered system that requires expertise in both disciplines for success.

2(b) Sample Design

2(b)(i) Sampling Frame

There is no mandate for AD facilities to be registered with any formal list, but databases exist where many facilities are listed. One of the databases that was used for this project is EPA’s Livestock Anaerobic Digester Database (https://www.epa.gov/agstar/livestock-anaerobic-digester-database), which provides key information on anaerobic digester projects on livestock farms in the United States. EPA’s AgSTAR program publishes updated data on this website periodically (generally, two times per year) to enhance public access to information and support the development of biogas recovery projects.

A database that was used for WRRF digesters is the Water Environment Federation (WEF) Water Science & Engineering Center’s Biogas Data page (http://www.resourcerecoverydata.org/biogasdata.php), which provides information on water resource recovery facilities with operating anaerobic digesters.

Articles from trade magazines, such as BioCycle (https://www.biocycle.net/), a leading publication on organics recycling, and groups like the American Biogas Council (https://americanbiogascouncil.org/), a national trade association championing the growth of the biogas industry, have also been used. Rounding this out we have accessed various state government websites for listings of facilities

2(b)(ii) Sample Size

The sample size for this data collection will include every AD facility that is known to the Agency to be processing food waste in the following categories: stand-alone facilities (62 facilities), water resource recovery facility (WRRF) facilities (77 facilities), and on-farm facilities (59 facilities).

We did not choose every known AD facility but decided to limit our survey collection to just those that collect food waste because of the magnitude of total digesters, and the usefulness of the data for our program.

2(b)(iii) Stratification Variables

We are not using a stratified sample for this data collection.

The category of questions is the same for all three digester types. The only small differences occur in possible question answers where a digester type does not make sense. For example, in sources of food and non-food waste, we do not ask WRRFs if they receive feedstock from WRRFs, as it is assumed, they receive feedstock from their own process. Also, for the question if a facility uses a wet or dry process, we do not ask WRRFs as all WRRF systems are wet.

2(b)(iv) Sampling Method

The facilities that we are surveying come from all across the United States. Currently, a majority of facilities exist in California and the northeastern United States, as well as in the midwest. When the data collection for the year begins, the EPA staff working on this project will send out the survey link that is hosted on our website via email to the facility operators for them to fill out. Once the survey is complete, the results come back to EPA where staff can export them to Excel spreadsheets for examination. EPA seeks to sample the complete population of stand-alone, on-farm, and WRRF AD facilities that process food waste in the United States.

We state very clearly in our report, that this data collection results only count the facilities EPA is aware of. EPA anticipates that it has a vast majority of AD facilities, as we are checking with industry trade groups, subject matter experts, and other sources that are familiar with known facilities. We are also posting the known facilities in the report so interested groups can check if there are some not being sampled. We regularly present the results of the report at anaerobic digestion related conferences to increase visibility of the survey.

We are very clear in the report on our sample size and frame and what can and cannot be done with the data in the report. For example, we state in the report that do not do any extrapolation or try to address those unknown facilities or facilities that do not respond to the survey.

Furthermore, we do not think that additional facilities that might not be surveyed are systematically different to the surveys that we are currently getting data from. In the past, when we have added new facilities to the list, we did not see anything that made the facilities seem fundamentally different than the current list. We are constantly communicating with stakeholders to ensure, however, that we are capturing facilities the meet our criteria for the survey.

We use an online survey system to administer the survey. Participants click on the link provided in an email to them (also located on the EPA AD website, when open, it is located here: https://www.epa.gov/anaerobic-digestion/how-submit-data-your-anaerobic-digestion-facility) and fill out the survey on the website. The questions are asked sequentially and then the respondents click next and go on to the next question. Once the respondents have finished answering the questions, the survey is submitted and the results are accessible by the EPA.

2(b)(v) Multi-Stage Sampling

Our design does not require multi-stage sampling.

2(c) Precision Requirements

2(c)(i) Precision Targets

Precision of survey estimates is a direct function of sample size. While we are contacting every facility that we are aware of, there are some facilities that are not in our sample frame. Further, not all contacted facilities will respond to our survey. As such, we are interested in how precisely we can estimate our key variables for the population given we are using a sample to perform inference. One of our key variables is whether facilities are able to utilize all the biogas produced at their facility, a question with a binary outcome. We can calculate the expected margin of error for a binomial variable using the following formula

Where z is the value from the standard normal distribution (e.g. 1.96 for 95% confidence), N is the total number of facilities, n is our sample size and p is the proportion of all facilities that would answer ‘yes’ to the question. In our 2019 survey report, our p = 0.69 because 69% of facilities answered “yes” to the question their facility is able to utilize all the biogas they produce. There are 62 facilities in our sample frame. If we assume that covers 78% of all facilities and we expect an 80% response rate, then N = 80 and n = 49.6. Using those values, we can expect a margin of error, e, of 0.078 which is well within our desired level of precision.

2(c)(ii) Non-sampling error

The only non-sampling error we anticipate is those facilities that are either unable to complete the survey or those that are not sent the survey due to the Agency not being aware of their existence.

Coverage Error:

• As stated in this Part B ICR, there is no statutory requirement to keep a list of all AD facilities, or all AD facilities that accept food waste. We have assembled, through many different lists (see section 2(b)(i) Sampling Frame) a thorough group of facilities that we survey. We have utilized many strategies such as holding meetings with stakeholders (e.g. Biocycle, AgSTAR) and presenting/communicating at solid waste and AD-related conferences (e.g. REFOR, ASTSWMO) to communicate about the survey.

Unit Nonresponse:

• We are working on using the Dillman Method to work on increasing our general survey submission rate. Key aspects of the method include:

  • Priming our survey respondents that we will be collecting information via our survey before sending the survey link.

  • Strategically remind participants who have not yet submitted surveys so that they can do so.

  • Follow-up with physical survey reminders in the mail, if necessary

• We are also working with our partners and are actually trying to incorporate part of an MOU with USDA and the dairy industry to increase awareness of the survey so that farms will submit more surveys.

Item Nonresponse:

• The main item nonresponse error we have is the question about tipping fees. During the 2018 survey, 92% of surveys did not answer this question. To account for this error, we state in our survey report that the majority of facilities did not answer this question and we do not report the data.

Response Error:

• The main response error we have is with the units that respondents use to respond with food waste processed or food waste capacity, and with including or leaving out a zero. To address this error, we analyze the data in our QA/QC. We specifically look for data that appears off through an outlier analysis, and through comparisons with previous year’s data. In the future, we are working with our survey design to implement strategies that will reduce these errors, and eliminate much of the follow up we would need to do when we identify response error.

Processing Error:

• We did not detect any processing error in our data. We take steps to ensure that the data we collect and process is free from human error. In certain instances, it is necessary to change individual cells in the data. This happens to correct response error from the survey. Whenever we change the data in this way, we make comments in the cell with the rationale behind the change (e.g. “food waste processed changed from 10,000 tons to 100,000 tons based on 12/13/19 email with requestor”).

2(d) Questionnaire Design

The questionnaire has been designed with subject matter experts in the field of anaerobic digestion to answer key questions that are of interest to the Agency. The results of these questions will be useful to our stakeholders as well as capacity and food waste processed is increased. The data over time will be useful for additional reports, as well as tracking the progress of the Food Loss and Waste 2030 Goal. The survey questions do not substantially change based on digester type. There are minor questions that are different and those have mentioned in other sections of this report.

The major sections of the survey are as follows:

• Facility and contact information: outlines the contact information and related details to identify a facility

• Facility operating status: qualifier if the digester will be included in the survey results as an operating facility and information on time of operation

• Facility total capacity: quantitative information on the capacity of the facility

• Food based feedstocks: quantitative information on the amount of food-based feedstocks processed

• Non-food-based feedstocks: quantitative information on the amount of non-food-based feedstocks processed

• Types of feedstocks: listing of the types of feedstocks accepted by the facility

• Tipping fees: quantitative information on tipping fees for specific materials

• Feedstock sources: listing of the sources of feedstocks accepted by the facility

• Pre-processing information: listing of the type of pre-processing, technologies implemented, and constituents removed

• Design specifications: further detailed information on the type of digester how it operates

• Product end use: how the digestate (liquid and solid) is used

3 PRETESTS AND PILOT TESTS

EPA expects to apply the same basic statistical methods that were used in the first three reports for the surveys that were administered and the data that was collected.

EPA did not conduct any focus groups with the respondents. However, we are continuously improving how we conduct our surveys to ensure a better process in the future. For instance, after the first year of data collection, we noticed a question was difficult for our survey respondents, so we added clarity in the survey, and it helped reduce confusion in subsequent years.

Some surveys seem to be incomplete because the respondents indicated that their facility was not operational yet, or still in the planning/construction phase. In this instance, the data is still useful, just a majority of the questions were unable to be answered, since most facilities will not know the answers until they are operational or close to it. We consider these surveys to be complete for this reason.

In terms of item nonresponse, the questions that most facilities did not answer were the ones about tipping fees. This makes sense, as this information is sometimes related to general operating practices and a facility might not want that information known by others.

4 COLLECTION METHODS AND FOLLOW-UP

4(a) Collection Methods

EPA sends an email out to those AD facilities we have contact information for. These are both for facilities that have responded to the survey in the past and those that we have contact information for. We also put out a call to AD trade organizations to encourage their members to take the survey. We give follow-up reminders to those who have not filled out the survey to remind them. The email we send has the link (that is also located on the EPA AD website) that respondents fill out via a form online. From there, the results are aggregated and exported via Excel so that EPA can examine the data.

4(b) Survey Response and Follow-up

High response rates (74% in 2018 and 68% in 2019) during prior data collections give EPA confidence that equivalent or better levels can be achieved during future data collections. EPA continues outreach efforts to facilities as needed.

Our target response rate for all facilities is 80%. Lessons learned during the first three data collections helped refine future efforts. The future effort:

• Uses better QA/QC collection features during the initial data collection so contact with the survey submitters will be less necessary.

• Ensuring proper guidance is available in the survey so that each question is more easily understood by submitters.

• Holding meetings with stakeholders (e.g. Biocycle, AgSTAR) and presenting/ communicating at solid waste and AD-related conferences (e.g. REFOR, ASTSWMO) to communicate about the survey.

• Continuing to follow-up with facilities that have responded in the past to continue reporting data.

5 ANALYZING AND REPORTING SURVEY RESULTS

5(a) Data Preparation

After facilities complete the survey at our online survey website, EPA allows time for QA/QC review by the anaerobic digestion technical team. This QA/QC process involves the following checks:

• Examining outliers in quantitative data (food waste capacity, food waste processed, biogas produced)

• Checking to ensure that food waste capacity is greater than food waste processed

• Comparing current responses to previous years responses if applicable for the survey participants

• Ensuring that units are correct

If at any time the QA/QC checks indicate an abnormality (e.g. the food waste capacity is less than food waste processed, reported biogas production orders of magnitude greater than normal), we contact the respondents to determine an accurate result.

Next the data is aggregated into a data report. This data report is published on the Agency’s anaerobic digestion webpage and is sent to our stakeholders for public access.

Data problems may occur, but EPA takes the following efforts to reduce problems and increase the dependability and quality of the occurrence data. The online survey data reporting systems and EPA QA/QC assessments do a number of data and unit checks that can correct for improper outliers and other incorrect data. Additional automated QA functions are in place to identify possible data quality issues such as duplicate data submissions and incomplete data. As part of the data QA/QC procedures, all edits or changes made to the data are documented.

5(b) Analysis

For the first three years of data collection, EPA developed its analytical approach for the evaluation of the data collected. EPA will use the same approach to analyze the data in the next rounds of data collection.

The first stage of the analysis involves retrieving the data from the online survey collection system. After that, the data is run through an independent QA/QC process where a number of data checks are performed to ensure accurate data. Certain data is converted, using appropriate conversion factors and density ratios to relate comparable data. The data is then aggregated in a form that does not allow facilities to be identified.

Based on this aggregated form, the data is summarized in columns or counted using Excel functions. From this Excel analysis of the data, it moves to and is presented in tables separated out by the response of the survey (e.g. responses involving the amount of food waste processed, amount of biogas produced, sources of feedstock, biogas cleaning procedures). Once these tables are completed, additional interpretation and analysis is written up in the form of a report where the tables and analysis are written. Once the report is completed, it is posted on the EPA website for our stakeholders to view and download.

Example table from the 2019 survey report. Shown here are the results of the question about the total amount of food waste facilities process.

5(c) Reporting Results

After the final review and formatting of the data collected through this ICR, the report is expected to be made available to the public through the anaerobic digestion web area of the EPA website (https://www.epa.gov/anaerobic-digestion/anaerobic-digestion-facilities-processing-food-waste-united-states-survey), as was done with the data collected for previous years. These reports (around 45-50 pages) summarize the data so no identifiable information can be attributed to any particular survey respondent. Survey documentation is provided, along with the list of survey questions embedded in the pdf report.

The analytical results from the survey will support data findings in the 2030 Food Loss and Waste Goal, further AD processing of food waste, SMM Facts & Figures Report, and general Agency goals involving AD. In doing so, EPA hopes to better divert food waste from landfill into better uses, such as AD, thereby reducing the environmental impacts of potential food waste. Further, the data and statistics gathered can help future research in improving AD, investments in siting and locations, and can feed into future projects to reduce wasted food.

11