IC Justification

Request for Approval under the “Generic Clearance for Citizen Science and Crowdsourcing Projects” (OMB Control Number: 2080-0083)

TITLE OF INFORMATION COLLECTION: Using Citizen Science to Analyze Underwater Video in the Great Lakes (Deep Lake Explorer)

PURPOSE:

This is a continuation of the Deep Lake Explorer effort that was started in 2017.

As part of the National Coastal Condition Assessment, over 1000 underwater videos were collected as ancillary information in understanding the benthic community in the Great Lakes. The distribution of invasive species (e.g. round goby, dreissenid mussels) and benthic habitat features in the Great Lakes nearshore zone are knowledge gaps. The primary goal of this project is to evaluate a web-based citizen science approach to analyzing underwater videos. A citizen science approach to underwater video analysis is cost-effective and allows for individual videos to be analyzed by multiple viewers. Multiple analysts increase the precision of the analysis and reduces bias. The underwater video footage interpretation is a growing field but is expensive if only done by trained professional analysts. Citizen scientists will be trained and tested for accuracy using a subset of high-quality videos. The relative precision, accuracy and cost-effectiveness of the citizen science approach will be compared to expert video interpretations. If proven successful, the Deep Lake Explorer application can be utilized to increase efficiency and timeliness of video analysis and reporting in future iterations of the National Coastal Condition Assessment.

The data produced through this project will also contribute towards understanding of invasive species presence and abundance across the Great Lakes, but the primary goal is to establish web-based citizen science underwater video analysis methods. Once methods are established, they can be implemented on a regular basis to analyze underwater video from the National Coastal Condition Assessment, allowing Office of Water and partners to track presence and relative abundance of invasive species in the Great Lakes nearshore through time.

NEED AND AUTHORITY FOR COLLECTION:

Clean Water Act § 104, 33 U.S.C. § 1254, authorizes EPA to encourage, cooperate with and render technical services to individuals, including the general public, to promote the coordination and acceleration of demonstrations, studies and training relating to the causes, effects, prevention and elimination of water pollution.

USES OF RESULTING DATA:

The main goal of this project is to establish methods for web-based citizen science video analysis. The analysis will provide data that can be used to assess ecological conditions of the Great Lakes nearshore.

The information will be used to increase our understanding of habitat characteristics and invasive species in the Great Lakes nearshore, as well as address the following questions:

• Can a citizen science approach to underwater video analysis meet the information needs of resource managers?

• How does the data produced by citizen scientists compare to the data produced by experts in terms of precision, accuracy, and relevancy to management needs? What effects if any does video quality and attribute selection have when comparing analysis of experts and citizen scientists?

These results will guide underwater video indicator development and will be integrated into the EPA’s National Coastal Condition Assessment (NCCA), which aims to fulfill the Clean Water Act assessment and anti-degradation provisions to provide estimates of ecological condition. Results will also support Great Lakes National Program Office (GLNPO) priorities identified under the Great Lakes Water Quality Agreement (GLWQA). The GLWQA Lakewide Management Annex and Great Lakes Nearshore Framework (https://binational.net/2016/09/27/nearshore-eaux-littorales-2/) include commitments for assessing nearshore health and identifying and communicating cumulative impacts and stresses to the Great Lakes’ nearshore environment. These activities are non-regulatory.

The primary audience of the data once interpreted are resource managers in the Great Lakes. It is also intended to inform the public of work that is being conducted in the Great Lakes and provide a view into what the world looks like under the water. The platform that is used to host the project is Zooniverse.org, which is available to the public and has an international presence. It currently has over 1.5 million registered users, see https://www.zooniverse.org/projects/USEPA/deep-lake-explorer.

DATA COLLECTION METHODS:

Underwater videos were collected as part of the National Coastal Condition Assessment in 2010 and 2015, as well as part of assessments of Lake Huron in 2017, and Lake Ontario and the Niagara River in 2018. Videos were collected according to protocol described in Standard Operating Procedure MED-D-SOP_UW Video (Reference No: FIE-054). In brief, underwater cameras are deployed from small or large vessels. Cameras and lights are lowered either attached to a line or attached to a stable carriage attached to a line, to the bottom of the lake. Videos collect approximately one minute of video before being returned to the vessel. Videos will then be segmented into short clips and uploaded to Deep Lake Explorer, where they will be reviewed by multiple analysts. Multiple analysts increase the precision of the analysis and reduces bias. Establishing detailed methods and process for citizen scientist review of underwater video is the objective of this project.

PARTICIPANT UNIVERSE:

Category of Respondent	No. of Respondents	Number of responses per respondent	Participation Time per response	Burden Hours
Video Reviewers	5500	200	0.07 hours	77,000 burden hours
Totals	=			77,000 burden hours

Estimated number of respondents and responses per respondent is based on similar existing projects on the Zooniverse platform which have had 3000 – 8000 respondents, and over 1,000,000 responses. Estimated time to respond per video is 4 minutes.

AGENCY COST:  The estimated annual cost to the Federal government is the estimated annual cost to the Federal government is $80,493, including salaries, overhead and benefits.

• 15 hours per week (ORISE fellow) x $50.00 hourly rate = $39,000 annual

• 10 hours per week (EPA Fed at GS-14) x $80.66 hourly rate = $41,493 annual

ORISE fellow will lead project development including developing web application materials, testing web application, reviewing, compiling data, running statistical analysis, reporting, and coordinating project team and stakeholders.

EPA Fed hours will be devoted to reviewing and testing web application and reports, and assisting with project administrative duties.

STATISTICAL ANALYSIS:

Data exported from Deep Lake Explorer will have 10-15 user analyses for each video clip. These data will be summarized by applying a minimum threshold for user agreement (precision) for each analysis. Analyses that do not meet the precision threshold will be classified as unknown and not used for further analysis. The precision threshold will be determined based on comparison of Deep Lake Explorer results with expert analysis of an initial beta test dataset. Video clip data will then be summarized by sampling site because most sites have multiple clips per site. Results for each site will include species presence/not detected, or generalized substrate classification. We will summarize the percent of sites with species of interest present by waterbody. Where possible (if site coverage was adequate), we will use ArcGIS to interpolate coverage across the waterbodies. For the Niagara River, where a probabilistic study design was used, we will estimate conditions across the waterbody to find percent of the study area with species present. Where available, we will supplement video results with invasive mussel presence from sediment grab samples. Additional statistical analyses will be performed as called for once results are available.

DATA QUALITY ASSESSMENT PROCEDURES:

Data quality assessment procedures are summarized in the quality assurance project plan for the project, titled “Quality Assurance Project Plan - Using Citizen Science to Analyze Underwater Video in the Great Lakes” (September, 2017).

ADMINISTRATION OF THE INSTRUMENT: (Check all that apply)

[ X] Web-based or Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

INSTRUMENT: Instrument script is attached below. Final online product will include mandatory OMB control number, expiration date, and burden statement.

CONTACT NAME: Mari Nord______________ EMAIL: [email protected]_______________

What kind of substrate is present?

• Hard bottom - includes rock, boulders, gravel, and/or cobble.

• Soft bottom - includes sand, silt, and/or clay

• Both hard and soft bottom

Do you see any vegetation?

• Yes

• No

Do you see any fish?

• Yes

• No

Are any of the fish round goby?

• Yes

• No

Do you see any zebra or quagga mussels?

• Yes

• No

Was the quality/resolution of this video adequate to complete all classifications?

• Yes, this video is adequate quality to make classifications.

• No, this video is too poor quality to make classifications.