2521.01 Part B - EPA Citizen Science ICR

STATISTICAL METHODS

Data collection methods and procedures will vary; however, the primary purpose of these collections will be for qualitative and quantitative data collections that might help inform scientific research, assessments, or environmental screening; validate environmental models or tools, or; enhance the quantity and quality of data collected across the country’s diverse communities and ecosystems to support the Agency’s mission.

1. Universe and Respondent Selection

Statistical methods will not be used in the selection of respondents. Participants in citizen science and crowdsourcing projects are self-selected. The method for soliciting participation will be described fully in each collection request, but participation may be advertised through targeted outreach and engagement methods like standard and social media outlets, collaborations with on-the-ground partners, public talks, and word-of-mouth.

The number of participants will vary by project submitted under this generic clearance. The variation in participation is likely due to multiple factors like personal interests, accessibility, perceived burden, outreach by the Agency, and success over time. For example, the citizen science program Nature’s Notebook (USGS, OMB Control Number 1028-NEW) reported that participation increased from 40 registered observers in 2008 to 530 registered observers in 2011.

Results will not be used to directly inform Agency regulations or policies. Data also will not be generalized beyond the scope of the sample.

2. Procedures for Collecting Information

Data collection methods and procedures will vary and the specifics of these will be provided with each collection request. Each request under this generic collection will include details on the statistical methodology for stratification and sample selection (if applicable to the collection – this is not applicable to the selection of participants), estimation procedure, degree of accuracy needed for the research purpose described in the justification, unusual problems requiring specialized sampling procedures, and any use of periodic data collection cycles (less frequent than annual) to reduce burden.

Citizen science and crowdsourcing collections submitted under this generic clearance can be stand-alone projects or the methods may be incorporated into an existing or new project, including, but not limited to, projects in the following typology¹:

• Data gathering projects. These projects may include 1) observation, characterization and documentation of natural phenomena or general environmental health observations, opinions, or preferences or 2) surveying participants or screening environmental conditions, including using specialized equipment provided by project leaders to record and submit data, or submitting samples plus descriptors (e.g. of air or water) for testing. Data may be collected using technologies mentioned above, through structured data forms, surveys, focus groups or interviews, submitting photographs or other media, surveys or questionnaires, or providing written observations.

• Classification/problem solving projects. Participants’ tasks may include: 1) observation of recorded materials provided by project organizers (images, video, etc.) through structured data submission forms, surveys or questionnaires in an online or computer program, clicking boxes, highlighting parts of text or image, and providing comments and/or annotations; 2) Classification of images or sounds using structured data submission forms or clicking boxes in an online or computer program; 3) Transcribing information, by typing handwritten logs or notes; 4) Performing a function meant to generate human behavior data; or 5) Problem-solving or manipulation of data. Tasks 1-5 may be conducted via structured actions or instructions or through the use of “human-based computational game” or “game with a purpose”, a human-based computational technique in which a computational process performs its function by presenting certain steps to humans in an entertaining way.

Data gathering and classification/problem solving projects may include participants providing health information, opinions or observations about a research subject’s environmental surroundings. To the extent applicable, these projects will accord with all Agency policies and regulations related to human subjects research and will follow the established approval paths through the Human Subjects Research Review Official².

Citizen science and crowdsourcing collections under this generic clearance may include the following types of questions or requests of participants:

• Profile/Preference information. Projects may request a username and/or password as well as user preference information to facilitate or customize the user experience. Participants may be asked to submit an email address, name, and zip code, as well as acknowledge a privacy policy or terms agreement. Participants may also be presented with an opportunity to be placed on a mailing list for the project. This includes projects administered through a web form or mobile application.

• Personal and Contact Information. Citizen science and crowdsourcing projects may solicit contact information. This information may be necessary to organize and analyze data (i.e., it may be necessary to know which data points are from the same observer). Projects may request contact information (name and email address, zip code, address and phone number) to provide participants with project updates and share data. Participants would be made aware that the publically available data on contact information will be anonymized and aggregated, for example, by census tract, zip code, city, or some other higher level than individual addresses.

• Experience and Expertise. For data quality purposes, citizen science and crowdsourcing projects may request information to evaluate the skill level of the participant by asking about their experience with the project topic. Questions may be about a person’s age range, level or topic of education, participation in organizations, or professional experience.

• Information about Observations. Projects may request accompanying information, such as the date and time of the activity, the location (e.g., GPS coordinates, address, zip code, etc.), the weather (e.g., temperature, precipitation, wind, humidity, visibility, etc.), and a description or characterization of the location (e.g., vegetation type, type of water body, environmental condition, etc.) or personal senses (e.g. smell, visual cues, sound, etc.).

• Project Evaluation. Citizen science and crowdsourcing projects may collect information on the participant’s experience for project evaluation and development. This may include questions on how the participant found out about the project, the amount of time spent, distance traveled, how difficult the task was for the participant, whether the participant enjoyed the experience, and if they will participate again. Projects may also request information to evaluate participant outcomes, such as changes in the participant’s understanding of the scientific process or project topic, through survey questions before and after participation.

• Training. Citizen science and crowdsourcing projects may need to train participants for the purpose of soliciting quality data and increasing participant benefits including education and engagement. Participants may be asked to read materials, watch training videos, or attend training sessions in-person or virtually via a webinar. To ensure that participants understand the training, they may be assessed through testing instruments like a questionnaire or survey, which may be administered online or through a computer program, on paper, in cell a phone app, or in-person.

3. Methods to Maximize Response

Participants will have complete control over their participation in citizen science or crowdsourcing projects. Participants will need to proactively seek out opportunities, respond to an email, or actively sign up for a project in order to participate. Outreach and advertising materials will provide information on how to participate but will not assume participation from anyone. Several existing citizen science and crowdsourcing projects employ engagement tactics to support continued participation, and reduce non-response including newsletters with appreciation, motivation and results delivered to participants, and optional bi-weekly reminders to observe. The collection requests under this generic clearance may utilize some of these techniques while acknowledging that participants have full control over whether to participate or not.

Each collection request under this generic clearance will specify methods to track and increase response rates. Some collection requests will provide opportunities for participants to submit negative data, for example, information on the time and effort to attempt to obtain an observation in the event of no observation.

4. Testing of Procedures

Pretesting may be done with internal staff or a limited number of external colleagues (less than 10). If the number of pretest respondents exceeds nine members of the public, the Agency will submit the pretest instruments for review under this generic clearance.

5. Contacts for Statistical Aspects and Data Collection

Projects submitted under this generic clearance can consult with statisticians in the development, design, conduct, and analysis of the data collection. Statistical expertise is available from agency statisticians or contractors and the Agency will include the names and contact information of persons consulted in the specific information collection requests submitted under this generic clearance as needed.

1 Typology adapted from: Teresa Scassa and Haewon Chung. 2015. Typology of citizen science projects from an intellectual property perspective: Invention and Authorship Between Researchers and Participants. Wilson Center, Commons Lab, Case Study Series, Vol. 5.

2 http://www2.epa.gov/osa/basic-information-about-human-subjects-research