IC Justification

Request for Approval under the “Generic Clearance for Citizen Science and Crowdsourcing Projects” (OMB Control Number: 2080-0083)

TITLE OF INFORMATION COLLECTION: Evaluating the Pilot Los Angeles Public Library Sensor Loan Program

PURPOSE:

Although air quality around Los Angeles (LA) has improved notably over the past four decades, the region still faces ongoing air quality challenges and citizen groups want increased access to neighborhood-scale air quality information that will help them make informed choices and improve health outcomes. The Los Angeles Public Library system (“LAPL”) is one of the largest in the country with 72 branches serving four million residents. EPA and LAPL are collaborating on a pilot Los Angeles Public Library Sensor Loan Program (“LAPL Pilot”) that will allow residents to check out air sensor equipment just as they would check out a book. Through this project, select branches of the LAPL will receive air sensors, instruction on their use, and training on implementing a related environmental education curriculum. Library staff will use and/or adapt materials to train and educate library patrons. Air sensors will then be loaned out to patrons with the goal of increasing LA communities' understanding of local air quality while testing a model for other communities.  EPA will provide 30 air quality sensors to LAPL and will train 24 “lead librarians” on the use of the sensors, air quality, and environmental education curriculum. Lead Librarians will then train up to 300 other librarians within LAPL prior to launching an air sensor loan program. We anticipate each sensor will be loaned out approximately 3 times for a total of 90 users during the pilot period. We are seeking approval to conduct an evaluation of the LAPL Pilot. The evaluation will consist of three surveys. The first survey will gauge the effectiveness of the training for the Lead Librarians and will be given to up to 24 librarians after the training. The second survey will be provided to all participating librarians (approximately 300) following the end of the pilot loan program to assess the loan program from the librarians’ perspective. The third survey will be provided to library patrons who borrow a sensor, and will be filled out upon return of the sensor to the library.

NEED AND AUTHORITY FOR COLLECTION:

In recent years, miniaturized, lower-cost air monitoring sensors have entered the market for use by researchers, industry, state and local government agencies, Tribal Nations, citizen scientists and the general public. The advancement of portable and lower-cost air sensor technology is making it possible for citizen scientists to measure air quality in their neighborhoods and communities where they live, work and play. Although several community groups, schools, and other groups are using these new air sensor technologies, for the most part these programs have not been formalized or evaluated. To evaluate the success of this program and to determine if the pilot should be retained as a permanent and possibly expanded program of the LA Public Library system, this evaluation will assess the following areas:

1. Individual learning outcomes and environmental literacy of Users and Librarians

2. Effectiveness of Train-the-Trainer training program

   1. Evaluation of the training provided BY EPA/Contractor TO Lead Librarians

   2. Evaluation of the training provided BY Lead Librarians TO other Librarians

   3. Evaluation of the training provided BY Librarians TO Users through community workshops and sensor loan program

3. Ease of use of sensors and effectiveness and suitability of related training materials

4. Implementation of program – costs (monetary/time), perceived benefit, barriers, outreach, etc.

The statutory authority is 42 U.S.C. §7403(a) [Clean Air Act/Research, Investigation, Training, and Other Activities].

USES OF RESULTING DATA:

The overall purpose of this evaluation is to capture lessons learned from the pilot Los Angeles Public Library Sensor Loan Program to provide recommendations for maintaining or expanding this program within the Los Angeles Public Library system. The project team may also seek approvals for Human Subjects Research from EPA’s Human Subjects Research Review Official and the UNC Chapel Hill IRB to expand findings into Best Practices for other programs.

DATA COLLECTION METHODS:

Surveys will be available online via SurveyMonkey or an alternate online system if preferred by the Los Angeles Public Library, or by paper. If a User prefers, they may choose to respond orally, in which case a Librarian will administer the survey verbally. Focus Groups may be formed after the end of the initial evaluation to gather more in-depth feedback and suggestions. Separate approvals for focus group evaluations will be sought later. Survey results will be aggregated and analyzed.

PARTICIPANT UNIVERSE:

Category of Respondent	No. of Respondents	Number of responses per respondent	Participation Time per response	Burden Hours
Sensor Users	90	1 survey	10 minutes	15
Lead Librarians	24	1 survey	15 minutes	6
All Librarians (including Lead Librarians)	300	1 survey	20 minutes	100
Totals	414			121

There will be three categories of respondents, outlined above and below:

“Users” – members of the public who check out a sensor during the pilot loan program period. Approximately 90 Users are expected to check out an air sensor during the pilot loan program. The survey will be administered to all Users when the sensors are returned to the library. All Users who check out a sensor will be asked to fill out a survey. Librarians may be available to assist Users. The survey will take approximately 10 minutes to fill out, for a total burden time of 900 minutes/15 hours.

“Lead Librarians” – EPA – with the assistance of a contractor – will train approximately 24 Lead Librarians on the purpose, use, and analysis of sensors and sensor data. These Lead Librarians will, in turn, train other librarians on these topics. The Lead Librarians will be surveyed immediately after the initial training, and again at the end of the pilot loan program period. The survey will take approximately 15 minutes to fill out, for a total burden time of 360 minutes/6 hours.

“Librarians” – Approximately 300 librarians throughout the Los Angeles Public Library system will be trained by Lead Librarians to implement the sensor loan program. The Librarians will interface with Users, loaning out the sensors, guiding Users on the use of sensors and sensor data, and answering questions and providing technical support to Users. Librarians will be surveyed at the end of the pilot loan program period. The survey will take approximately 20 minutes to fill out, for a total burden time of 6,000 minutes/100 hours.

AGENCY COST: 

The estimated annual cost to the Federal government is $30,436.16. This is based on the salary of a GS-13 Step 10 employee located in San Francisco with a loaded hourly rate (base wage x 1.6 to reflect overhead and benefits) of $107.26. This analysis estimates a maximum of 1 hours work per respondent (1hr x 414 respondents $107.26/hr. = $44,405.64), as well as 10 hrs. to set up a data analysis system and 30 hrs. to aggregate and analyze the data (40hrs x $107.26/hr. =$ 4290.40). The project is anticipated to take 1 year for a total cost of $48,696.04.

STATISTICAL ANALYSIS:

Survey responses will be presented using descriptive statistics (means, medians, modes, standard deviations) and frequency plots.  Responses to open-ended survey questions will be analyzed using thematic analysis (Miles, Huberman, and Saldana, 2013) to identify key themes that emerge across the respondents. [Miles, M. B., Huberman, A. M., & Saldana, J. (2013). Qualitative Data Analysis: Sage.]

DATA QUALITY ASSESSMENT PROCEDURES:

After data is received by EPA investigators, Project Staff will perform cleaning and quality checks on the data.  Because we are seeking feedback on individual experiences we do not expect there to be outliers in the data. However, data will be examined for potential outliers, such as checking multiple boxes on a question where only one answer is requested. Any observed outliers will be examined for influence on the data using sensitivity analyses with and without their inclusion. In this way impact of outlier removal can be ascertained. Any outlier removal from further analyses will be documented, in both study procedures and in any resulting study publications. Due to the nature of citizen science, missingness of report responses is a potential issue. Participants are not required to answer any items in the instrument. Data will therefore be examined for patterns of missing data, in particular data missing at random (MAR) or data missing not at random (MNAR). Responses to open-ended survey questions will be coded separately by two researchers and compared for accuracy. Each researcher will independently apply descriptive codes to the responses and reconcile any differences in codes. Any differences in coding and decisions to reconcile will be documented.

ADMINISTRATION OF THE INSTRUMENT: (Check all that apply)

[x] Web-based or Social Media

[ ] Telephone

[x] In-person

[ ] Mail

[ ] Other, Explain

INSTRUMENT: A copy of the surveys used in this collection are provided below.

CONTACT NAME: _Kathleen Stewart, EPA R9 EMAIL: [email protected]

1. Survey of End Users

EPA Form Number: EPA6000-005

SURVEY

Los Angeles Public Library Air Quality Sensor Loan Program

Thank you for checking out the Los Angeles Public Library’s new air quality sensors!

Can you help us by providing some feedback on your experience using these sensors?

This loan program is a first-of-its kind pilot program between the Los Angeles Public Library system and the U.S. Environmental Protection Agency. Your feedback on your experience using these air quality sensors is extremely important to us. It will be used to see if we should continue and expand this type of program. The information provided will remain confidential, and you will not be contacted unless you tell us you would like to be contacted to provide more feedback or to receive the results of this program. There is space on the final page to provide contact information. This page can be removed from the rest of the survey to keep your responses anonymous.

INSTRUCTIONS:

Please answer questions to the best of your ability. If you were not the main user of the air sensor you checked out, please answer on behalf of the main user, to the best of your ability. You may skip questions if you do not know the answer or are uncomfortable answering any question.

QUESTIONS:

1. How many people used the air sensor checked out to you?

 0  1  2-4  5-15  >15

2. Who used the air sensor checked out to you? (check all that apply)

 Myself  No one  Another member of my household

 Someone outside of my household

3. Who outside of your household used the air sensor? (check all that apply)

 Another family member  A friend or colleague  A school group or teacher

 A community group  A faith-based organization

 Other (please indicate) __________________________________________

4. What is the age of the person/people who used the sensor the most? (select one)

 0-10 years old  11-14 years old  15-19 years old  20-25 years old  26-39 years old  40-62 years old  63+ years old

5. Why did you check out an air sensor from the library? (select all that apply)

 to learn about air quality near a pollution source in my community (like a factory or highway)

 to provide data to decision makers (like a school official, city council, air district, etc.)

 to make a change in my community

 to reduce my (or my family’s) exposure to air pollution

 to learn more about air sensors and air quality

 to conduct research or an experiment

 to check on air quality inside my home, office, school, or another indoor environment

 to compare air quality between locations (such as at my home vs. near downtown, etc.)

 Other __________________________________________________________________

6. Where was the air sensor used (check all that apply)?

 at school or daycare  at work  at home  at a place of worship  at a park or sports field

 near a busy road, port, railroad, or other location with lots of traffic

 near a facility (like a gas station, autobody shop, refinery, etc.)

 while moving (in a car, walking, biking, on public transport, etc.)

 Other ___________________________________________________________________

7. How do you plan to use the air quality data you gathered? (check all that apply)

 To present the data to a decision-maker (such as a school official, supervisor, city official, etc.)

 To present the data to a group (such as a community group, neighbors, a classroom, etc.)

 To request a change be made to reduce air pollution (such as changes in policy or behavior, etc.)

 To make changes in my personal routine to reduce exposure (such as where or when I exercise, etc.)

 To reduce my own impact on air quality (such as changing how I clean my home, driving less, burning less wood, smoking less, etc.)

 To convince someone else to reduce their impact on air quality

 For a school project

 For a research project

 Other __________________________________________________

If you would like to tell us more about how you plan to use your data, please do so here: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

8. How often did you use the air sensor? (select one)

 Never  Once  Rarely  Multiple times a day  Daily  Often  Always

9. If you answered “never” “once” or “rarely” what are the reasons you did not use it often? (select all that apply)

 I couldn’t get the sensor to work  I didn’t have time  It wasn’t useful  Someone else had it  I lost it  I didn’t feel like using it  I thought the sensor would be different  I only needed to use it once or twice  Other

Please use the space below to tell us more about why you did not use the sensor often: __________________________________________________________________

10. How easy was it to use the sensor? (select one)

 1 = Very difficult

 2 = Somewhat difficult

 3 = Neither easy, nor difficult

 4 = Easy

 5 = Very easy

 Impossible: I could not use the sensor

11. How clear were the written instructions? (select all that apply)

 1 = Very unclear

 2 = Somewhat unclear

 3= Clear

 4 = Very clear

 5 = Extremely clear

 I did not receive written instructions

 I lost the instructions

12. How could we improve the instructions? (select all that apply)

 More pictures and diagrams

 More text

 Use a video

 More hands-on instruction at the library

 Make the written information easier to understand

 Make the air sensor easier to use

 Translate into another language (please tell us which language): ____________________

 Other ____________________________________

If you would like to tell us more about your experience using the instructions or your recommendations for improving the instructions, please do so here: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

13. Did you need help from the library while the sensor was checked out to you? (select one)

 No, I didn’t need any help

 I needed help but found the answers myself (through a friend, website, trial and error)

 I needed help but did not ask for it

 I needed help and got help from the library

 I asked for help from the library but did not get help

 I got help from the library but it was not enough

14. How much did you learn about air pollution from participating in the air sensor loan program? (select one)

 Nothing  A little  A moderate amount  A lot

15. How did using the sensor make you feel about your ability to reduce your exposure to air pollution?

 More able to reduce exposure  Did not affect my ability to reduce exposure  Less able to reduce exposure  Other (please explain): _____________________

16. Did you understand this sensor only measures particulate matter pollution?

 Yes  No  I don’t know what that means

17. Are you interested in using sensors that measure other types of pollutants?

 Yes, I would like to measure these pollutants (please list)_______________________

___________________________________________________________________________

 Yes, but I don’t know which pollutants

 No

 I don’t know

18. Please rate how easy or difficult it was to access the data you collected?

 1 = Very difficult

 2 = Somewhat difficult

 3 = Neither easy, nor difficult

 4 = Easy

 5 = Very easy

 I was not able to access the data

19. Please rate how easy or difficult it was to understand the data you collected?

 1 = Very difficult

 2 = Somewhat difficult

 3 = Neither easy, nor difficult

 4 = Easy

 5 = Very easy

 I was not able to understand the data

20. How did you hear about the air sensor loan program? (select all that apply)

 a librarian

 a flier at the library

 a flier in the community (where?) ____________________________________________

 Social media (please list) ___________________________________________________

 Email

 a website _______________________________________________________________

 a friend

 a colleague

 a teacher

 a student

 someone in my community

 a member of my faith-based community

 Other ___________________

21. Do you have any recommendations for improving this loan program? ___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

22. Is there anything else you’d like to tell us about your experience using these air sensors (difficulties, how you used the data, etc.)?

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

OPTIONAL: SOLICITING INTEREST IN FOCUS GROUP

Thank you for providing feedback on your experience using air quality sensors! We may form a small focus group to get more in-depth feedback from people who borrowed an air sensor. Participation would be completely voluntary and unpaid. If you would be interested in providing more feedback through a focus group, please provide your first name and a way to contact you.

Please separate this page from your completed survey if you would like your answers to the survey to be anonymous.

First Name: _________________________________________________________________________

Email Address: ________________________________________________________________________

Phone Number: ( ____ )______________________

OPTIONAL: RECEIVING RESULTS OF THIS PROGRAM

Thank you for providing feedback on your experience using air quality sensors! We will be analyzing the results of surveys collected from people who used the sensors and from participating librarians and will write a report of the findings. If you would like to receive a copy of the report, please provide your email address. If you do not have an email address, please provide your phone number and someone will contact you to obtain a mailing address.

Please separate this page from your completed survey if you would like your answers to the survey to be anonymous.

First Name: _________________________________________________________________________

Email Address: _______________________________________________________________________

Phone Number: ( ____ )_______________________

2. Survey of Librarians

EPA Form Number: EPA6000-006

SURVEY

Los Angeles Public Library Air Quality Sensor Loan Program

Thank you for helping pilot the Los Angeles Public Library Air Quality Sensor Loan Program!

Can you help us by providing some feedback on your experience with this program?

This loan program is a first-of-its kind pilot program between the Los Angeles Public Library system and the U.S. Environmental Protection Agency. Your feedback on your experience loaning these sensors to library patrons is extremely important to us. It will be used to see if we should continue and expand this type of program, will help us understand and address some of the barriers and challenges you may have encountered, as well as help us make improvements to the training, technical support, and instructional materials used. The information provided will remain confidential, and you will not be contacted unless you tell us you would like to be contacted to provide more feedback or to receive the results of this program. There is space on the final page to provide contact information. This page can be removed from the rest of the survey to keep your responses anonymous.

INSTRUCTIONS:

Please answer questions to the best of your ability. You may skip questions if you do not know the answer or if you are uncomfortable answering any question.

QUESTIONS:

1. Which training did you attend (select all that apply)?

 The training conducted by the US Environmental Protection Agency  The training conducted by other librarians  I did not attend any training

2. Please rate your confidence level in responding to a library patron asking you the following questions. Select one answer for each question.

1. What are some types of air pollutants?
   
   
   

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

2. What is PM2.5 and PM10?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

3. Where does PM2.5 come from?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

4. How can I reduce my exposure to PM2.5?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

3. Please rate how confident you would feel helping library patrons in the following scenarios (select one for each scenario):

1. A patron is concerned about pollution from a freeway near their home and asks if the library’s sensors could help them measure the pollution.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

2. Providing a demonstration on how to use the sensor.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

3. A library patron who has checked out a sensor calls and tells you they cannot get the sensor to work (for example, trouble charging, seeing the screen, or hearing it work) and asks for your help.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

4. A library patron who has checked out a sensor calls and asks how to access the data they collected (for example, can’t access the web portal, see data, download data, or open and manipulate a downloaded data file).

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

5. Explaining possible reasons for the differences in air pollution levels measured by a sensor vs levels reported by another source of data (such as a monitor from the South Coast Air Quality Management District or AirNow).

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

4. If a library patron were to ask you where to find information about air pollution levels in their community, where would you tell them to go for information (select all that apply)?

 South Coast Air Quality Management District’s website  AirNow website  A Weather app  A private air sensor data website (like PurpleAir)  Other (please list) _______________________________________________________________________

5. Did you understand this sensor only measures particulate matter pollution?

 Yes  No  I don’t know what that means

6. Are you interested in loaning sensors that measure other types of pollutants?

 Yes, I would like to loan sensors that measure these pollutants (please name): ___________________________________________________________________________

 Yes, but I don’t know which pollutants

 No

 I don’t know

7. Please indicate how difficult was it for you to learn to use the sensor? (select one)

 1 = Very difficult

 2 = Slightly Difficult

 3 = Neither easy, nor difficult

 4 = Easy

 5 = Very easy

 I could not learn how to use the sensor.

8. How difficult was it for a typical library patron to learn to use the sensor? (select one)

 1 = Very difficult

 2 = Slightly Difficult

 3 = Neither easy, nor difficult

 4 = Easy

 5 = Very easy

 library patrons could not learn to use the sensor
 I don’t know

9. Did you notice if certain groups of users had a harder time using the sensor than other groups?

 No

 Yes, the following groups seemed to have a harder time (please list or explain):

_______________________________________________________________

_______________________________________________________________

10. Did you notice if certain groups of users seemed interested in the sensor, but tended to not check them out (i.e. asked about them, but seemed concerned about learning how to use them, or concerned about other challenges?)

 No, I did not experience this

 Some people asked about the sensors but did not check them out, but there was no pattern

 Yes, the following groups seemed to be interested but did not tend to check the sensors out (please list or explain):

_______________________________________________________________

_______________________________________________________________

11. How helpful was the sensor quick start guide? (select one)

 Not helpful at all  A little helpful  Very helpful  I did not receive the quick start guides  I lost the quick start guides

12. How clear was the sensor quick start guide? (select all that apply)

 It was very easy to follow the instructions

 It was a little easy to follow the instructions

 It was challenging to follow the instructions

 The quick start guide did not help at all

13. How could we improve the sensor quick start guide? (select all that apply)

 More pictures and diagrams

 More text

 Use a video

 More hands-on instruction at the library

 Make the text easier to understand

 Make the air sensor easier to use

 Translate into another language (specify which languages): ___________________ _____________________________________________________________________

 Other ____________________________________

14. How helpful was the environmental education curriculum you received? (select one)

 Not helpful at all  A little helpful  Very helpful  I did not receive the curriculum  I lost the curriculum

15. How could we improve the curriculum? ________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

16. Please rate how satisfied you are with the training you received on air quality and sensors? (select one)
    
    

 1 = Dissatisfied

 2 = Slightly dissatisfied

 3 = Somewhat satisfied

 4 = Satisfied

 5 = Very satisfied

17. Overall, how prepared did you feel to start and implement the loan program? (select one)

 1 = Very unprepared

 2 = Moderately unprepared

 3 = Moderately prepared

 4 = Prepared

 5 = Very prepared

18. If you encountered difficulty with the sensors or had questions from library patrons or other librarians you needed help answering, how satisfied are you with the amount of technical support you received? (select one)

 1 = Dissatisfied

 2 = Slightly dissatisfied

 3 = Somewhat satisfied

 4 = Satisfied

 5 = Very satisfied

 Not applicable

19. Overall, how satisfied are you with the pilot sensor loan program (select one)?

 1 = Dissatisfied

 2 = Slightly dissatisfied

 3 = Somewhat satisfied

 4 = Satisfied

 5 = Very satisfied

20. How much would you support continuing the pilot program as a permanent part of the library’s offerings? (select one)

 I strongly support continuing this program  I somewhat support continuing this program  I do not support continuing this program  I would support continuing this program but do not want to be involved

21. Which statement best reflects how you feel about the amount of time expended on the following aspects of the pilot loan program? (select one per item)

1. Amount of time spent on the pilot program, overall:

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

2. Amount of time spent getting trained on air quality and sensors (including time you spent on your own becoming familiar with the sensors):

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

3. Amount of time spent preparing for the start of the loan program (setting up loan procedures, preparing materials, training other librarians, etc):

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

About how many hours total did you spend on this activity? _________________________

4. Amount of time spent training community members on air quality and sensor use:

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

About how many hours total did you spend on this activity? _________________________

5. Amount of time spent troubleshooting sensor operation with library patrons or other librarians:

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

About how many hours/week did you spend on this activity? _________________________

6. Amount of time spent answering questions from library patrons or other librarians about air quality or sensor data:

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

About how many hours/week did you spend on this activity? _________________________

7. Amount of time spent on other tasks related to the pilot sensor loan program not captured above (Please also explain what these tasks were.):

 1 = I would have liked to have spent a lot more time on this

 2 = I needed a little more time on this

 3 = The time spent was just right

 4 = I spent a little too much time on this

 5 = I spent way too much time on this

 I did not spend any time on this

About how many hours/week did you spend on these other activities? _________________

Explanation: ________________________________________________________________

22. How did your library let patrons know about the sensor loan program?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

23. Which age group seemed most interested in using the sensors?

 Children (0-12)  Teens (13-18)  College Students  Adults (19-61)  Seniors (62+)

24. What was the most successful, creative, or interesting use of the sensors that you heard of?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

25. Did you hear of sensor users changing their personal behavior or advocating for policy change (such as reducing wood burning, changing to a less polluted walking route, or presenting data to decision makers)? If yes, please share some examples.

_____________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

26. What were the greatest challenges in implementing the loan program?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

27. Based on your personal experience and your interaction with library patrons using the sensor, complete this sentence: “The sensor was great, but I really wished it _____:”

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

28. What recommendations do you have for improving the loan program (for instance, amount or type of training, time spent training, instructions or curriculum, resource list, train-the-trainer program, advertising, etc)?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

29. Now that the pilot loan period is complete, is there any support you still need on the sensors, training, or the data?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

OPTIONAL: SOLICITING INTEREST IN FOCUS GROUP

Thank you for providing feedback on your experience loaning air quality sensors! We may form a small focus group to get more in-depth feedback from librarians and people who borrowed an air sensor. Participation would be completely voluntary and unpaid. If you would be interested in providing more feedback through a focus group, please provide your first name and a way to contact you. Please separate this page from your completed survey if you would like your answers to the survey to be anonymous.

First Name: _________________________________________________________________________

Email Address: ________________________________________________________________________

Phone Number: ( )______________________

3. Survey of Lead Librarians, post-training

EPA Form Number: EPA6000-005

LEAD LIBRARIAN POST TRAINING SURVEY

Los Angeles Public Library Air Quality Sensor Loan Program

Thank you for helping pilot the Los Angeles Public Library Air Quality Sensor Loan Program!

Can you help us by providing some feedback on the training you just received?

This loan program is a first-of-its kind pilot program between the Los Angeles Public Library system and the U.S. Environmental Protection Agency. Your feedback on the quality and content of the training is very important to us. It will be used to determine if additional support is needed to make this pilot a success and to improve the training for future uses. The information you provide will remain anonymous, unless you choose to provide your name and contact information to provide additional feedback (Question __).

INSTRUCTIONS:

Please answer questions to the best of your ability. You may skip questions if you do not know the answer or if you are uncomfortable answering any question.

QUESTIONS:

1. How would you rate your overall satisfaction with the training? (select one)

 Very satisfied  Moderately satisfied  A little satisfied  A little unsatisfied  Very unsatisfied

2. How would you rate your overall satisfaction with the trainers? (select one)

 Very satisfied  Moderately satisfied  A little satisfied  A little unsatisfied  Very unsatisfied

3. How would you rate your overall satisfaction with the instructional materials used in the training? (select one)

 Very satisfied  Moderately satisfied  A little satisfied  A little unsatisfied  Very unsatisfied

4. Did you practice using the sensors? (select one)

 Yes  No

5. If you answered yes to Question 4, how difficult was it to use the sensor? (select one)

 Not difficult at all  A little difficult  Very difficult  Impossible

6. How helpful were the verbal instructions for operating the sensor? (select one)

 Not helpful at all  A little helpful  Very helpful  I did not hear the instructions

7. How clear were the written instructions for operating the sensor (quick start guide)? (select all that apply)

 It was very easy to follow the instructions

 It was a little easy to follow the instructions

 It was challenging to follow the instructions

 The instructions did not help at all

8. How could we improve the quick start guide? (select all that apply)

 More pictures and diagrams

 More text

 Use a video

 More hands-on instruction at the library

 Make the text easier to understand

 Make the air sensor easier to use

 Translate into another language (specify which languages): ___________________ _____________________________________________________________________

 Other ____________________________________

9. How helpful was the environmental education curriculum you received? (select one)

 Not helpful at all  A little helpful  Very helpful  I did not receive the curriculum  I lost the curriculum

10. How could we improve the curriculum? ________________________________________

_________________________________________________________________________

_________________________________________________________________________

_________________________________________________________________________

11. How would you feel if the training was web-based and not in-person? (select one)

 In-person was essential  I prefer in-person, but web-based could also work  web-based training would have been just as good as in-person  I would prefer web-based  I don’t know

12. How prepared do you feel to train other librarians on the air quality concepts covered in today’s training? (select one)

 Very prepared  Somewhat prepared  A little prepared  Not prepared at all

13. Please rate your confidence level in responding to a library patron asking you the following questions. Select one answer for each question.

5. What are some types of air pollutants?
   
   
   

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

6. What is PM2.5 and PM10?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

7. Where does PM2.5 come from?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

8. How can I reduce my exposure to PM2.5?

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

14. Please rate how confident you would feel helping library patrons in the following scenarios (select one for each scenario):

1. A patron is concerned about pollution from a freeway near their home and asks if the library’s sensors could help them measure the pollution.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

2. Providing a demonstration on how to use the sensor.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

3. A library patron who has checked out a sensor calls and tells you they cannot get the sensor to work (for example, trouble charging, seeing the screen, or hearing it work) and asks for your help.

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

4. A library patron who has checked out a sensor calls and asks how to access the data they collected (for example, can’t access the web portal, see data, download data, or open and manipulate a downloaded data file).

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

5. Explaining possible reasons for the differences in air pollution levels measured by a sensor vs levels reported by another source of data (such as a monitor from the South Coast Air Quality Management District or AirNow).

 1 = Very unconfident

 2 = Slightly unconfident

 3 = Slightly confident

 4 = Moderately confident

 5 = Very confident

15. Do you understand this sensor only measures particulate matter pollution?

 Yes  No  I don’t know what that means

16. What additional training or support do you need to be able to train other librarians, assist patrons, or otherwise implement the pilot sensor loan program?

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

17. What suggestions do you have for improving the training?

______________________________________________________________________

______________________________________________________________________