1024-0224 NPS Programmatic Review Form - BTW

NPS Form 10-201 (Rev. 09/2016) OMB Control No. 1024-0224

National Park Service Expiration Date 5-31-2019

PROGRAMMATIC REVIEW AND CLEARANCE PROCESS

FOR NPS-SPONSORED PUBLIC SURVEYS

The scope of the Programmatic Review and Clearance Process for NPS-Sponsored Public Surveys is limited and will only include individual surveys of park visitors, potential park visitors, and residents of communities near parks. Use of the programmatic review will be limited to non-controversial surveys of park visitors, potential park visitors, and/or residents of communities near parks that are not likely to include topics of significant interest in the review process. Additionally, this process is limited to non-controversial information collections that do not attract attention to significant, sensitive, or political issues. Examples of significant, sensitive, or political issues include: seeking opinions regarding political figures; obtaining citizen feedback related to high-visibility or high-impact issues like the reintroduction of wolves in Yellowstone National Park, the delisting of specific Endangered Species, or drilling in the Arctic National Wildlife Refuge.

SUBMISSION DATE: March 14, 2019

PROJECT TITLE: National Park Service and Alice Ferguson Foundation’s Bridging the Watershed Partnership Program Assessment

ABSTRACT: (not to exceed 150 words)

The National Park Service (NPS) partners with the Alice Ferguson Foundation to engage schools in the national capital region in the Bridging the Watershed (BTW) outreach program. The program is designed to promote student academic achievement, personal connections with the natural world, lifelong civic engagement, and environmental stewardship through hands-on curriculum-based outdoor studies in national parks and public lands. BTW will identify objectives to improve the quality and effectiveness of its outreach program. The feedback needed to assess program quality and effectiveness will be collected through surveys and interviews.

PRINCIPAL INVESTIGATOR CONTACT INFORMATION:

Name: Karen Kortecamp, PhD Title: Associate Professor/President, CEA

Affiliation: The George Washington University/Curriculum & Evaluation Associates (CEA) Phone: (301)674-6823

Address: 1029 Ancora Blvd, North Venice, FL 34275

Email: [email protected]

PARK OR PROGRAM LIAISON CONTACT INFORMATION:

Name: Linda Lutz-Ryan Title: Regional Chief of Interpretation and Education

Affiliation: National Capital Region, NPS Phone: (202)619-7245

Address: 1100 Ohio Dr. SW, Washington, DC 20242

Email: [email protected]

PROJECT INFORMATION:

Where will the collection take place?

Antietam National Battlefield; Catoctin Mountain Park; GWMP-Great Falls; Fort Washington Park; Harper’s Ferry NHP; Manassas NBP; Monocacy National Battlefield; National Mall and Memorial Parks;

Fredericksburg and Spotsylvania NMP; Piscataway Park; Oxon Cove Park; Rock Creek National Park, Chesapeake and Ohio Canal NHP; Prince William Forest Park; Richmond National Battlefield Park

Sampling Period Start Date: July 1, 2019 Sampling Period End Date: December 30, 2019

Type of Information Collection Instrument: (Check ALL that Apply)

Mail-Back Questionnaire Face-to-Face Interview Focus Groups

On-Site Questionnaire Telephone Survey

Other (List) Online Questionnaire; Telephone Interview

Will an electronic device be used to collect information?

No Yes – Type of Device: Computer; iPad

SURVEY JUSTIFICATION:

Social science research in support of park planning and management is mandated in the NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS pursues a policy that facilitates social science studies in support of the NPS mission to protect resources and enhance the enjoyment of present and future generations (National Park Service Act of 1916, 38 Stat 535, 16 USC 1, et seq.). NPS policy mandates that social science research will be used to provide an understanding of park visitors, the non-visiting public, gateway communities and regions, and human interactions with park resources. Such studies are needed to provide a scientific basis for park planning and development.

Bridging the Watershed (BTW) outreach program has experienced steadily increasing numbers of participants, in 2017-2018 serving more than 7,000 students and their teachers from 14 regional school systems and several independent schools in 16 national parks. The program requires teachers to participate in a workshop that prepares them for field studies with their students who engage in authentic experiential learning involving application of science methods. Curriculum resources and lessons available online to participating teachers are designed to prepare students for their field studies, guide data analysis after the field study, and provide background information on the subjects covered. BTW offers 5 core curriculum modules: (1) Exotic Invaders: Assessing Exotic Invasive Species; (2) Don’t Get Sedimental: Runoff and Sediment in the River; (3) Talkin’ Trash: Make a Litter Difference; (4) Water Canaries: Assessing Benthic Macroinvertebrates; (5) Watershed Watchdogs: Assessing Water Quality. Not every module is offered at every participating national park. Although the program has expanded significantly in the past decade, BTW has not been assessed since its inception in 1998.

The proposed data collection that will address teachers’ use of BTW training and curriculum resources, teachers’ and students’ attitudes toward their experiences, and administrators’ perceptions of the program can be used to improve BTW programming at the national park sites and outreach to regional schools. The program leads are interested in identifying how effective the training programs, field studies, and curriculum resources are in meeting the program’s objectives to promote student academic achievement, personal connections with the natural world, lifelong civic engagement, and environmental stewardship through hands-on curriculum-based outdoor studies in national parks and public lands. On-line and on-site surveys and phone interviews will be used to collect information. First-time and returning BTW teachers, teachers attending a training workshop, and students completing a field study will be surveyed. There are four collections bundled within this request:

1. Teacher on-line survey will collect information on teachers’ perceptions of the training, program resources, and field study;

2. Teacher workshop survey will collect information on teachers’ perceptions of BTW training;

3. Teachers and school administrator interviews will collect qualitative information about their perceptions of the quality and effectiveness of the BTW program in meeting the needs of secondary students; and

4. Student survey will gather data on students’ perceptions of learning through field study in a national park and what they recall about what they learned.

SURVEY METHODOLOGY:

1. Respondent Universe:

The respondent universe for this collection will consist of five respondent categories that will include teachers, their school administrators and students participating in BTW field studies, spring 2018 – fall 2019.

• Category 1: First-time and returning teachers that participate in BTW field study;

• Category 2: Returning teachers that participated in one or more BTW field studies;

• Category 3: School administrators in 5 school systems with multiple teachers participating in BTW;

• Category 4: Students that participate in BTW field study; and

• Category 5: Teachers completing BTW training workshop in July 2019.

2. Sampling Plan / Procedures:

BTW participants in spring 2018 - fall 2019 will be sampled.

Category 1: First-time and returning teachers (on-line survey)

BTW maintains a database of teachers that participate in the program. Invalid email addresses were removed from the database in fall 2018. The sample for Category 1 (n=175) will consist of 100 first-time and 75 returning teachers, teachers that have participated in more than one field study. Using the BTW database we will invite all 175 to complete an on-line survey.

Category 2: Returning teachers (phone interviews)

A second group of 25 returning teachers, teachers not being asked to complete the on-line survey, will be invited to participate in a phone interview. Returning teachers have engaged their students in field studies at the following national parks: C & O Canal MD side, Great Falls VA, Catoctin, Rock Creek, Harpers Ferry, Prince William Forest Park, Chancellorsville Battlefield, National Mall, Kennilworth Aquatic Gardens, and Antietam National Battlefield. We will contact all of the returning teachers (n=25) by email inviting them to participate in a phone interview. The first 8 teachers that volunteer to participate will be contacted by email to schedule the interview. If for some reason, a teacher that is scheduled to participate in an interview is unable to do so, the 9^th volunteer will be contacted and so on, until 8 teachers have been interviewed.

Category 3: School administrators (phone interviews)

BTW has a strong relationship with 5 regional school systems. We will invite 25 administrators (department chairs, resource specialists, program leaders, assistant principals and principals) in the secondary schools in these systems to participate in a phone interview. The first 5 administrators that volunteer to participate will be contacted by a second email to schedule the interview. If for any reason, an administrator that is scheduled for an interview is unable to participate, the 6^th volunteer will be contacted and so on, until 5 administrators have been interviewed.

Category 4: Students (on-site survey)

Students participating in the any one of 5-core curriculum module field studies will complete a post-field study survey designed to gauge their attitudes towards national parks, the environment, learning through field study, and their knowledge of the curriculum topic. Students will be asked to complete the on-site survey immediately following their field study. BTW estimates that 1,000 students will attend the program and that 500 students will complete the survey during the sampling period (August - November 2019).

Category 5: Teachers completing BTW training workshop (post workshop survey)

BTW routinely schedules training workshops as a precondition for teachers’ participation in field studies with their students. The average number of teachers that participate in the workshop is 15. At the conclusion of the workshop the researcher will describe the BTW study and ask all teachers participating in the workshop to complete a post-workshop survey.

INSTRUMENT ADMINISTRATION:

Category 1: First-time and returning teachers (on-line survey)

Using an online survey tool, all 100 first-time and 75 returning (n=175) BTW participating teachers will receive an email invitation to complete the BTW teacher survey. The email correspondence will thank respondents for their participation in BTW, introduce the BTW study, and inform them that: 1) the purpose of the survey is to assess the quality of the BTW education program and inform future directions; 2) the online survey will take approximately 15 minutes to complete; 3) their input is highly valued and their participation is voluntary; and, 4) their responses will be anonymous. Respondents will be asked to use the URL in the email correspondence that will direct them to the on-line survey. The survey will consist of scaled items that are used to indicate levels of agreement with statements about the program, importance and effectiveness of BTW’s program and satisfaction with BTW resources. Five open-ended items ask respondents to provide comments on their training, participation, importance and effectiveness of the program, use of curriculum resources, and the quality of the field experience with students.

The email invitation will be sent through the on-line survey tool that will allow us to identify respondents and non-respondents while anonymizing their responses so that responses cannot be matched to an individual. Respondents that complete the survey will be sent a ‘thank you for completing the survey’ email. Two weeks after the initial invitation, we will send a second email invitation to all non-respondents.

Categories 2 & 3: Returning teachers and school administrators (phone interviews)

The 25 returning teachers and 25 school administrators will be contacted by email correspondence to invite their participation in a phone interview. The email correspondence will introduce the study and invite these respondents to share their experiences and insights about BTW. Additionally, the correspondence will explain: 1) that the purpose of the interview is to collect information that will be used to inform BTW future directions; 2) the phone interview will take approximately 20 minutes; 3) no preparation is required to participate; 4) their responses will be anonymous and the information they provide will be summarized as a collective response from everyone interviewed; 5) if they are amongst the first 8 respondents (teachers) or 5 respondents (administrators) to agree to participate in the interview they will be contacted by the researcher to schedule the interview; and 6) they will be asked to return a reply of either ‘Yes’ or ‘No’ to indicate their choice to participate or not. If at least 8 teachers or 5 administrators have not agreed to participate in the interview two weeks after the initial invitation is sent, those that have not replied to the initial email will receive a second email invitation.

In the teacher interview the researcher will ask respondents to share their perceptions of the training; the extent to which they used the curriculum resources; their assessment of the quality of those resources; alignment between the curricula and standards they are expected to meet; and, their characterization of the quality and impact of the field studies on students. Administrators will be asked to share their what they know about the program and their insights about the impact of the program on teachers and students, alignment between BTW curriculum and school curriculum standards, and to what extent they would be willing to provide financial support for teachers and students to continue their participation in BTW.

At the time of the interview, the researcher will inform respondents that: 1) the interview is being conducted as part of the assessment of the BTW program; 2) the interview will take approximately 20 minutes; 3) the questions will center on their experiences with the BTW training, teaching materials and resources, and field study at National Park Sites; and 4) with their permission, the interview will be recorded; otherwise the researcher will take hand written notes of the conversation.

Category 4:

All students (n=1,000) grade 6 – 12 participating in at least one of the five field studies modules will be asked to complete an on-site survey at the conclusion of the field study. The survey will be administered and collected by the BTW educator leading the field study. The BTW educator will introduce the BTW study and inform students that: 1) the purpose of the study is to assess the program, not them; 2) the survey will take approximately 4 minutes to complete; 3) their responses will be anonymous and the information they provide will be summarized as a collective response from everyone that completes the survey; 4) when they finish, they will return the survey the BTW educator. The survey will cover five individual field studies modules. Each participating group will have one experience per field trip. The survey will consist of three parts: (1) general demographics, (2) attitudes and perception towards National Parks and field studies and (3) questions specific to particular field study. In essence there, will be five surveys, each distinguished by the field study module

Category 5:

Teachers participating in a BTW training workshop (n=15) will be informed of the BTW study at the conclusion of the workshop. The researcher will introduce the BTW study and inform teachers that: 1) the purpose of the study is to assess the program, not them; 2) the survey will take approximately 2 minutes to complete; 3) their responses will be anonymous and the information they provide will be summarized as a collective response from everyone that completes the survey; and 4) when they finish, they should turn the survey into the researcher. The survey asks teachers the extent to which they found the training beneficial in preparing them to take their students to a national park for field study.

3. Expected Response Rate / Confidence Level:

We anticipate that a total of 1,240 individuals including teachers, students and school administrators will be asked to participate in this collection. Of these, first-time and returning BTW teachers (n=175) will be asked to respond to an on-line survey, returning BTW teachers (n=25) and school administrators (n=25) will be invited to participate in phone interviews, students (n=1,000) will be invited to complete an on-site survey, as will teachers (n=15) attending a training workshop.

Based on the strong partnership between BTW and schools, we estimate that 50% of first-time and returning teachers (n=88) in Category 1 will agree to participate and complete the survey. Previous surveys of similar populations conducted by the Alice Ferguson Foundation had a return rate of between 50% - 80%.

Due to the commitment to BTW exhibited by returning teachers (Category 2) tempered by the knowledge that teachers are challenged by daily time constraints, we estimate that 32% (n=8) of those contacted will agree to participate in the telephone interviews. For similar reasons, we estimate that 20% (n=5) of school administrators (Category 3) will agree to be interviewed.

Because the students (n=1,000) will be contacted on-site and the BTW educators will collect the surveys at the end of the field study, we estimate that 50% (n=500) of the students will complete the survey. This is a typical response rate for student surveys. Likewise, teachers in Category 5 will be contacted in-person by the researcher who will collect the on-site survey at the conclusion of the training workshop. We estimate that 87% (n=13) will complete the survey.

Table 1. Expected Response Rates

	On-line and On-site Contacts
Respondent Group	Initial Contacts	Acceptance	Non-respondents (Soft refusals)	Non-response survey	Hard Refusals
Category 1: On-line teacher survey	175	88 (50%)	(50%) 87	(10%) 8	102
Category 2: Teacher Interviews	25	8(32%)	(68%) 17	(10%) 2	15
Category 3: School administrator interviews	25	5 (20%)	(80%) 20	(10%) 2	18
Category 4: On-site student survey	1000	50 (50%) 0	(50%) 500	(10%) 50	450
Category 5: On-site teacher workshop survey	15	13 (87%)	(13%) 2	(100%) 2	0
TOTAL	1,240	614	626	64	585

4. Strategies for dealing with potential non-response bias:

We will use shorter versions of each surveys and interview questions to conduct the non-response bias check.

Category 1:

All non-respondents to the on-line survey invitation will receive 2 email invitations. After continued non-response this group will be contacted by additional emails asking them to respond to 6 salient statements from the full survey about the quality of the BTW program (i.e. confidence to conduct field study, using parks to teach, involving students in conservation of nature, giving students opportunities to interact with NPS staff, park setting, and immediate impact on students’ learning). A seventh survey item asks why they chose not to complete the full survey.

Categories 2 & 3:

Non-respondents to the phone interview invitation will receive 2 email invitations. Those that remain as non-respondents will be contacted by additional emails asking them to answer key interview questions. Category 2 (teachers) will be asked 3 questions about the quality of the BTW program (i.e. training, curricula, field studies) and why they didn’t respond to the invitation to be interviewed. Category 3 (administrators) will be asked 2 interview questions about the BTW program (i.e. alignment with standards, impact on teachers and students) and why they did not respond to the invitation to be interviewed.

Category 4:

Student non-respondents will be asked to respond to two statements from the full survey about doing a field study, visiting more national parks in the future, and why they did not respond to the full survey.

Category 5:

Non-respondents that attended the training workshop will be asked two questions relating to whether the workshop was beneficial and if it enhanced their comfort with bringing their students to a national park to participate in a BTW field study. Additionally, they will be asked why they did not complete the full survey.

The responses to these questions will help to estimate any non-response bias for the shared variables in each category and offer opportunities to weight data by the variables if differences between respondents and non-respondents are statistically significant.

5. Description of any pre-testing and peer review of the methods and/or instrument:

All data collection instruments were peer-reviewed by BTW and NPS staff. The teacher survey uses well-known and commonly used questions from the NPS Pool of Approved Questions (OMB Control Number 1024-0224, Expiration Date 5/31/2019). Questions were slightly adapted to fit with the BTW project. Based on an informal pre-test (n=3) users required as few as 13 and as many as 17 minutes to complete the survey or an average of 15 minutes. The student survey was developed based on field study reflections students have routinely been completed as part of the BTW curriculum. An informal pre-test (n =5) indicated that the average time to complete the survey is 4minutes. The pre-test (n = 2) of the teacher workshop survey revealed an average completion time of 3 minutes. Teacher and administrator interview questions represent a synthesis of the teacher survey topics. Based on researcher experience the estimated time it will take to complete the interviews is 20 minutes.

BURDEN ESTIMATES:

Category	Explanation of Burden
1 First time Teachers	We estimate that it will take about two minutes for teacher respondents to read each email correspondence inviting them to complete the survey (2 teacher invitation emails = 4 minutes x 88 respondents = 6 hours). It will take an average of 15 minutes per respondent to complete the survey (15 minutes x 88 respondents = 22 hours). The burden hours for the teacher survey is estimated to be 28 hours.
2. Returning teacher Interviews	Of the 25 teachers contacted, we anticipate that 32% (n=8) will read and reply to the email invitations to participate in an interview (2 minutes x 8 respondents = <1 hour); subsequently, engage in scheduling the interview via email (8 minutes x 8 respondents = 1 hour); and complete the interview (20 minutes x 8 respondents = 2.5 hours). The total burden hours for the teacher interview is about 4 hours.
3. School administrator interviews	We anticipate that 5 of the 25 administrators contacted (20%) will read and reply to the email invitations to participate in an interview (4 minutes x 5 respondents = .5 hour); subsequently, engage in scheduling the interview via email (8 minutes x 5 respondents = 1 hour); and complete the interview (20 minutes x 5 respondents = 1.5 hours). The total burden hours for the administrator interview is 3 hours.
4. Student on-site survey	Each respondent (n=500) will listen to a 2-minute introduction and explanation of the student survey by the BTW Educator (2 minutes x 500 respondents = 17 hours). We expect it will take each respondent 4 minutes to complete the survey (4 minutes x 500 respondents = 33 hours). The total burden hours for the respondent student survey is 50 hours.
5. Teacher workshop survey	We expect that that the burden hours for each of the 13 respondents to complete the teacher workshop survey, including listening to the researcher’s introduction is 4 minutes (4 minutes x 13 respondents = <1 hour). The total burden hours for the workshop survey is <1 hour.
Non-response bias check	The average collective burden for the non-response surveys from each category is estimated to be less than 1 hour.

Table 3. Expected Burden Hours

Completed Data Collection	Completed Responses	Completion Time (minutes)*	Burden Hours
Category 1: Teacher on-line survey	88	19	28
Category 2: Teacher interviews	8	32	4
Category 3: Administrator interviews	5	32	3
Category 4: Student survey	500	6	50
Category 5: Teacher workshop survey	13	3	<1
Total number of non-response bias surveys	64	2	2
Total burden hours requested under this ICR:	678		88

* Contact time added to completion time

REPORTING PLAN:

The researcher will submit a written report to program leads and participate in a teleconference to discuss findings at the conclusion of the study period in December 2019.

NOTICES

Privacy Act Statement

General: This information is provided pursuant to Public Law 93-579 (Privacy Act of 1974), December 21, 1984, for individuals completing this form.

Authority: National Park Service Research mandate (54 USC 100702)

Purpose and Uses: This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service.

Effects of Nondisclosure: Providing information is mandatory to submit Information Collection Requests to Programmatic Review Process.

Paperwork Reduction Act Statement

We are collecting this information subject to the Paperwork Reduction Act (44 U.S.C. 3501) and is authorized by the National Park Service Research mandate (54 USC 100702). This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service. All parts of the form must be completed in order for your request to be considered. We may not conduct, or sponsor and you are not required to respond to, this or any other Federal agency-sponsored information collection unless it displays a currently valid OMB control number. OMB has reviewed and approved The National Park Service Programmatic Review Process and assigned OMB Control Number 1024-0224.

Estimated Burden Statement

Public Reporting burden for this form is estimated to average 15 minutes per collection, including the time it takes for reviewing instructions, gathering information and completing and reviewing the form. This time does not include the editorial time required to finalize the submission. Comments regarding this burden estimate or any aspect of this form should be sent to the Information Collection Clearance Coordinator, National Park Service, 1201 Oakridge Dr., Fort Collins, CO 80525.

RECORDS RETENTION - PERMANENT. Transfer all permanent records to NARA 15 years after closure. (NPS Records Schedule, Resource Page 1 of 10

Management and Lands (Item 1.A.2) (N1-79-08-1)).