Supporting Statement_PartB

Supporting Statement for the National Park Service’s Evaluation of Pilot Interventions to Increase Healthful Physical Activity in Parks

OMB Control Number 1024-new

B. Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the OMB Form 83-I is checked "Yes", the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Trail-Intercept Surveys (Acadia, Point Reyes, Sitka, Zion)

The respondent universe for the trail-intercept surveys includes all users of specified trails who are18 years of age or older during the weeks surveying will be conducted. The sampling frame for each park is comprised of those eligible users on the trails during sampling.

Individuals will be sampled as they walk trails during specified time blocks (e.g., 8-10 a.m., 12-2 p.m., 4-6 p.m.), which will vary on a systematic schedule throughout the sampling periods. Interviewers will survey one adult 18 years of age or older and then interview the next eligible person approaching until the day’s sampling period ends. At parks with relatively high levels of trail use (e.g., Sitka and Zion), multiple interviewers may be stationed at different locations along the trails. When groups of trail users are encountered, the interviewer will select the person who is walking on the left (or in front if in single file) of the other walkers/hikers. If this person declines, then the eligible respondent closest to that person will be invited to participate. A log will be maintained recording the disposition of all trail contacts.

Participant Surveys (C&O Canal, Cuyahoga Valley, Timucuan)

C&O Canal. The respondent universe for the CHOH surveys consists of all persons 18 years of age or older employed by cooperating businesses belonging to the Georgetown Business Improvement District (GBID). The GBID membership currently includes over 1,000 businesses; however, no reliable estimate of the total number of employees is available.

A non-random, volunteer sample will be used for this pilot project. Selected businesses will be asked to participate in the intervention program and their employees invited to register online for “Your Towpath to Healthy Living.” Businesses will also be recruited through flyers, the GBID newsletter, and snowball sampling. All persons registering for the program will receive the baseline survey through an e-mail notification. (E-mail and postal mailing addresses will be solicited during registration.) Participants will be directed to a website and supplied with a password giving them access to the baseline questionnaire. For those who request it during registration, a paper copy of the survey will be mailed to their home addresses. Respondents will have the option of returning the completed paper questionnaires at their worksite (using drop boxes) or mailing them back using a stamped, pre-addressed envelope. Follow-up thank you/reminder notices will be sent by e-mail or postal mail one week after registration to those participants who have not completed the baseline survey. A final reminder will be sent two weeks after this (three weeks after the initial mailing) to remaining non-respondents. The U.S. mail follow-up will include a replacement questionnaire. At the conclusion of the walking program, the post-intervention survey will be administered using procedures similar to the baseline survey.

Cuyahoga Valley. The respondent universe for the CUVA student surveys is students enrolled in the 6^th and 7^th grades at Reidinger Middle School in Akron during the 2006-2007 academic year and in the 7^th and 8^th grades during the 2007-2008 year. A control school serving a similar population of students is being recruited. All students in the respondent universe will be asked to complete the baseline survey during May-June 2007. The post-intervention survey will be administered to the same cohort in September-October 2007. The survey will be Web-based and completed in homerooms under the supervision of homeroom teachers. Students who are absent on the day of the administration will be able complete the survey at a later date. It is expected that at least 90% of all eligible students in the universe will be surveyed, minimizing non-response issues.

Timucuan. The potential respondent universe for the TIMU baseline and post-intervention surveys is comprised of all youths 10-12 years of age participating in Sierra Club’s Inner City Outings program in Jacksonville, Florida. No reliable estimate of the number of participants in this age group is available. For the pilot project, the sample will be non-random, comprised of approximately 70 youths who elect to take part in the Timucuan Explorer kayaking program. The program lasts two days, with the baseline survey administered at the start of the first day after participants have arrived in the park. The post-intervention survey will be given to the same participants at the conclusion of the second day, also in the park. A total or near-total enumeration of those taking part is anticipated.

Table B1 summarizes information on the respondent universe, sample size, and estimated response rates for each information collection.

Table B1. Respondent Universe, Sample Sizes, and Expected Response Rates for Trail-Intercept and Participant Surveys.

Pilot park	Respondent universe	Respondent universe size (estimated)	Sample size	Response rate	Estimated responses
Acadia	Great Meadow Loop Trail users	500 (avg. 25 trail users/day)	400 (avg. 10 trail users per day)	90%	360
Point Reyes	Users on 3 trails	700 (avg. 35 trail users/day)	400 (avg. 10 trail users per day)	90%	360
Sitka “High” capacity stratum	Passengers on ships on days of > 2400 capacity	9200 (avg. 1150 trail users/day)	200 (avg. 25 trail users per day)	90%	180
Sitka “Low” capacity stratum	Passengers on ships on days of < 2400 capacity	6400 (avg. 800 trail users/day)	200 (avg. 25 trail users per day)	90%	180
Zion	Pa’rus Trail users	3200 (avg. 160 trail users/day)	680 (avg. 17 trail users per day)	90%	612
C&O Canal	GBID employees	No reliable estimate available	615	65%	400
Cuyahoga Valley students	6th and 7th grade students at two Akron middle schools	650	650	90%	585
Timucuan	Participants 10-12 years old in Sierra Club outing programs	No reliable estimate available	70	90%	63

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Trail-Intercept Surveys

As mentioned above, in the four parks using the trail-intercept survey, visitors will be sampled on trails during specified time blocks. Sampling intervals and schedules will vary by park according to traffic volumes and use patterns at each location. Because the surveys will be conducted during peak visitation seasons at each park, the baseline and post-intervention samples—although independent—will be drawn from the same population of park users, without the confounding effects of seasonal differences in visitor types. Therefore, in addition to trail counts, the effect of the interventions can be assessed through a statistical comparison of responses to identical items in the baseline and post-intervention surveys.

Sitka NHP presents unusual problems for sampling because its trail use is heavily dependent on the capacity of cruise ships arriving in port. None of the other parks has such a strong exogenous influence on trail traffic. Because cruise ships visit Sitka on a pre-arranged schedule, the total passenger capacity of arriving vessels is known in advance. Therefore, the experimental design at Sitka will take cruise ship capacity into account as a blocking variable. Capacity will be used to select which days receive the intervention and which days are control days. Two capacity blocks are planned: block 1 will consist of days when the vessel capacity is less than 2,400 passengers, and block 2 will consist of days when the capacity is larger then 2,400. Within blocks, days will be randomly assigned to the intervention or control groups.

Participant Surveys

At CHOH, the employee sample will be non-random and voluntary, drawn from workers in businesses that are willing to cooperate with the park. Estimating the effectiveness of the intervention will be accomplished through the weblog of steps and/or distance covered by each registrant, as well as statistical analyses comparing responses to identical items in the baseline and post-intervention questionnaires. Unlike the trail-intercept surveys, the baseline and post-intervention samples at CHOH will be comprised of the same individuals, requiring statistical tests based on repeated (rather than independent) measures.

As previously mentioned, the CUVA evaluation will attempt a complete enumeration of 6^th and 7^th grade students at two Akron middle schools, which will be repeated when the students move to the 7^th and 8^th grades at the conclusion of the summer program. CUVA involves a formal control, in addition to the intervention group. Students at the intervention school will be invited to participate in “Get Up, Get Out, and Go!” through school visits by a park ranger, while students at the control school will not receive this invitation. The effect of the intervention can be assessed through a statistical comparison of gain scores for specific questionnaire items between the baseline and post-intervention periods. If accelerometer use is deemed feasible, then these data will provide a criterion measure of baseline and post-intervention physical activity that can be cross-checked against survey responses.

The TIMU evaluation is qualitative, based on a small purposive sample of inner city youth recruited into the kayaking program. Therefore, unlike the other pilot parks, the assessment at TIMU will not involve a statistical comparison of specific baseline and post-intervention measures. Instead, the assessment will involve questioning the participants directly about the Health and Recreation outcomes, supplemented by non-participant observation of the children’s reactions to the program and the park environment, along with the voucher-redemption strategy described earlier. For this reason, the TIMU evaluation will focus on qualitative insights and lessons-learned, rather than on a formal statistical analysis.

Table B2 shows the required sample sizes for given parameters (baseline and post-intervention proportions, power, and alpha=0.05) for a chi-square test for independent samples generated using the PS Program (Version 2.1.31). The sample sizes projected are the minimum needed to achieve sufficient statistical power for detecting significant and meaningful effects of the intervention between baseline and post-intervention periods. For parks employing repeated-measures designs, smaller samples are required. (The number after each question in the first column of Table B2 refers to the question in the trail-intercept survey.)

Table B2. Sample Sizes Required to Detect Statistically Significant Differences in Outcome Effects in Post-Intervention vs. Baseline Measures from Trail-Intercept and CHOH Surveys.

Outcome	Outcome #	Baseline	Post -intervention	Difference	Sample size (power=0.80)
Proportion of trail users with physical health as an important or very important reason for visit (#7)	1	0.20 0.30 0.40	0.55 0.55 0.55	0.35 0.25 0.15	29 60 173
Proportion of trail users spending  30 minutes on trails (#8)	2	0.40 0.50 0.60	0.75 0.75 0.75	0.35 0.25 0.15	30 58 152
Proportion of trail users with high awareness of importance of being physically active (#17)	2	0.40 0.50 0.60	0.75 0.75 0.75	0.35 0.25 0.15	30 58 152
Proportion of trail users with high knowledge of how to be physically active (#18)	2	0.40 0.50 0.60	0.75 0.75 0.75	0.35 0.25 0.15	30 58 152
Proportion of trail users not physically active and who do not intend to be in next 6 months (#16)	3	0.40 0.30 0.20	0.10 0.10 0.10	-0.30 -0.20 -0.10	32 62 199
Odds ratio for increasing physical activity since using the trail, comparing those exposed to intervention to those who were not exposed	1, 3	N/A	1.5 1.7 1.9	N/A	424* 229* 145*

* Assuming prevalence of exposure to intervention among those who did not increase their physical activity is 15%.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Trail-Intercept Survey (Acadia, Point Reyes, Sitka, Zion)

At the four parks employing the trail-intercept survey, data will be collected using a standardized survey instrument. Methods to maximize response rates include the following: 1) interviewers will conduct the interview while walking with respondents to avoid interfering with their physical activity; 2) an option to complete the survey at a table in the shade will be offered; and 3) interviewers will wear official identification, including a name badge, NPS t-shirt and/or hat.

Previous participants will be excluded if they are contacted during the same intervention period (e.g., a baseline period). However, if any individuals agree to complete the survey twice—once during baseline and once during post-intervention periods—then their responses will not be linked because unique identifiers will not be assigned to individuals. These responses will be treated as independent observations and pooled with the baseline and post-intervention sample data.

An interview log will be kept describing each contact. The log will note the time of the contact, whether the interview was accepted or refused, record the date and weather conditions, and note observable characteristics that are also recorded on completed interview forms. These include the type of physical activity the visitor is doing (e.g., walking, jogging, etc.), whether the person is alone, with a pet, a baby stroller, or with others, and the person’s sex. In addition, refusers will be asked to respond to one question from the trail-intercept survey, “How would you rate the quality of your experience on this trail today?” This will allow statistical comparisons to be made between those who consent to be interviewed and those who do not, permitting an assessment of non-response bias. Non-response bias will be described in all reports and the implications (if any) for interpreting the results will be discussed.

Focus Groups (C&O Canal, Cuyahoga Valley)

At CHOH, focus groups participants will be provided with modest incentives for participation. At CUVA, youth will be asked to participate in focus groups as part of their summer outings in the park, thus maximizing participation. Because the focus groups are qualitative, there is no statistical basis for generalizing their findings to all visitors. Therefore, non-response bias does not present the same concerns as it does in quantitative studies based on probability samples and employing inferential statistical analysis.

Participant Surveys (C&O Canal, Cuyahoga Valley, Timucuan)

Several steps will be taken to minimize non-response in the participant surveys at CHOH, CUVA, and TIMU.

At CHOH, data will be collected using a standardized pre- and post-intervention instrument sent to participants in the study. To maximize response rates, a multi-phased contact approach will be used. Both the baseline and post-intervention surveys will be administered online, with a mailback option. Surveys will be delivered either through e-mail or U.S. mail to employees of cooperating businesses in the Georgetown Business Improvement District. The mail-back envelope will have a return address printed on it and will include postage for respondents to mail the completed questionnaire to West Virginia University for data entry and analysis. A follow up thank you/reminder e-mail will be sent out one week after initial mailing; and a second e-mail/letter and questionnaire will be sent out three weeks after initial distribution. To identify potential non-response bias, final sample characteristics will be compared with information from participants collected on their study registration form (age, gender, place of employment). The implications of non-response bias (if any) for interpreting the results will be discussed.

At CUVA, baseline and post-intervention surveys of students will be administered by homeroom teachers in classrooms prior to the beginning of instructional periods. Students who are absent on the days surveys are administered will be given make-up opportunities. For these reasons, response rates to the student surveys are expected to be 90% or higher, minimizing non-response issues.

At TIMU, response rates to the participant baseline and post-intervention questionnaires are expected to be high because they will be administered on-site immediately prior to and after the Timucuan Explorer program. The five-minute baseline survey will be completed by program participants on the first day of the intervention before they are allowed to participate in the kayaking activity. The ten-minute post-intervention questionnaire will be administered at the conclusion of the second day while participants are waiting for transportation back to homes. Because response rates should exceed 90% under these circumstances, non-response issues will be minimized.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Where possible, surveys to be deployed in the pilot parks are derived from existing instruments that have undergone psychometric testing.^12 In most cases, drafts of survey instruments were also completed by fewer than 10 persons who represented the eventual target populations. These tests were used to refine the content and format of questions and to estimate hour burden.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The following individuals provided input on the statistical aspects of the design:

Dr. Ross Brownson, Saint Louis University School of Public Health, 314-977-8110

Dr. Jim Gramann, NPS Social Science Program, 979-845-4920

Dr. Christine Hoehner, Saint Louis University School of Public Health, 314-977-8502

The following table presents lists the principal investigators and individuals who will be responsible for the data collection and analysis for each park. Technical assistance will be provided by the coordinating PIs, Drs. Brownson and Hoehner, and by the NPS Visiting Chief Social Scientist.

Table B3. Principal Investigators and Research Staff.

Pilot park	Principal investigators	Additional individuals responsible for data collection and analysis
Acadia	Dr. Jessica Leahy Dr. John Daigle University of Maine	Michael Shugrue (data collection and analysis)
Point Reyes	Dr. Diara Spain Dr. Bill Kind Dr. Sibdas Ghosh Dominican Univ. of California	Undergraduate students from Dominical University of California (data collection); Dr. Diara Spain, Dr. William King (analysis)
Sitka	Joseph Liddle University of Alaska Southeast	Trained park staff (data collection); Joe Liddle (analysis)
Zion	Dr. Timothy Behrens University of Utah	Trained Zion National Park staff (data collection); Tim Behrens (analysis)
Cuyahoga Valley	Dr. Myron Floyd North Carolina State University	Trained park staff (data collection); Dr. Floyd and Dr. Jim Gramann (analysis)
C&O Canal	Dr. David Smaldone West Virginia University	Linda Cooper and trained park staff (data collection); Dr. Smaldone and Dr. Jingxin Wang (analysis)
Timucuan	Dr. John Eisler University of North Florida	Trained park staff (data collection); Dr. Eisler (analysis)

References:

ATTACHMENTS

A: Executive Order 13266

B: NPS Advisory Board Health and Recreation Committee Final Report

C: National Park Service Act of 1916

D: National Park Omnibus Management Act of 1998 (passages)

E: Trail-Intercept Survey Materials

F: CHOH Focus Group Questions

G: CUVA Focus Group Questions

H: CHOH Baseline Survey Materials

I: CHOH Post-Intervention Survey Materials

J: CUVA Baseline Student Survey

K: CUVA Post-Intervention Student Survey

L: TIMU Baseline Survey

M: TIMU Post-Intervention Survey

10