The 2013 Supporting Statement B for OMB-NEW
Arapaho-Roosevelt National Forest Transportation System Alternatives Study
B. Collections of Information Employing Statistical Methods
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
This study will be administered at three popular recreation sites in the Arapaho-Roosevelt National Forest, including: 1) Brainard Lake Recreation Area (BLRA), including the Indian Peaks Wilderness (IPW); 2) Guanella Pass (GP); and 3) Mount Evans Recreation Area (MERA). Sampling locations will include popular trailheads for day use and overnight (backpacking) visitation and primary parking areas in each of the three recreation areas. The study includes one survey, entitled “ARNF Visitor Survey”, and includes a separate version of the survey, adapted to each of the three study areas, plus a version adapted to the IPW in BLRA, for a total of 4 survey instruments (Instrument A through Instrument D). Instrument A will be administered at BLRA; Instrument B will be administered at GP; Instrument C will be administered at MERA, and Instrument D will be administered at the IPW.
Study Period: The study period is designed to include ARNF's peak summer visitation period, from mid-May through Labor Day weekend, 2013. The study period will be extended to mid-May through Labor Day weekend, 2014, if necessary due to sample size requirements or road construction-related delays in administering the study. Traffic data will be collected during the 2013 study period, and again during the 2014 sampling period (if the extended study period is required) in order to report and interpret study results in the context of corresponding traffic conditions. While it is expected that traffic patterns will be similar from one year to the next, the traffic data provide a basis to verify this.
To collect traffic data, automatic traffic recorders (ATR) will be installed at entry and exit points at each of the study sites to measure directional traffic volumes. The traffic volume data will be recorded in timestamp format, 24-hours per day during the study period. At each location where traffic volume data will be recorded, the ATR will be contained in a watertight Pelican case and chained to a fixed object (e.g., tree, sign, etc.) to minimize the chance of theft or vandalism. Pneumatic road tubes will be attached to the ATR and extended across the road. The ATR’s are “triggered” to record an event (i.e., vehicle count) each time a passing vehicle causes air pressure to pulse through the road tubes. The direction of travel of each passing vehicle is recorded by the ATR’s, based on the sequence in which air pressure pulses through the two road tubes. The speed of each passing vehicle is recorded by the ATR’s, based on the timing between the air pressure pulse triggered in the first road tube the vehicle passes and the air pressure pulse in the second road tube the vehicle passes. Field staff will download data from the ATR’s on a bi-weekly basis, during which time they will inspect the devices to ensure they are operating properly. The cost of traffic data collection is reflected in the table containing the estimated cost to the government (Table 3 in Supporting Statement A). While the ATR’s are designed to collect data unattended for months at a time, the bi-weekly downloads dramatically reduce the risk of data loss due to equipment malfunction. Related, construction indicators will be recorded by the survey administrators as “special events” on the cover of questionnaires administered to visitors. In addition, the ATR’s adequately account for hitched wheels, campers, trailers, and other vehicle types.
Stratifying survey sampling by day and time will be done, if necessitated by the volume and flow of visitors, and will be determined based on traffic data collected at the study sites in the period preceding survey sampling. However, stratifying by time is not expected to be necessary, due to the fact that the study focus is on peak periods of visitor use, and the vast majority of visitor departures from the study sites during these periods occur over a six to eight hour period. Moreover, peak periods of visitor use tend to be concentrated on weekend days; correspondingly, sampling will occur on all weekend days during the study period, and a subset of weekdays. Additional weekdays will be added, as needed, based on the number of completes observed per day through the course of the study period. Analysis of survey data will compare responses on weekends to those on weekdays, and report separate results, if there are statistical differences.
It should also be noted, survey sampling will intentionally exclude holiday weekends to ensure data are representative of “typically busy” summer days, rather than unusually busy holiday weekends. The sampling period includes opportunities to shift sampling days and “off days”, as needed, to collect data on days with “typical” summer weather and avoid sampling on days with particularly unusual weather conditions. Related, as part of field staff orientation, the subjective nature and need for reliability for recording weather conditions during survey sampling periods will be emphasized.
Population Representation:
Instrument A: The respondent universe for the visitor survey will be all recreational visitors (18 years of age and older) to BLRA during the study period.
Instrument B: The respondent universe for the visitor survey will be all recreational visitors (18 years of age and older) to GP during the study period.
Instrument C: The respondent universe for the visitor survey will be all recreational visitors (18 years of age and older) to MERA during the study period.
Instrument D: The respondent universe for the visitor survey will be all recreational visitors (18 years of age and older) to the IPW from trails at BLRA during the study period.
For each survey instrument (Instrument A through Instrument D), a total of 425 visitor groups will be contacted during the sampling period. Past response rate examples for nearly identical visitor surveys conducted by the principal investigator include Isle Royale National Park (100% response), Yosemite National Park (ranging from 81% to 98% response, depending on sampling site in the park), Rocky Mountain National Park (73% response), and Mount Rainier National Park (68% response); OMB control numbers for these previous studies are as follows: Isle Royale National Park (OMB #: 1024-0224); Yosemite National Park (OMB #: 1029-0220 and OMB #: 1024-0224); Rocky Mountain National Park (OMB #: 1024-0224; NPS #: 08-028); Mount Rainier National Park (OMB #: 1024-0224).
Based on response rates from these previous and nearly identical studies, it is expected that 300 (approximately 70%) will agree to participate in the survey. It should be noted, the lower bound of the range of response rates from the previous similar studies noted (70% to 95%) was used to estimate expected response rate for this study to ensure a conservative estimate of sampling effort needed to achieve target sample sizes. That being said, response rates and number of completed surveys will be tracked on a daily basis, which will provide the information necessary to manage survey sampling level of effort according to the study’s sampling plan and burden hours budget. Once target sample size is reached, sampling at that location will cease.
The number of refusals will be recorded and reported in a survey log, and will be used in calculating the response rate and testing for non-response bias within the survey data. Based on the projected sample size for each survey instrument (300 completed questionnaires), there will be 95% confidence that the findings from each survey will be accurate to within 5.6 percentage points, and will have a power level greater than .80 for the range of statistical tests that will be conducted with the data in this study (two-tailed independent samples t-test, chi-square tests of independence, and simple linear and multivariate regression), at the .05 alpha-level. This level of accuracy and statistical power is generally accepted as sufficient in peer-reviewed social science quantitative study findings. Thus, the proposed sample size will be adequate for bi-variate comparisons and will allow for comparisons between study sites and more sophisticated multivariate analysis if deemed necessary. It should be noted, identifying the appropriate sample size and measurement of accuracy does not depend on the estimate of the universe (Fowler, 1993. Survey Research Methods, 2nd Edition. Sage Publications: Newbury Park, CA). In cases where a study is sampling 10% or more of a population, the fraction of the population sampled can have a substantive effect on sampling error estimates. However, in this study, as in the case of majority of survey samples, the sample size for each study site constitutes a small fraction (2.75%, on average; estimated based on USFS Fee Station Data and visitor use data collected at the study sites during summer 2011) of the target population/universe for each study site. Thus, identifying the appropriate sample size and measurement of sampling error is not dependent on the estimate of the universe in the case of this study. In summary, because the target sample sizes for the study constitute a small fraction of the universe (<10%) it is not necessary to know the size of the universe to identifying the appropriate sample size for the target level of accuracy of study results.
Additional Information
The most intensive period of visitor use at the study areas occurs during the summer months (June through September). Visitor use data from the USFS suggest visitor use at BLRA ranges from approximately 10,000 to 20,000 visitors per month during June through September, and visitor use at MERA ranges from approximately 20,000 to 35,000 per month during June through September. Visitor use data for Guanella Pass are not currently available.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection,
Estimation procedure,
Degree of accuracy needed for the purpose described in the justification,
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Visitors at each of three study sites (BLRA, GP, MERA) will be contacted onsite at primary trailheads and parking areas for this information collection. At the start of each information collection day, the first visitor that arrives at the survey intercept location at the end of their visit to the site will be asked if they are willing to participate and informed that their participation is voluntary and responses are anonymous.
Respondents who are contacted for the information collection will be read the following script:
“Hello, my name is _________. I am conducting a survey for the US Forest Service to better understand your attitudes toward transportation and visitor use in this area of the Arapaho Roosevelt National Forest. Participation is voluntary and all the responses are anonymous. Would you be willing to spend a few minutes to answer some important questions regarding your visit here? This will only take about 10 minutes.”
If
“YES” to introductory script then, “Thank you,
has any member of your group (friends and/or family with whom you
are here today) participated in this survey before?”
If “YES” then, “Thank you for participating in this study but you have already provided us with the information we need. Have a great day.”
If “NO” then, “Thank you for agreeing to participate in this study. The results from this survey will help the Forest Service improve transportation, recreation experiences, and resource management in this area. If you have any questions about the survey instructions while you are completing the questionnaire, please let me know, and I’ll be happy to answer your questions.”
If
“NO” to introductory script then, “I understand.
In that case, would you mind answering just three short questions;
it should take less than a minute of your time?”
If “YES” to non-response question recruitment script then:
What was your primary activity during your visit to [STUDY SITE] today?
What was the primary location or destination you visited today?
Did you have difficulty finding a place to park here today?
If
“NO” to non-response question recruitment script then,
“Thank you, I hope you enjoyed your visit.”
If the contacted visitor does not wish to participate, he/she will be thanked for their consideration. If the contacted visitor agrees to participate and is 18 years of age or older, the survey administrator staff person will administer a survey instrument to the respondent and instruct him/her to complete it onsite. The “ARNF Visitor Survey” will be administered to visitors at trailheads and at primary parking areas.
For each visitor contacted, whether they agree to participate in the study or not, the survey administrator will record an entry on a survey log form to track survey response rates and collect observable information about contacted visitors to use for statistical tests of non-response bias in the survey response data.
After completing the visitor contact, the survey administrator staff person will then contact the next visitor that arrives at the intercept location, and the process will be repeated throughout the sampling day. Sampling days will be stratified by day of the week (weekend days versus weekdays) and time of day (early morning through mid-afternoon versus late morning to early evening) to assess day of week and time of day effects on visitor response.
This sampling procedure is based on proven successful methods applied in numerous national forest recreation areas and national parks. Moreover, there is an abundance of peer reviewed, scientific literature documenting the successful collection of this type of information from recreation visitors. Typically, response rates are high (70 to 95 percent), and respondent burden is relatively low; in fact, many recreation visitors who respond to surveys of this type express gratitude and appreciation for the opportunity to participate in studies to help protect and improve national forest recreation areas and national parks.
As noted, the survey instruments will be administered as onsite intercept surveys to visitors as they are completing their trips in the study areas. There are several advantages to this sampling approach. For example, onsite contact with visitors generally increases survey response rates and delivering instructions directly to respondents, rather than with written instructions, helps reduce respondent confusion and burden. Another advantage of the onsite intercept approach is that asking visitors about their trip while they are still onsite and just after completing their visit minimizes “recall bias” in survey response, and allows visitors to reflect on their responses while they are still in the immediate vicinity of the study area.
The survey instruments will be printed with the following important OMB information: 1) the OMB clearance number and expiration date clearly visible on the cover; 2) information regarding the burden hours associated with responding to this information collection and the USDA nondiscrimination statement; and 3) a clear explanation that all responses are voluntary and anonymous, and that names and addresses will not be able to be connected to any information study participants provide in response to the survey.
A trial period of survey administration will be conducted prior to actual survey administration, to identify any unanticipated logistical or other issues, and maximize data quality. In particular, a one-day trial period of survey administration will be conducted at each study site and sampling location. After each trial-period respondent completes the questionnaire, the survey administrators will conduct a debriefing interview with them. During the debriefing interviews, respondents will be asked to indicate if they experienced any confusion with questions, question wording, survey instructions, and/or survey format (e.g., skip patterns, multi-item response scales). In addition, the survey administrators will note the number of completes per hour, which will be used to determine if stratification by time of day is necessary.
Measurement Instrument:
As noted, the study includes one survey, entitled “ARNF Visitor Survey”, and includes a separate version of the survey, adapted to each of the three study areas, plus a version adapted to the IPW in BLRA, for a total of 4 survey instruments (Instrument A through Instrument D). Instrument A will be administered at BLRA; Instrument B will be administered at GP; Instrument C will be administered at MERA, and Instrument D will be administered at the IPW. The survey instruments are described below:
Instrument A: The survey instrument will be administered onsite and in hard-copy. The survey instrument has four sections, with question wording and response categories tailored to the specifics of BLRA. The first section of the survey instrument, entitled “Trip Description”, includes questions concerning respondents’ group sizes, the presence or absence of children under the age of 16 in respondents’ groups, time of arrival to the recreation area, locations visited in the recreation area, and activities engaged in during their visit. The second section of the survey instrument, entitled “Travel and Parking”, includes questions concerning visitors’ routes of travel to the recreation area, the number of vehicles in which visitor groups traveled to the recreation area, the location(s) and perceptions about where visitors parked their vehicle(s), the potential effects of advanced information on visitors’ parking and transportation mode choice decisions, and visitors’ attitudes about potential actions to manage parking congestion during peak visitation periods. The third section of the survey instrument, entitled “Planning your Trip to BLRA”, includes questions about when visitors decided to take the current trip to BLRA, information they used to plan their trip, and the potential effects of advanced information about crowding and transportation conditions on their trip planning. The fourth section of each survey instrument, entitled “Background Information”, includes questions concerning respondents’ gender, age, state or country of residence, level of formal education, ethnicity, and race.
Instrument B and Instrument C: Each of the survey instruments will be administered onsite and in hard-copy. The survey instruments have five sections, with question wording and response categories tailored to the specifics of GP and ME (for Instrument B and Instrument C, respectively). The first section of the survey instruments, entitled “Trip Description”, includes questions concerning respondents’ group sizes, the presence or absence of children under the age of 16 in respondents’ groups, activities engaged in during their visit, and locations visited in the recreation area. The second section of the survey instruments, entitled “Hike to Mt. Bierstadt Summit” (Instrument B) or “Walk/Hike to Mt. Evans Summit (Instrument C) includes questions for those visitors who hiked part or all of the way to the Mt. Bierstadt (Instrument B) or Mt. Evans (Instrument C) summit on the day they were contacted for the survey, and ask about visitors’ perceptions of crowding on the trail and at the mountain summit, and attitudes about potential management actions to prevent crowding. The third section of the survey instruments, entitled “Travel and Parking”, includes questions concerning visitors’ routes of travel to the recreation area, the number of vehicles in which visitor groups traveled to the recreation area, visitors’ time of arrival, the location(s) and perceptions about where visitors parked their vehicle(s), visitors’ perceptions concerning vehicle traffic, the potential effects of advanced information on visitors’ parking and transportation mode choice decisions, and visitors’ attitudes about potential actions to manage parking congestion during peak visitation periods. The fourth section of the survey instrument, entitled “Planning Your Trip to Guanella Pass” (Instrument B) or “Planning Your Trip to MERA” (Instrument C), includes questions about when visitors decided to take a trip to the recreation area, information they used to plan their trip, and the potential effects of advanced information about crowding and transportation conditions on their trip planning. The fifth section of each survey instrument, entitled “Background Information”, includes questions concerning respondents’ gender, age, state or country of residence, level of formal education, ethnicity, and race.
Instrument D: The survey instrument will be administered onsite and in hard-copy. The survey instrument has four sections, with question wording and response categories tailored to the specifics of the IPW. The first section of the survey instrument, entitled “Trip Description”, includes questions concerning respondents’ group sizes, the presence or absence of children under the age of 16 in respondents’ groups, length of stay, hiking route and schedule, perceptions of crowding while hiking, and attitudes about potential management actions to prevent crowding. The second section of the survey instrument, entitled “Travel and Parking”, includes questions concerning visitors’ routes of travel to the recreation area, the number of vehicles in which visitor groups traveled to the recreation area, the location(s) and perceptions about where visitors parked their vehicle(s), the potential effects of advanced information on visitors’ parking and transportation mode choice decisions, and visitors’ attitudes about potential actions to manage parking congestion during peak visitation periods. The third section of the survey instrument, entitled “Planning Your Trip to the IPW”, includes questions about when visitors decided to take a trip to the recreation area, information they used to plan their trip, and the potential effects of advanced information about crowding and transportation conditions on their trip planning. The fourth section of each survey instrument, entitled “Background Information”, includes questions concerning respondents’ gender, age, state or country of residence, level of formal education, ethnicity, and race.
It should be noted, the survey instruments include questions about trip origin (i.e., starting point), from which it is possible to derive estimates of mileage. Currently, it is not possible to access any of the study sites via public transportation, thus, there aren’t questions asking if respondents used public transportation to travel to the study sites. However, some questions ask respondents if they would be likely to use public transit for travel to and/or within the study sites, if it was made available in the future. Such transit options are not currently available; thus, there are no questions asking if visitors used transit on their current trip.
Topic Area labels were added to each question in the questionnaires to indicate the topic area of the question and corresponding project/measurement objective using the following Topic Areas and project objectives:
Topic Area 1-Visitor and Trip Characteristics: Questions in Topic Area 1 address the project’s objectives to understand visitors’ personal and group characteristics and visitors’ trip characteristics, including trip planning and travel to and within the study sites.
Topic Area 2-Perceptions and Evaluations of Transportation-related Conditions: Questions in Topic Area 2 address the project’s objective to understand visitors’ perceptions of and tolerances for transportation-related impacts (e.g., traffic congestion, parking shortages) at the study sites.
Topic Area 3-Crowding and Visitor Experience Quality: Questions in Topic Area 3 address the project’s objective to understand visitors’ perceptions of and tolerances for crowding-related impacts to visitor experience quality at the study sites.
Topic Area 4-Attitudes and Opinions about Services, Facilities, and Management: Questions in Topic Area 4 address the project’s objectives to understand visitors’ opinions about the acceptability of actions to: 1) improve transportation facilities, services, and conditions at the study sites; and 2) help mitigate transportation- and visitor experience-related impacts at the study sites.
Topic Area 5-Transportation-related Preferences: Questions in Topic Area 5 address the project’s objective to understand visitors’ preferences for existing and potential transportation systems, services (including advanced traveler information for trip planning), and facilities.
This information about people’s perceptions of and tolerances for impacts (traffic congestion, parking shortages, crowding, etc.), along with their opinions about the acceptability of actions to help mitigate issues, is critical information for identifying publicly acceptable strategies and actions for improving transportation and managing visitor use at the study sites. There are many examples of previous, peer-reviewed studies to support this conclusion, including:
Pettengill, P., Lee, B., and Manning, R. 2012. Traveler Perspective of Greenway Quality in Northern New England. Transportation Research Record. 2314, pp:31-40.
Pettengill, P., Manning, R., Anderson, L., Valliere, W., and Reigner, N. 2012. Measuring and Managing the Quality of Transportation at Acadia National Park. Journal of Park and Recreation Administration. 30(1)pp:68-84.
Lawson, S., Chamberlin, R., Choi, J., Swanson, B., Kiser, B., Newman, P., Monz, C., Pettebone, D., and Gamble, L. (2011). Modeling the effects of shuttle service on transportation system performance and visitor experience quality in Rocky Mountain National Park. Transportation Research Record, No. 2244, 97-106.
Pettebone, D., Newman, P., Lawson, S., Hunt, L., Monz, C., & Zwiefka, J. (2011). Estimating visitors' travel mode choices along the Bear Lake Road in Rocky Mountain National Park. Journal of Transport Geography, 19(1), 1210-1221.
White, D. D., Aquino, J. F., Budruk, M., & Golub, A. (2011) Visitors’ experiences of traditional and alternative transportation in Yosemite National Park. Journal of Park and Recreation Administration, 29(1), 38-57.
Manning, R. and J. Hallo. 2010. The Denali Park Road Experience: Indicators and Standards of Quality. Park Science 27(2): 33-41.
Holly, F. M.*, Hallo, J. C., Baldwin, E. D., & Mainella, F. P. (2010). Incentives and disincentives for day visitors to park and ride public transportation at Acadia National Park. Journal of Park and Recreation Administration, 28(2), 74-93.
Hallo, J., and R. Manning. 2009. Analysis of Social Carrying Capacity of a National Park Scenic Road. International Journal of Sustainable Transportation, 4:75-94.
Hallo, J. and R. Manning. 2009. Transportation and Recreation: A Case Study of Visitors Driving for Pleasure at Acadia National Park. Journal of Transport Geography, 17:491-499.
Lawson, S., Newman, P., Choi, J., Pettebone, D.*, & Meldrum, B. (2009). The numbers game: Integrated transportation and user capacity research in Yosemite National Park. Transportation Research Record, No. 2119,83-91.
Data Management:
All data will be stored in electronic and hardcopy, and archived according to established data management procedures required by the federal government. The project manager will verify the quality of questionnaire electronic data entry. Upon study completion, the survey data collected in this study will be available from the United States Forest Service (USFS) in a suitable electronic format, along with proper documentation.
Analysis of Study Results:
Instruments A-D: Analysis of the quantitative survey data collected in this study will use standard methods for survey research in parks and recreation settings (Vaske, 2008). Key estimates from the data will be descriptive in nature, primarily measures of central tendency (mean and median), dispersion (standard deviation), and frequency distributions. Some tests for differences in means and proportions by various sub-groups are expected.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
Expectations are that response to the survey will be very high. As noted, past response rate examples for nearly identical visitor surveys conducted by the principal investigator include Isle Royale National Park (100% response), Yosemite National Park (ranging from 81% to 98% response, depending on sampling site in the park), Rocky Mountain National Park (73% response), and Mount Rainier National Park (68% response); OMB control numbers for these previous studies are as follows: Isle Royale National Park (OMB #: 1024-0224); Yosemite National Park (OMB #: 1029-0220 and OMB #: 1024-0224); Rocky Mountain National Park (OMB #: 1024-0224; NPS #: 08-028); Mount Rainier National Park (OMB #: 1024-0224). Moreover, in both the current study and the previous studies noted, data collection was not conducted by federal employees, rather the federal government contracted out the data collection. Thus, the connection between response rates and the perception of who is conducting the study should be no different in this study than in the previous studies noted.
The script used by the survey administrator staff to recruit participants for this study is adapted from the script used at the previous studies with high response rates listed above, and is designed to help maximize response rate by emphasizing the importance of each respondent’s answers to the questions to help protect and enhance the national forest recreation area where they were contacted. As noted, visitors contacted in similar previous studies have often expressed gratitude and appreciation for the opportunity to participate in studies of this nature.
However, some visitors who are asked to participate in the study will decline to do so. Thus, measures will be taken in this study to assess whether the accuracy of the survey data are affected by non-response bias, and if so, to correct for non-response bias effects on the data. In particular, to track visitor survey response rates, survey administrators will record a survey log entry for each visitor group asked to participate in the survey; a copy of the survey log instrument is included with the submission. Information recorded on the survey log for each contacted group will include: 1) current time; 2) visitor group size; 3) presence or absence of children in the group; 4) whether the group accepted or refused to participate; 5) the survey ID number for those groups who participated; and 6) current weather conditions. In addition, all who decline to complete the questionnaire will be asked if they could answer just a few questions that would take less than a minute; if they say no to this, they’ll be thanked and sent on their way (refer to Section 2 of this document for the script that will be used for this purpose). Those who agree to answer the “non-response questions” will be asked:
What was your primary activity during your visit to [STUDY SITE] today?
What was the primary location or destination you visited today?
Did you have difficulty finding a place to park here today?
There will be a field on the survey response log to distinguish between visitor who decline to participate in the survey but respond to the three non-response questions and those visitors who refuse to answer any question. That being said, all who decline to complete the questionnaire, whether they agree or not to answer the three short “non-response questions” will be treated as refusals in computation of response rates. The survey log data and responses to the non-response questions will be used to test for differences between visitor groups who participated in the survey and those who did not, based on the group and trip characteristics measures noted. In particular, chi-square tests of independence and an independent samples t-tests of means will be used for statistical comparisons of respondents and non-respondents. If statistical differences are found, data will be weighted using algorithms available in conventional statistical software packages for social science (e.g., SPSS) such that results are generalizable to the intended target populations noted.
Results of the non-response analysis will be reported and the implications for management discussed. In particular, information will be reported about subgroups of visitors to whom the results may not be generalizable, if non-response bias is detected. In addition, if the results of the non-response analysis suggest there is potential non-response bias in the survey data, additional statistical tests will be performed to assess whether there are non-response bias effects on results for key questions in the survey instruments (e.g., questions about visitor experience quality, mode choice, travel and parking issues). In cases where there are non-response bias effects on the results for key questions, standard practices for weighting survey response data will be used to account for non-response bias effects.
Thus, measures to test for and correct non-response bias issues, coupled with the accuracy and statistical power associated with the projected sample size for each survey instrument (300 completed questionnaires), are expected to result in levels of accuracy and reliability that are generally accepted as sufficient in peer-reviewed social science quantitative study findings.
The chief statistical consultants for this study will be Dr. Steven R. Lawson and Jeffery Dumont, Resource Systems Group, Inc.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The survey methods and instruments for this study were reviewed by project managers at the Arapaho-Roosevelt National Forest and Federal Highway Administration, Central Federal Lands Division, and by cooperating university faculty and scientists at Colorado State University and Utah State University. Further, the questions in this survey are similar to those used in previous studies at several other national forest recreation areas and national parks that were reviewed and approved by the Office of Management and Budget as noted earlier in this document.
Moreover, and as noted, pre-testing and consultation were conducted with 5 volunteer participants employed by the USFS contractor, and with no specific background or training in survey research methods or analysis (i.e., representative of the general public, rather than survey experts). In particular, the individuals were asked to complete the questionnaire, and asked a series of debriefing questions after to elicit their feedback on the practical utility of the study, questionnaire/respondent burden, quality and clarity of the questionnaires and instructions, and ways to minimize respondent burden. Participants were also asked to indicate if they had any difficulty or confusion with skip patterns, multi-item response scales, and/or instructions for recording responses (e.g., “Check one box” or “Check all that apply”).
The feedback from the pre-test participants was unanimously very positive. Participants mentioned that it was reassuring to have survey administrators nearby, but didn’t feel the need to consult them. Participants also indicated that the layout of the questionnaires, and question wording were straightforward, all of which helped to minimize respondent burden. Participants reported that they had no trouble with skip patterns, multi-item response scales, or other instructions for recording responses. In fact, one participant reported, “The skip pattern was very good. I really liked the skip pattern.”
Participants felt that the study had important practical utility, with several comments emphasizing the importance of the information collected in the survey to ensure that people who have spent a great deal of time and planning to visit the USFS recreation areas are not disappointed. Other comments had to do with the importance of the information for planning how to provide information to visitors about parking, visitor use management and other actions to minimize impacts to the quality of visitors’ experiences at the study sites due to unexpected circumstances, conditions, or policies.
The time it took each respondent to complete the questionnaire was recorded by the pre-test administrators, and ranged from 8 minutes to 12 minutes, with an average completion time of 10 minutes. This finding helps to validate the burden estimates reported in the submission, and suggests that participation in the study does not cause undue/excessive respondent burden. Finally, the completed questionnaires were inspected by the pre-test administrators, after the pre-test was concluded. Inspection of the completed questionnaires indicated that respondents followed skip patterns correctly, answered all of the relevant questions, and recorded their answers correctly.
Participants in the pre-test offered minor suggestions to improve the wording or format of specific questions in the survey instruments, and, as described in detail in Supporting Statement A, revisions to the questionnaires were made accordingly.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Carol Kruse (Agency Project Manager)
Arapaho-Roosevelt National Forest
USDA Forest Service
2150
Centre Ave, Bldg. E
Fort
Collins, CO 80526-8119
970-295-6663
970-295-6696
fax
[email protected]
Dr. Steven R. Lawson (Principal Investigator, responsible for data collection and analysis)
Director, Resource Systems Group, Inc.
55 Railroad Row
White River Junction, VT 05001
802-295-4999
707-826-4145 fax
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT FOR APPLICATION OF |
| Author | USDA FOREST SERVICE |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |