06/07/07
Supporting Statement A
Request for OMB Review and Approval
Revision
June 7, 2007
STUDY TITLE
Survey of Illness and Injury among Backcountry Users in Yellowstone National Park
(PRA 0920-0727).
CDC PROJECT OFFICER
Sharon Roy, MD, MPH
Water and Environment Activity
Division of Parasitic Diseases
Centers for Disease Control and Prevention
4770 Buford Highway, N.E., Mailstop F22
Atlanta, GA 30341-3724
Tel: 770-488-4412
Fax: 770-488-7761
Email: [email protected]
NATIONAL PARK SERVICE LIAISON
LCDR David Wong, MD
United States Public Health Service
Medical Epidemiologist
National Park Service
Office of Public Health
1201 Eye Street NW
11th Floor, Room 1135
Washington, DC 20005
Tel: 202-513-7160
Fax: 202-371-1349
Email: [email protected]
A. JUSTIFICATION
Circumstances Making the Collection of Information Necessary
This information collection request builds upon the results of the OMB-approved pilot study of illness and injury among backcountry users in Sequoia and Kings Canyon National Parks in September 2006 and now seeks approval for a large-scale study in Yellowstone National Park.
Backcountry travel in the United States is an increasingly popular activity. The backcountry is defined as those primitive or wilderness areas that lack most facilities and services, and that are reached primarily by hiking, boating, or horseback. In general, backcountry users must bring in their own supplies (such as shelter, food, water, or water treatment supplies). In 2004, an estimated 12% of Americans age 16 years and older (about 26 million persons) went backpacking for one or more nights in backcountry areas during the previous 12 months.1 This was a 170% increase from the approximately 8% (15 million) who went backpacking in 1994-95.2 Similarly in 2004, an estimated 15% of Americans age 16 years and older (about 33 million persons) camped in primitive settings that usually lacked restrooms, hookups, and most facilities and services.1 This was a 118% increase from the 14% (28 million) who went primitive camping in 1994-95 and a 184% increase from the 10% (18 million) who went primitive camping in 1982-83.2 While people travel in the backcountry in many locations and on both private and public lands, many travelers hike, backpack, and camp in the backcountry in national parks. In 2006, there were more than 272 million recreational visits to national parks and more than 1.6 million overnight stays in the backcountry.3 Studies have indicated that as many as 56% to 94% of long-distance hikers and backpackers experience illnesses or injuries during their time in the backcountry.4-9 The high burden of illness and injury among backcountry users on extended trips suggests that there might also be a health burden for persons on shorter trips. This burden could have significant medical and economic implications, given the increasing popularity of backcountry use.
Currently, the National Park Service (NPS) provides information about health, safety, and sanitation to all backcountry users in national parks. This information is offered through face-to-face discussions, videos, and reading materials.10-11 Every park visitor is required to be familiar with this information before entering the backcountry. However, the advice NPS currently provides is general in nature and based only on standard disease prevention principles and a few studies. Little is known about the risk factors for illness and injury in the backcountry and about the health outcomes of visitors who use the backcountry areas of our nation’s parks. There have only been four small cohort studies (described in six papers) 4-9 that provide information on risk factors for illness and injury among backcountry users in the United States (see Section A.4). These studies were limited by small sample sizes (N = 1558-3437) and insufficient statistical power to detect some common risk factors. For example, none of the studies could differentiate the risks associated with disease transmission through food versus water versus person-to-person contact. Furthermore, these studies were limited to persons who spent several consecutive days in backcountry areas and did not consider persons traveling in the backcountry for shorter periods of time. Therefore, further information is required to better define the risks associated with backcountry travel in order to develop evidence-based guidelines and recommendations that can be disseminated in public health messaging. It is for this reason that NPS and CDC are partnering to conduct this study in a national park.
We know that the burden of illness and injury in a single park is not generalizable to other national parks or other backcountry areas. However, many of the issues experienced by backcountry users are similar in every park (e.g., backcountry water treatment and food preparation, backcountry hygiene and sanitary practices). Therefore, the lessons learned about the risk factors for illness and injury among backcountry users in Yellowstone National Park will be relevant to backcountry users in other parks as well. Yellowstone National Park is an appropriate site for this study because:
The park has a long history of research. Therefore, the employees are supportive of research activities and are familiar with the use of questionnaires.
The park has the infrastructure and resources to support our study (e.g., an electronic database of backcountry permit issuances).
The park has a large number of backcountry visitors each year. This large number is required to support the number of participants required in each arm of our study (see Section B.1).
This NPS and CDC collaborative study is consistent with the mission of both agencies. NPS Office of Public Health is charged through Directors Order 83 (Appendix A, Section III - General Policy, Subsection A)12 and through the Inter-Agency Agreement between NPS and the Public Health Service with assisting park managers to protect the health of approximately 272 million annual visitors.3 Directors Order 83, which is official policy of the agency, states:
“It is the policy of the NPS to protect the health and well-being of NPS employees and park visitors through the elimination or control of disease agents and the various modes of their transmission to man…”
In order to accomplish this mission, it is essential to detect and understand disease transmission issues and use this knowledge to strengthen management decisions that affect visitor safety.
Similarly, this data collection will also further CDC’s goals and fit with CDC’s research agenda. Under Section 241 of United States Code Title 42 (Appendix B, Subsection A – Authority of Secretary),13 CDC has the mandate to cooperate with and assist other public authorities and scientists in the conduct of studies related to the causes, control, and prevention of disease, including water purification, sewage treatment, and pollution of lakes and streams. This regulation provides an impetus for CDC to assist NPS in conducting this study. Furthermore, strengthening backcountry health and sanitation recommendations based on the information collected during this study will help CDC pursue its goals of achieving optimal life spans for persons at every stage of life and protecting persons from infectious and environmental threats. Evidence-based health messaging on the safe and healthy use of backcountry areas will help protect persons engaged in this increasingly popular form of exercise and recreation, which conforms to a healthy lifestyle.
2. Purpose and Use of the Information Collection
The purpose of this study is to assess the burden of illness and injury among backcountry users in Yellowstone National Park and to investigate behavioral and environmental risk factors that may be associated with such illness and injury. The health burden and risk factor information generated by this study will be used by NPS and CDC to improve and formulate evidence-based guidelines and health messaging for the prevention of illness and injury among backcountry users in national parks. In 2007, the NPS budget for visitor health and safety was $16,983,000. The FY 2008 budget request is expected to be approximately $18.2 million.14 Therefore, the data collection undertaken with this study and subsequent evidence-based heath messaging will support this federally-funded aspect of the NPS program.
Based on Yellowstone National Park utilization statistics from 2006, we estimate that 12,673 persons of all ages traveled into the Yellowstone backcountry on private permits from May 1, 2006, through Oct. 31, 2006 (considered the backcountry season at this park) and we expect a similar number to do so for the 2008 backcountry season. The previously mentioned cohort studies4-9 indicate that as many as 56%5 to 94%7 of long-distance hikers and backpackers experienced illnesses or injuries during their time in the backcountry. Not all national park visitors travel into the backcountry for extended periods of time like the participants in these studies. Nevertheless, these studies suggest that as many as 7,000 persons may have suffered from an illness or injury during the backcountry season at Yellowstone National Park in 2006. With more than 1.6 million overnight stays in the backcountry at all national parks in 2006,3 the national incidence of illness and injury each year in the backcountry may be very high, in spite of the current general health and safety advice NPS offers to park visitors. Without further study of the risk factors for illness and injury in the backcountry, refined evidence-based illness and injury prevention recommendations will not be available to help reduce this public health burden. Furthermore, some outdoor use groups have recently questioned some of the existing general advice, such as the universal need for careful filtration and disinfection of backcountry drinking water to prevent disease. The health consequences of ignoring the standard advice are not well defined and the existing studies provide little insight into the etiology of illness and injury in this setting. Therefore, without further information to better define the risks associated with backcountry travel, even the general advice may be ignored, potentially further jeopardizing the health and safety of hundreds of thousands of backcountry users.
In the Yellowstone study, we anticipate at least 10 times the number of participants of any previous cohort study, 4-9 which will allow us to evaluate illness and injury risks associated with specific activities, choices, and behaviors of backcountry visitors, such as water purification, sanitation practices, and hygiene. We will also be able to differentiate the risks associated with different disease transmission pathways (e.g., foodborne vs. waterborne vs. person-to-person transmission) (see Section B.1 for power calculations). Because of the large anticipated sample size/power, the inclusion of short-term and long-term backcountry users, and the detail of the questionnaire, this study will be a definitive study on the topic of backcountry illness and injury. With a better understanding of injury risks, disease transmission pathways, and the interactions of agent, environment, and host, NPS and CDC will be able to improve and strengthen evidence-based guidelines for backcountry health and sanitation practices and address many of the questions raised by outdoor users and public health officials.
Sequoia and Kings Canyon National Parks (SEKI) Pilot Study
In 2006, CDC and NPS submitted an information collection request to OMB for a large-scale study of backcountry illness and injury in Yellowstone National Park during the 2006 backcountry season. In August 2006, OMB gave permission for a small pilot study to assess the study methodology, logistics, response rates, and usefulness of the results for a larger-scale study (see Appendix C for the pilot study methodology). Due to the lateness in the backcountry season when approval for the pilot study was received, Yellowstone National Park was not willing to undertake the pilot study. Therefore, at the last minute, the pilot study was switched to Sequoia and Kings Canyon National Parks (SEKI) and was conducted in September 2006 at one of the NPS backcountry permitting offices. However, due to staffing and budget constraints, SEKI is unable to host a large-scale study so, again, we propose to conduct the large-scale study in Yellowstone National Park. The following is a discussion of the pilot study logistical findings. Note—while the questionnaire used in SEKI is the same as the proposed questionnaire to be used in Yellowstone National Park, the sample size of the pilot study was too small (N=43) to make analysis of the data on illnesses, injuries, and risk factors statistically meaningful. Therefore, this health and risk factor information was not analyzed and cannot be generalized to the universe of the study. Further data collection in a large-scale study in Yellowstone National Park with a large sample size based on power calculations is required to formulate evidence-based health recommendations.
From September 9 to 28, 2006, rangers at one SEKI backcountry permitting office contacted 64 persons regarding participation in our study. By completing a SEKI consent-to-further-contact form (Appendix D), 43 (67%) persons consented to be contacted after their backcountry trips. Of the 43 persons who completed the consent-to-further-contact form, all (100%) requested to be contacted by email following the completion their backcountry travels. Emails were sent to these 43 persons requesting that they complete an Internet-based questionnaire. Twenty-six persons (41%, 26/64) completed the Internet-based questionnaire after two email reminders. Thirteen of the 17 non-responders also provided postal addresses and were sent a paper-based version of the questionnaire along with the second email reminder – none (0%) returned a paper-based questionnaire within the 8 week deadline from the expected backcountry travel completion date. After the 8 week deadline, multiple attempts were made to phone the seven non-responders who provided phone numbers. Three were contacted and said they had not received the emails; they had received the paper-based questionnaires but did not have time to complete them. The remaining four persons could not be reached. Three did not answer the phone after multiple attempts and one provided the incorrect phone number.
The 64 persons who were initially contacted can be divided into 3 groups. The persons in the first group are referred to as non-participants. These were the 21 persons who did not complete a SEKI consent-to-further-contact form. The only information we had concerning the non-participants came from observations made by the park rangers and recorded on the SEKI refusal log (Appendix E) (i.e., sex, number of males and females in the group, presence of children, reason for refusal) and from their SEKI backcountry permits. The persons in the second group are referred to as non-responders. These are the 17 persons who completed a consent-to-further-contact form but failed to complete a questionnaire. The persons in the third group are referred to as responders. These are the 26 persons who completed a consent-to-further-contact form and who also completed a questionnaire. The demographic and backcountry experience information from each group is presented in Table A.2 below. There are no statistically significant differences in demographics or backcountry experience between the three groups.
Table A.2 – Demographic and Experience Characteristics of the SEKI Pilot Study Backcountry Users*
Variable |
Non-Participants |
Non-Responders |
Responders |
Number of Persons |
21 |
17 |
26 |
Mean Nights in Backcountry |
3 |
4 |
3 |
Mean Group Size |
3 |
2 |
2 |
Sex |
76% male |
76% male |
62% male |
Traveled with Children? |
0% |
0% |
4% (1 child) |
Age Category |
100% adult |
100% adult |
100% adult |
Mean Age Group |
Not available |
45-54 years |
45-54 years |
Race |
Not available |
100% white |
100% white |
Ethnicity |
Not available |
100% non-Hispanic |
96% non-Hispanic |
Residence |
Not available |
70% Urban or Suburban |
81% Urban or Suburban |
Education |
Not available |
100% > High School |
100% >High School |
Mean Backcountry Trips |
Not available |
10-19 |
10-19 |
Mean Maximum Duration of Trips |
Not available |
1-2 weeks |
1-2 weeks |
* No statistically significant differences between the three groups for any of the variables.
Of note, all 64 of the backcountry users were adults, most were male, and only one person (a responder) was traveling with a child. The group size and number of nights spent in the backcountry were not statistically different between non-participants, non-responders, and responders. Data concerning the remaining demographic and backcountry experience variables were not available for the non-participants. Among the non-responders and responders, the average age group was 45-54 years, all were white, most were non-Hispanic, and all had higher education. The levels of backcountry experience with respect to the mean number of backcountry trips and mean maximum duration of backcountry trips were the same for non-responders and responders. The lack of differences between the non-participants, non-responders, and responders with respect to these variables suggests that non-response bias was not significant.
When we looked at the answers provided by the 26 responders who completed the questionnaire, we found that nine persons (35%) had developed an injury or illness while in the backcountry (some had developed more than one). These included 3 (12%) with a wound, 4 (15%) with a musculoskeletal strain, 5 (19%) with blisters on their feet, 1 (4%) with a cough, 1 (4%) with abdominal cramps, and 1 (4%) with constipation. While the sample size of the pilot study was too small to have statistical significance, these findings suggest that there may be a considerable burden of illness and injury associated with backcountry travel in SEKI and such a burden may also be present in other national parks. We also found that 23 responders (90%) drank water from a backcountry water source (including lakes, ponds, rivers, streams, and springs). Of these, 17 persons (74%) treated this water and six persons (26%) did not (these six persons did not include the persons who developed abdominal cramps or constipation). Again, while the sample size is too small to have statistical significance, these findings raise the question about the universal need for treatment of backcountry water supplies to prevent gastroenteritis. Therefore, the preliminary findings of the pilot study indicate there is a need to continue data collection to better define the burden of and risks for illness and injury in the backcountry and that a large-scale study of sufficient power in Yellowstone National Park will provide useful information upon which to base evidence-based backcountry health and sanitation recommendations.
The 41% (26/64) response rate obtained in the pilot study is higher than the response rates obtained by some large continuing health surveys that provide key national health statistics and by some NPS Social Science Program surveys. For example:
The Foodborne Diseases Active Surveillance Network (FoodNet), a collaborative project involving state health departments, has administered four 12-month cycles of a population-based telephone survey to determine the burden of diarrheal illnesses. The fifth cycle is currently underway. The overall FoodNet population telephone survey response rate was 33% in the last completed survey cycle (2002-2003).15
The seven NPS Visitor Survey Card Studies conducted from 1999-2005 using mail-back customer satisfaction cards each had response rates of 25%-26%.16-22
In spite of these low response rates, such surveys, and others, still provide valuable data to inform recommendations, policies, and programs and we believe the Yellowstone study will have similar value.
Several recent studies have presented strong cases that larger non-response rates, as seen in the examples above, do not necessarily indicate larger biases when the variables of interest (e.g., water and food handling, sanitation and hygiene practices in the backcountry) are unrelated to the factors that produce nonresponse.23-25 We believe that this is the case with the SEKI pilot study and the Yellowstone study. Additionally, the SEKI pilot study data suggest that non-response bias was not significant among the backcountry users, which further supports this assumption.
Therefore, we propose to conduct the Yellowstone study, even with a 41% response rate. In fact, we believe that the 41% response rate obtained in the pilot study is lower than the response rate we will achieve in Yellowstone because of the short period of time we had to shift the pilot from Yellowstone to SEKI and the minimal preparation we provided to SEKI staff due to the abrupt switch.
However, for the Yellowstone study, we propose to modify the non-response bias assessment we used in the SEKI pilot study. Logistical constraints associated with the information collection instruments used in SEKI and the availability of an alternate data source in Yellowstone compels us to modify the methodology. The SEKI park rangers who collected data on the non-participants using the refusal log commented that the refusal log was time-consuming and impractical when large numbers of persons were waiting for their permits. They requested that the refusal log not be used in the future because the burden of work was too great and it interfered with their efficiency in performing other job duties. The park rangers and backcountry users in SEKI also said that the demographic information collected on the consent-to-further-contact form was time-consuming to complete and that this was a barrier to completion of the form. Accordingly, the park rangers and backcountry users requested an abbreviated form that asks only for consent and contact information and not for demographic information and information on previous backcountry experience.
Because we plan to approach the entire cohort of backcountry users in Yellowstone National Park for the entire backcountry season (see Section B.1 for power calculations), we will have information on all backcountry users through all the permits issued during that time. Yellowstone uses a different permit26 (Appendix F) than SEKI and stores its data electronically whereas SEKI uses a paper-based system. Therefore, we will have access to group size, group affiliation if applicable (e.g., school, church, scouts), duration of backcountry travel, route taken in the backcountry, and method of travel (i.e., foot, animal, boat) for all backcountry users in Yellowstone and can perform tests for non-response bias on all non-participants, non-responders, and responders without the need for a refusal log or the additional variables on the consent-to-further-contact form. This reduction in burden on the public and the rangers should also improve response rates, since 72% of SEKI non-participants said they refused to participate because they did not have time to fill out the consent-to-further-contact form and 23% of the persons who consented to further contact failed to answer the demographic questions on the same form.
Use of Improved Information Technology and Burden Reduction
This study relies heavily on automated, electronic data collection techniques to reduce the burden of data collection on both the study participants and the investigators. Nevertheless, only the minimum information necessary will be collected to determine risk factors for illness and injury in the backcountry. We propose to use an Internet-based self-administered questionnaire. However, study participants are given the option of requesting a paper version of the same questionnaire if they wish. Unless a paper-based questionnaire is specifically requested by the study participant, the only paper data collection instrument will be a Yellowstone study consent-to-further-contact form (Appendix G) accompanied by a Yellowstone study introduction page (Appendix H) that each backcountry user receives during the in-person NPS backcountry registration process.
Use of Internet-based questionnaire technology will reduce the burden to study participants and allow for electronic data submission. The self-administered Internet-based questionnaire has built-in skip patterns and internal logic controls for efficiently routing the respondent to the relevant questions. While the paper-based questionnaire has the same questions and the same skip patterns as the Internet-based version, participants completing the paper-based questionnaire must manually follow these skip patterns, which involves more effort and time and may increase the risk for data entry errors. Additionally, the Internet-based questionnaire will employ a variety of prompts to encourage survey completion, whereas the paper-based questionnaire will have no such prompts. Pilot testing of the Internet-based questionnaire by 3 CDC staff members, 1 layperson, and 3 NPS staff members indicated that the average completion time was about 15 minutes versus 25 minutes for the paper-based questionnaire. The average completion time for the 16 responders who completed the Internet-based questionnaire in the SEKI pilot study in one sitting was 30 minutes (range 11-55 minutes). No data is available for the paper-based questionnaire but the assumption is that it would also take twice as long as the initial pilot testing, perhaps 50 minutes. Therefore, electronic data collection reduces the reporting burden for study participants.
Use of Internet-based data collection and electronic data management tools will also reduce the burden for investigators and allow study data to be stored and maintained electronically. Consent forms will be scanned directly into a CDC study-related database, eliminating the need for data entry by investigators. The software program controlling the database will automatically generate introductory and reminder emails for those persons consenting to participate electronically. The software will also identify persons consenting to complete the paper-based questionnaires and those who are late in returning their completed paper-based questionnaires. Data entry into the database is automatic for the Internet-based questionnaires, thereby eliminating the need for manual data entry and limiting data entry errors. Furthermore, the Internet-based questionnaire has data entry validation to limit data entry errors and reduce data cleaning efforts.
A secondary outcome of this study is the opportunity to use Internet survey methodologies. Presently, many of the population-based surveys and outbreak investigations are conducted by telephone, such as the FoodNet population surveys15. However, the FoodNet population telephone survey response rate has been declining over the past four survey cycles, from 71% in the 1996-1997 cycle27 to 33% in the 2002-2003 cycle.28 This is representative of an overall decline in survey response rates.23,29 In particular, telephone surveys in the United States have been affected by the increase in private telemarketing, and the introduction of “do-not-call” lists and caller screening devices.23 The growth in households using only cellular telephones (without a land line) may also be of concern. The reduction in response rates for telephone surveys is necessitating the investigation and use of new methodologies. One of these new methodologies is the Internet-based survey. Internet-based survey technology in the form of a software package called mrInterview has recently been introduced to CDC but experience with this package at CDC is limited to date.
Our study provides CDC with the opportunity to assess this software and the protocol for Internet survey use. Furthermore, this study will be the first use of Internet survey technology by the NPS Public Health Program and will provide NPS with the opportunity to evaluate a new methodology for reaching widely dispersed populations, such as national park visitors who return to their homes across the United States and the world. Therefore, this assessment will allow both agencies to incorporate technologically-improved respondent reporting and to inform the understanding and future use of this technology at CDC and NPS.
Efforts to Identify Duplication and Use of Similar Information
A literature review has revealed that, to date, there have only been four small cohort studies (described in six papers) 4-9 that provide information on risk factors for illness and injury among backcountry users in the United States. This finding has been supported by consultation with the NPS Office of Public Health and with the principal investigator (Dr. David Boulware) of three of the six papers.4-6
All of these studies had small sample sizes (range 1558 to 3437), asked a small number of questions, had insufficient statistical power to detect some common risk factors, and were limited to persons who spent at least several consecutive days in backcountry areas. Because of these constraints, these studies have provided limited information about the burden of illness and injury among backcountry users (both short-term users and long-term users) and the risk factors associated with these conditions. For example, these studies have not provided sufficient evidence to tease out the contribution of waterborne disease vs. foodborne disease vs. person-to-person spread of disease. As a result, NPS has had limited information on the risk factors for illness and injury in the backcountry and has, therefore, had to rely heavily on general public health principles to derive its current backcountry health and sanitation recommendations. A larger study of backcountry users, such as ours, will provide valuable data with which to formulate evidence-based recommendations.
Impact on Small Businesses or Other Small Entities
Not Applicable - No small businesses will be involved in this study.
Consequences of Collecting the Information Less Frequently
Backcountry activities are growing in popularity, with 1.6 million overnight stays in the backcountry of national parks in 2006.3 Limited data suggest a substantial burden of illness and injury among visitors who use the backcountry4-9 yet these data provide little insight into the etiology of illness and injury in this setting. Therefore, backcountry health and sanitation advice to park managers and the public is currently general in nature, based only on standard disease prevention principles and a few small studies. Without more information, NPS and CDC have limited capacity to develop effective evidence-based public health messages for backcountry users. Recently, some outdoor use groups have questioned some of this standard advice, such as the universal need for careful filtration and disinfection of backcountry drinking water. This study will provide an estimate of the burden of illness and injury among backcountry users as well as information about a variety of risk factors for illness and injury in the backcountry, including the risks associated with drinking untreated water from lakes and streams. Because of the large anticipated sample size/power, the inclusion of short-term and long-term backcountry users, and the detail of the questionnaire, this study will be a definitive study on the topic of backcountry illness and injury. With this information, we will be able to address many of the questions raised by outdoor users and public health officials, and improve and strengthen evidence-based guidelines for backcountry health and sanitation practices. Without such information and recommendations, the rates of illness and injury in the backcountry will remain high and Americans will continue to experience hundreds of thousands of cases of preventable illness and injury each year associated with backcountry use.
Participants in this study are asked to respond to the data collection one time only. There are no technical or legal obstacles to reduce the burden.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
Our request of study participants to provide information fully complies with all regulations.
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A. A 60-day Federal Register Notice was published in the Federal Register on October 28, 2005, Vol. 70, No. 208, pp. 62121-62122. (Appendix I). There were no public comments.
B. This study is a collaborative effort of CDC and NPS. Members of NPS, both within the Office of Public Health and Yellowstone National Park, have been consulted on all aspects of the study: logistics, visitor participation, availability of data on backcountry visitation and permits, backcountry permitting processes, questionnaire development, website development, consenting procedures, instructions for data collection, frequency of data collection, instructions to the public and staff, record keeping, reporting format, and the data elements to be recorded/disclosed/reported. No problems occurred that could not be resolved during consultation.
Consultation inside CDC began in 2005 with the following persons and is ongoing:
Michael J. Beach, Ph.D.
Acting Associate Director for Healthy Water
National Center for Zoonotic, Vector-Borne and Enteric Diseases (NCZVED)
&
Team Leader, Water and Environment Activity
Division of Parasitic Diseases
National Center for Zoonotic, Vector-Borne and Enteric Diseases (NCZVED)
Centers for Disease Control and Prevention
Mailstop F-22
4770 Buford Highway, NE
Atlanta, GA 30341-3724
Phone: (770) 488-7763
Email: [email protected]
Consultation outside the CDC (with NPS) began in 2005 with the following persons and is ongoing:
Capt. Charles L. Higgins
U.S. Public Health Service
Director, Office of Public Health
National Park Service
1849 C St. NW, (Org. Code 2480)
Washington, D.C. 20240
Phone: (202) 513-7217
Fax: (202) 371-1349
Email: [email protected]
Lt. George A. Larsen
U.S. Public Health Service
NPS Public Health Consultant
National Park Service
Business Management Office
P.O. Box 168
Yellowstone National Park, WY 82190
Phone: (307) 344-2273
Fax: (307) 344-2279
Email: [email protected]
Frank Walker
Deputy Superintendent – Yellowstone National Park
National Park Service
P.O. Box 168
Yellowstone National Park, WY 82190
Phone: (307) 344-2033
Fax: (307) 344-2005
Email: [email protected]
In 2005, consultation on study feasibility, logistics, questionnaire development, and consenting procedures also occurred with the principal investigator (Dr. David Boulware) of three of the six backcountry illness and injury papers previously cited.4-6
Dr. David Boulware
Infectious Diseases and International Medicine
University of Minnesota
Infectious Disease Office of
Medicine
MMC 250 Mayo 8250
420 Delaware St SE
Minneapolis,
MN 55455
Phone: (612) 624-9996
Email: [email protected]
Explanation of Any Payment or Gift to Respondents
Participation in this data collection is voluntary and respondents will not receive direct remuneration.
Assurance of Confidentiality Provided to Respondents
The proposed project has been reviewed by CDC’s Privacy Officer, who determined that the Privacy Act applies to this data collection. The applicable system of records is 09-20-0136, “Epidemiologic Studies and Surveillance of Disease Problems.” The Privacy Act applies because personally identifying information will be requested and could potentially be linked to survey responses, although its intended use is to facilitate follow-up contacts for (a) study consent, and (b) completion of the study questionnaire. Personal identifiers will be maintained in a confidential manner by study staff for approximately 2 to 8 months (from the start of the study in May until data collection is completed in December). Personal identifiers will then be destroyed.
To safeguard respondent privacy, the study team has put the following procedures in place:
Personal identification information (e.g., name, email address, street address, phone number) will not be collected from adults 18 years of age or older who decline to be contacted after their backcountry trips. The only data that will be collected from non-participating adults are their permit numbers. Permit numbers are required to assess response rates for the study and will be determined for non-participants through a comparison of the Yellowstone permit database with the permit numbers of those who consent to further contact.
Personal identification information (e.g., name, email address, street address, and phone number) will not be collected from children younger than 18 years of age since they will not be included in this study.
The consent-to-further-contact form, which contains personally identifying information such as full name, email address and/or mailing address, and phone number, is presented to backcountry users when they pick up their backcountry permits. For safety purposes, NPS requires backcountry users to pick up their permits in person at one of nine Yellowstone backcountry permitting offices as part of the backcountry check-in procedure. The consent-to-further-contact forms will be put in a secure box or location in the NPS backcountry permitting offices. These forms will be picked up weekly by NPS study staff and sent to CDC study staff by FedEx®. At CDC, the consent-to-further-contact forms will be kept in a locked office.
After the data collection is completed in December, all personal identifiers in the database will be de-linked from the questionnaire responses and all personal identification information (both Internet-based and paper-based) will be destroyed. Thus, permanent data will be anonymous. Respondents will not and cannot be contacted for further follow-up.
CDC and NPS do not plan to include participant names in internal reports or any publications based on this study. CDC and NPS plan to report only anonymous, aggregate data.
CDC and NPS investigators have no plans to share participant personal identification information and will keep individuals’ answers private and confidential to the extent allowed by law, but respondent privacy cannot be guaranteed without formal confidentiality protection. Study participants are made aware of these facts in the study consent information they read before they begin to fill out the questionnaire.
Access to the Internet-based questionnaire is obtained through the use of a personal pass code that is sent via email to each respondent who has consented to further contact following his/her backcountry trip. The personal pass code only allows the respondent to access his/her personal data.
Personal identification information and questionnaire answers (both Internet-based and paper-based) will be stored on a CDC password-protected computer server. Completed consent-to-further-contact forms and paper-based questionnaires will be kept in a locked office. Access to the electronic data and to the paper documents will only be granted to authorized personnel at CDC who are working on the study.
The measures to safeguard privacy are described to respondents in the informed consent process.
This study and the planned measures for safeguarding respondent privacy have been reviewed by CDC’s Human Research Protection Office, which determined that the study is exempt from requirements for IRB review and approval under 45 CFR 46.101(b)(2).30 (Appendix J) However, the study will be conducted in a manner consistent with best practices for research.
Justification for Sensitive Questions
Respondents will be asked for potentially sensitive information about medical history (presence of chronic diarrhea, Crohn’s Disease, irritable bowel syndrome, colitis, intestinal surgery, and immunosuppression; see Q#204). Because these conditions are associated with non-infectious diarrhea, questions about them are necessary to the scientific purposes and analysis of study data. Since the etiology of diarrhea among backcountry users during this study will not be laboratory confirmed, the inclusion of persons with pre-existing diarrheal conditions in the analysis may overestimate the burden of acute diarrheal illness associated with backcountry use. A subset analysis can be performed on persons with pre-existing diarrheal illness to see if they are at higher risk for diarrheal episodes in the backcountry. Similarly, questions concerning the presence of immunosuppression are also important for this study. Immunocompromised persons are at higher risk for developing many infectious diseases and experiencing more severe symptoms compared to non-immunocompromised persons. Therefore, including immunocompromised persons in the analysis may overestimate the burden and severity of acute infectious disease associated with backcountry use. A subset analysis can be performed on immunocompromised persons to determine specific risk factors for this group, which will help inform the development of evidence-based guidelines and health messaging for this special population. Potentially sensitive questions about health status are thus important to the study team’s ability to interpret the data and to tailor risk reduction recommendations to specific groups.
Respondents will also be asked two potentially sensitive questions about mental health (see Q#144 and Q#146). These questions will assess whether backcountry users experienced psychological or social conditions (e.g., loneliness, depression, or problems with other backcountry users), and if so, whether the conditions were associated with premature termination of a backcountry trip. Previous research conducted among Appalachian Trail hikers4 identified psychological conditions as more influential than illness in decisions to abandon a hike. Such findings, if confirmed in the proposed study, would have important implications for CDC and NPS in formulating appropriate, inclusive, evidence-based recommendations and health messages for backcountry users.
Finally, respondents will be asked about their race/ethnicity. This information is important in order to detect differences that may inform the development of evidence-based guidelines and health messaging targeted for minority populations.
Participation in the study is voluntary and respondents may stop at any time, or refuse to answer any question(s) that they do not wish to answer.
Estimates of Annualized Burden Hours and Costs
Based on Yellowstone National Park utilization statistics from 2006, we estimate that 12,673 persons of all ages will travel into the Yellowstone backcountry from May 1, 2008 through October 31, 2008. This study will be confined to adult respondents and all adult backcountry users will be screened for potential participation. Approximately 10,138 (80%) of the estimated 12,673 backcountry users will be adults who will be asked to participate and to complete a consent-to-further-contact form, and 2,535 (20%) will be children under the age of 18 years who will not be asked to participate or to complete a consent-to-further-contact form. Children will not be included in this study because (1) children are a minority of backcountry users, (2) we will obtain indirect information about children traveling into the backcountry from the questionnaire answers provided by the adult(s) traveling with them, and (3) Internet-based consent processes are not well established where children are involved as research participants (research consent for minors involves both the permission of a parent or guardian, as well as the assent of the minor). Our core participation and burden projections are thus based on the potential respondent universe of 10,138 adults. From this universe, we anticipate that 3,423 persons will complete Internet-based questionnaires and 109 persons will complete paper-based questionnaires, for a total of 3,532 completed questionnaires. For details on these calculations, please see Section B.1.
For each of the three forms (Yellowstone study consent-to-further-contact form, Internet-based questionnaire, and paper-based questionnaire), we estimate the following average burdens per response:
Consent-to-further-contact form: pilot testing at CDC and NPS by 3 CDC staff members, 1 layperson, and 3 NPS staff members indicated that the average completion time was 2 minutes or less.
Internet-based questionnaire: responders in the SEKI pilot study took an average of 30 minutes to complete the Internet-based questionnaire. This is double the time taken to complete the Internet-based questionnaire at CDC and NPS during pilot testing by 3 CDC staff members, 1 layperson, and 3 NPS staff members (average completion time was 15 minutes).
Paper-based questionnaire: no data are available from the SEKI pilot study. Pilot testing at CDC and NPS by 3 CDC staff members, 1 layperson, and 3 NPS staff members indicated that the average completion time was 25 minutes. Given that the SEKI pilot study completion time estimate for the Internet-based questionnaire was double that observed in pilot testing, we now estimate that the paper-based questionnaire will take about 50 minutes to complete.
Table A.12.A – Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Consent-to-Further- Contact Screening Form |
10,138 |
1 |
2/60 |
338 |
|
Consenting Eligible Adult Yellowstone Backcountry User |
Internet-based Questionnaire |
3,423 |
1 |
30/60 |
1,712 |
Paper-based Questionnaire |
109 |
1 |
50/60 |
91 |
|
Total |
2,141 |
B. A study in 2000 of backcountry campers at Yellowstone National Park31 estimated the annual household income of campers to be in the range of $40,000 to $59,000. Using $50,000 as an average estimate, and assuming a 40-hour workweek for 52 weeks per year, the estimated average hourly wage is $24. Therefore, the estimated annualized cost for all respondents for 2,141 total hours (see Table A.12.A above) is $51,366.00.
Table A.12.B – Estimated Annualized Burden Costs
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response (in hours) |
Average Hourly Wage Rate |
Total Respondent Costs |
Adult Applicant for Yellowstone Backcountry Permit |
Consent-to-Further- Contact Screening Form |
10,138 |
1 |
2/60 |
$24 |
$8,110 |
Consenting Eligible Adult Yellowstone Backcountry User |
Internet-based Questionnaire |
3,423 |
1 |
30/60 |
$24 |
$41,076 |
Paper-based Questionnaire |
109 |
1 |
50/60 |
$24 |
$2,180 |
|
Total |
$51,366 |
Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
There are no costs to respondents, other than their time (i.e., no capital or maintenance costs associated with completing the questionnaire).
Annualized Cost to the Federal Government
The costs incurred by the government for this one-time data collection include the costs for personnel time, travel, printing documents, mailing paper-based questionnaires, and publication charges. There are no equipment or hardware costs because all the equipment used is already in place at CDC and NPS and is used for other purposes (see Table A.14 on next page).
Table A.14 – Estimated Annualized Cost to the Federal Government
Resource |
Description/Role |
Unit Cost |
Unit |
Cost |
||
PERSONNEL |
||||||
CDC project officer |
Protocol & questionnaire development, OMB applications, travel to Yellowstone to set up logistics, data management, data analysis, manuscript development and publication (18% FTE) |
$55.77 per hour |
410 hours |
$22,866 |
||
NPS project officer |
Protocol & questionnaire development, field trip to set up logistics, consent form collection & mailing, data analysis, manuscript development and publication (18% FTE) |
$40.16 per hour |
176 hours |
$7,069 |
||
CDC IT contractor |
Internet questionnaire & database development, data management, data analysis, manuscript development |
$80 per hour |
160 hours |
$12,800 |
||
Seasonal NPS contractor |
Routine visits to backcountry permitting offices throughout the park to assist NPS project officer, visit offices, replenish supplies, facilitate collection and shipping of forms (GS-5, Step 1 with 7.65% social security & 25% Sundays) (Mon, Thurs, Fri, Sat, Sun) |
$15.70 per hour |
1039 hours |
$16,400 |
||
SUPPLIES & EQUIPMENT |
||||||
Paper-based questionnaires and consent forms |
Paper-based questionnaires (45 pages per questionnaire x 325 questionnaires), paper consent forms, and introduction pages (2 pages x 13.000) = 40,625 pages (82 reams) |
$4.00 per ream of paper (500 sheets) |
82 reams |
$330 |
||
Posters |
Posters to promote study participation that will be posted in each backcountry permitting office |
$30 |
9 posters |
$270 |
||
TRAVEL EXPENSES |
||||||
Travel to Yellowstone by CDC project officer |
5-day trip for CDC project officer to set up logistics |
round trip airfare & hotels for 40 hours |
1 trip |
$3,000 |
||
Vehicle costs in Yellowstone |
NPS contractor will need to visit all 9 backcountry permitting offices each week to pick up forms and assist with recruitment (400 miles per week x 22 weeks = 8,800 miles) |
$0.48 per mile |
8,800 miles |
$4,230 |
||
OTHER EXPENSES |
||||||
Postage |
Mailing paper-based questionnaires with postage-paid return envelopes |
$4.50 for postage; |
325 questionnaires |
$1,470 |
||
FedEx® |
Shipping 1 box of consent forms per week from Yellowstone to CDC |
$65 per FedEx® box |
22 boxes |
$1,430 |
||
Publication charges |
Publication of backcountry study paper |
Typical page charges $55 per page for first 6 pages; $85 per page thereafter |
|
$500 |
||
$70,365 |
Explanation for Program Changes or Adjustments
This is a new data collection. The SEKI pilot study was conducted in preparation for the large-scale study at Yellowstone National Park (see Section A.2).
Plans for Tabulation and Publication and Project Time Schedule
Table A.16 – Project Time Schedule
Activity |
Time Schedule |
Begin data collection by distributing consent forms and questionnaires to backcountry users |
May 1, 2008
The backcountry season at Yellowstone National Park is May 1 – October 31. Data collection MUST begin May 1 in order to capture a full backcountry season at Yellowstone National Park.
|
Stop distributing consent-to-further-contact forms |
Nov. 1, 2008
The last day of consent form distribution will be October 31 in order to capture a full backcountry season at Yellowstone National Park.
|
Stop data collection |
Dec. 27, 2008 |
Clean and validate data |
Dec. 28, 2008 – Feb. 28, 2009 |
Analyze data |
Mar. 1 – Aug. 15, 2009 |
Write manuscripts & NPS guidelines for backcountry users |
Aug. 16 – Nov. 16, 2009 |
Clear manuscripts & NPS guidelines for backcountry users |
Nov. 17, 2009 – Feb. 17, 2010 |
Submit manuscripts to journals and work on revisions |
Feb. 18 – June 18, 2010 |
Publish manuscripts & NPS guidelines for backcountry users |
Approximately September 2010 (determined by journal) |
This is a prospective cohort study. Data will be analyzed using SAS (SAS System for Windows, version 9.1; SAS Institute, Inc., Cary, NC) with multivariate conditional logistic regression based on the Cox proportional hazards model. Crude relative risks (RR), 95% confidence intervals (95%CI), and p-values (P) will be generated for each exposure variable (e.g., water consumption, water preparation habits, food consumption, food preparation habits, sanitation practices, recreational water use, and animal exposure). Incidence rates will be generated for each outcome variable (e.g., injuries incurred while in the backcountry and illness symptoms developing during and shortly after backcountry visits). Adjusted RRs will then be generated, controlling for demographics and health status. Variables with statistically significant (P < 0.05) crude RRs and/or those that have biologic plausibility or are known risk factors for backcountry illness and injury will be included in larger multivariate models. Interaction of variables will be explored and assessed in all multivariate models.
Illustrative table shells:
Incidence Calculations:
Symptom / Injury |
Date of illness/injury onset |
Number of cases |
|
|
|
|
|
|
Univariate Analyses:
Exposure Variable |
Stratification |
Ill/injured with exposure |
Well with exposure |
Ill/injured without exposure |
Well without exposure |
RR |
95% CI |
p-value |
Variable # 1 |
Unstratified |
|
|
|
|
|
|
|
|
Stratified by demographic variable # 1 |
|
|
|
|
|
|
|
|
Stratified by demographic variable # 2 |
|
|
|
|
|
|
|
|
Stratified by demographic variable # n |
|
|
|
|
|
|
|
Variable # 2 |
Unstratified |
|
|
|
|
|
|
|
|
Stratified by demographic variable # 1 |
|
|
|
|
|
|
|
|
Stratified by demographic variable # 2 |
|
|
|
|
|
|
|
|
Stratified by demographic variable # n |
|
|
|
|
|
|
|
Variable # n |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Analysis of data relating to education, income, and other demographics may also inform NPS/CDC efforts to improve public health messages and to target public health messages to specific audiences. A previous study31 has indicated that the Yellowstone backcountry-user population has, in general, a higher income and educational status than the general population, which may limit the generalizability of the study findings and must be accounted for if the same demographic patterns hold true for our study population.
Reason(s) Display of OMB Expiration Date is Inappropriate
Not Applicable – there will be no data collection for this study beyond Dec. 27, 2008.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
LIST OF ATTACHMENTS
Appendix A: NPS Directors Order 83
Appendix B: US Code Title 42 Public Health and Welfare
Appendix C: SEKI Pilot Study Methodology
Appendix D: SEKI Consent-to-Further-Contact Form
Appendix E: SEKI Refusal Log
Appendix F: Yellowstone Study Backcountry Permit
Appendix G: Yellowstone Study Consent-to-Further-Contact Form
Appendix H: Yellowstone Study Introduction Page
Appendix I: 60-Day Federal Register Notice
Appendix J: IRB Exemption Determination
Appendix K: Yellowstone Study Introductory Email
Appendix L: Yellowstone Study Consent – Internet Version
Appendix M: Yellowstone Study Questionnaire – Internet Version
Appendix N: Yellowstone Study Introductory Letter
Appendix O: Yellowstone Study Consent – Paper Version
Appendix P: Yellowstone Study Questionnaire – Paper Version
Appendix Q: NPS Study Website
Appendix R: CDC Study Website
REFERENCES
1. Cordell K, Green G, Betz C, Fly M, Stephens B. Recreation Statistics Update. Update Report No. 1, August, 2004 – Participation Rates for Outdoor Activities in 2004. Downloaded 04/24/2007. Available at: http://www.srs.fs.usda.gov/trends/RECUPDATES/recupdate0804.pdf
2. Cordell HK, McDonald BL, Teasley RJ, Bergstrom JC, Martin J, Bason J, Leeworthy VR. National Survey on Recreation and the Environment: Outdoor Recreation Participation Trends. Downloaded 04/24/2007. Available at: http://www.srs.fs.usda.gov/pubs/ja/ja_cordell010.pdf
3. Statistical Abstract 2006, National Park Service, Public Use Statistics Office. Downloaded 04/24/2007. Available at: http://www2.nature.nps.gov/stats/abst2006.pdf
4. Boulware DR, Forgey WW, Martin WJ. Medical Risks of Wilderness Hiking. Am J Med. 2003 Mar;114(4):288-93.
5. Boulware DR. Influence of hygiene on gastrointestinal illness among wilderness backpackers. J Travel Med. 2004 Jan-Feb;11(1):27-33.
6. Boulware DR. Gender differences among long-distance backpackers: A prospective study of women Appalachian Trail backpackers. Wilderness Environ Med. 2004 Fall;15(3):175-80.
7. Twombly SE, Schussman LC. Gender differences in illness and injury rates on wilderness backpacking trips. Wilderness Environ Med. 1995;4:363-76.
8. Gardner TB, Hill DR. Illness and injury among long-distance hikers on the Long Trail, Vermont. Wilderness Environ Med. 2002 Summer;13(2):131-4.
9. Crouse BJ, Josephs D. Health care needs of Appalachian trail hikers. J Fam Pract. 1993 May;36(5):521-5.
10. Backcountry Trip Planner – Yellowstone National Park. National Park Service. Downloaded 04/24/2007. Available at: http://www.nps.gov/yell/planyourvisit/backcountrytripplanner.htm
11. Beyond Road’s End: Regulations and Guidelines for Backcountry Travel in Yellowstone National Park. National Park Service. Downloaded 04/24/2007. Available at: http://www.nps.gov/yell/planyourvisit/upload/beyondroadsend.pdf
12. National Park Service Director’s Order #83: Public Health. Downloaded 04/24/2007. Available at: http://www.nps.gov/policy/DOrders/DOrder83.html
13. United States Code, Title 42, Chapter 6A, Subchapter II, Part A, Section 241 Research and Investigations Generally. Downloaded 04/24/2007. Available at: http://uscode.house.gov/download/pls/Title_42.txt
14. National Park Service Visitor Services Funding. Downloaded 04/24/2007. Available at: http://home.nps.gov/applications/budget2/documents/visitor_services_funding.pdf
15. Jones TF, McMillian MB, Scallan E, Frenzen PD, Cronquist AB, Thomas S, & Angulo FJ. A population-based estimate of the substantial burden of diarrheal disease in the United States; FoodNet, 1996-2003. Epidemiol Infect. 2007;135:293-301.
16. National Park Service. National Park System 1999 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/napa99.pdf
17. National Park Service. National Park System 2000 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/napa00.pdf
18. National Park Service. National Park System 2001 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/napa01.pdf
19. National Park Service. National Park System 2002 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/napa02.pdf
20. National Park Service. National Park System 2003 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/napa03.pdf
21. National Park Service. National Park System 2004 Visitor Survey Card Data Report. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/vsc_napa04.pdf
22. National Park Service. Serving the Visitor 2005: A Report on Visitors to the National Park System. Downloaded 04/27/2007. Available at: http://www.nature.nps.gov/socialscience/pdf/STV05.pdf
23. Tourangeau R. Survey research and societal change. Annu Rev Psychol. 2004; 55, 775-801.
24. Curtin R, Presser S, Singer E. 2000 The effects of response rate changes on the index of consumer sentiment. Public Opin Q. 2000;64(4):413-28.
25. Keeter S, Miller C, Kohut A, Groves RM, Presser S. Consequences of reducing nonresponse in a national telephone survey. Public Opin Q. 2000 ;64(2):125-48.
26. Backcountry Campsite Reservations – Yellowstone National Park. National Park Service. Downloaded 04/24/2007. Available at: http://www.nps.gov/yell/planyourvisit/upload/campsitereserv.pdf
27. Herikstad H, Yang S, Van Gilder TJ, Vugia D, Hadler J, Blake P, Deneen V, Shiferaw B, Angulo FJ, & the FoodNet Working Group. A population-based estimate of the burden of diarrheal illness in the United States: FoodNet, 1996-7. Epidemiol Infect. 2002;129(1), 9-17.
28. Jones TF, McMillian MB, Scallan E, Frenzen PD, Cronquist AB, Thomas S, & Angulo FJ. A population-based estimate of the substantial burden of diarrheal disease in the United States; FoodNet, 1996-2003. Epidemiol Infect (In press).
29. Atrostic BK, Bates N, Burt G, & Silberstein A. Nonresponse in U.S. government household surveys: consistent measures, recent trends, and new insights. J Official Statistics. 2001; 17(2), 209-26.
30. Department of Health and Human Services. Code of Federal Regulations: Title 45 Public Welfare, Part 46 Protection of Human Subjects. Downloaded 03/24/2006. Available at: http://www.hhs.gov/ohrp/humansubjects/guidance/46.101
31. Oosterhous TS. Attitudes, Opinions, Characteristics, and Management Preferences of Backcountry Campers in Yellowstone National Park. Masters thesis. Stephen F. Austin State University, Texas. May 2000.
32. Madden M. Internet penetration and impact April 2006. Pew Internet & American Life Project. Downloaded 05/01/2007. Available at: http://www.pewinternet.org/PPF/r/182/report_display.asp
Page
File Type | application/msword |
File Title | Supporting Statement |
Author | Sharon Roy |
Last Modified By | arp5 |
File Modified | 2007-06-12 |
File Created | 2007-06-07 |