Supporting_Statement_PartB 6-28-2011

Supporting Statement B

Programmatic Review for NPS-sponsored Public Surveys

OMB Control Number 1024-0224

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The potential respondent universe will consist of visitors, potential visitors, and residents of communities near parks. All study proposals must include a description of a survey’s particular respondent universe.

Based on experience with the existing Programmatic Clearance, we estimate that there will be approximately 50,000 on-site/mail back survey respondents, and 7,500 respondents participating by other means annually. Respondent types will primarily include visitors/recreationists and community stakeholders.

Based on our experience with the existing Programmatic Clearance, we anticipate response rates at or above levels needed to obtain statistically viable results. The Social Science Program Office will encourage applicants to consider a range of approaches that can be used to steps to maximize response rates and ensure that these are integrated as part of the survey methodology in the supporting statement for each information collection to be considered under the clearance.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

All submissions under this program will be carefully evaluated to ensure consistency with the intent, requirements, and boundaries of this programmatic clearance. Proposed collection instruments and procedures must comply with OMB guidance in “Guidance on Agency Survey and Statistical Information Collections (January 20, 2006).” The sampling methods and reporting statistical data must include a specific description of:

• the sampling plan and sampling procedure (including stratification and selection methods for individual respondents);

• how the instrument will be administered to respondents;

• the planned analysis; and

• expected confidence intervals.

Program managers and investigators submitting information collection requests under this Programmatic Clearance process are strongly encouraged to pretest any information collection instruments to be used. Further, we will strongly encourage use of the Programmatic Clearance to obtain approval to conduct any pretesting that falls under the requirements of the Paperwork Reduction Act (i.e., more than nine individuals are being surveyed, etc.). In these cases, requests for approval to pretest surveys will be subject to the same requirements (i.e., a supporting statement, copy of the instrument, etc.) as a standard information collection.

The NPS Social Science Program office will conduct an administrative review of each request and oversee technical reviews of each request to ensure statistical validity and soundness. All information collection instruments will be designed and deployed based upon acceptable statistical practices and sampling methodologies, where appropriate, and will be used to obtain consistent, valid data that are representative of the target populations, account for non-response bias, and achieve response rates at or above levels needed to obtain statistically useful results.

All submissions under the program of expedited approval must fully describe the survey methodology. The description must be specific and describe, as appropriate, each of the following: (a) respondent universe, (b) the sampling plan and all sampling procedures, including how individual respondents will be selected, (c) how the instrument will be administered, (d) expected response rate and confidence, and (e) strategies for dealing with potential non-response bias. A description of any pre-testing and peer review of the methods and/or instrument is highly recommended. Further, all submissions under this clearance process will describe how data will be presented to managers and any others that will use results of the surveys, particularly in cases where response rates were lower than anticipated. In these cases, program managers must take steps to ensure that the appropriate that the results will not be generalized outside the population of interest and explanations are provided with data presentations and reports so that users of the data understand any possible biases associated with the data.

The NPS Social Science Program has developed a form that program managers must complete for any submission under the Programmatic Clearance Process for NPS Public Surveys. The form addresses all points mentioned above. This form appears in the detailed Guidelines Document that explains the full process for submitting a specific instrument for expedited approval under this Programmatic Clearance. This Guidelines Document is attached in ROCIS as part of this submission for an extension of OMB control number 1024-0224.

In its technical and administrative review, the NPS Social Science Program will work with researchers to ensure that information-collection procedures are appropriate for the intended uses of the data.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

For surveys designed to infer from a sample to a population, the NPS requires that proposals address issues of potential non-response. Surveys must incorporate best practices to maximize initial response rates (i.e., multiple follow-ups or call-backs, minimal hour burden). Further, specific strategies for detecting and analyzing non-response bias are to be included in the submission form accompanying survey instruments. These may involve the use of survey logs in which observable characteristics of all those initially contacted on-site are recorded and/or a short interview asking a small number of questions to survey respondents and non-respondents. Investigators conducting telephone surveys may use their most experienced interviewers to convert “soft refusals” to completed interviews in order to maximize response rates.

The NPS Social Science Program requires that the results of non-response bias analyses be included in technical reports, and that the likely effects of this bias (if any) on the interpretation of data must be made clear to managers. In some cases, it may be feasible to balance or post-weight a sample to align important sample statistics, e.g., demographic or zip code characteristics, with known population parameters. However, this does not guarantee that there will not be non-response bias in attitude, knowledge, or belief variables.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Pre-testing and peer review of methods, procedures, and data-collection instruments are strongly recommended by the NPS as a means to reduce respondent burden and maximize the validity of NPS-sponsored surveys. In its review of submissions under the programmatic approval, the Social Science Program considers the need for pre-testing and other developmental work prior to fielding a full survey. In some cases, as when novel questions are employed or a new topic is investigated, NPS recommends that developmental work or pre-testing be conducted. This may be under a separate clearance. However, for many surveys likely to be submitted under this program, the methods and questions have a long history of successful application in the field. In problematic cases, NPS will ask for documentation in peer-reviewed literature of a question’s previous use. In all cases, NPS strongly encourages pre-testing on nine or fewer respondents prior to proposal submission to verify respondent comprehension, identify sources of measurement error, and refine estimates of hour burden. Ideally, participants in pre-tests should be drawn from the same respondent universe as the full sample. However, if this is not feasible, a similar respondent universe (e.g., visitors to a nearby park comparable to the one where the survey will be conducted) should be used.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The names and contact information of the responsible NPS liaison and the principal investigator(s) who will collect and analyze the data are included on all submission forms received under the programmatic approval. In addition, the following individuals were consulted on statistical and other design aspects of this program.

NPS Social Science program in concert with The Visitor Services Program (VSP) at Idaho State University, Moscow, will serve as the “secondary office of control,” helping to coordinate the survey design and research efforts of NPS and ensuring that statistical aspects of all surveys remain consistent with the programmatic approval. The Social Science Program office will require that each proposed information collection is reviewed by a qualified statistician prior to submission of the collection. Further, The NPS Information Collections Review coordinator will review and approve the proposed information collection prior to submission of the collection request to OMB for expedited approval.

Steve Hollenhorst , Director, Park Studies Unit Phone: 208-885-7911 email: [email protected]	Phil Cook, Research Associate Phone: 208-885-5980 email: [email protected]
Margaret LittleJohn, VSP Director National Park Service Phone: 208-885-7863 maragaret@psu.uidaho.edu	Dr. Lena Le, Assistant Visitor Services Project Director University of Idaho (208) 885-2585 lena@psu.uidaho.edu
Marc Manni, Research Team Supervisor Phone: 208-885-2819 email: [email protected]	David Vollmer,IT Database Analyst Phone: 208-885-2585 email: dave@psu.uidaho.edu

5