CSS-5 Justification and Certification



U.S. Department of the Interior

Office of Policy Analysis



Guidelines for Accessing the Department of the Interior’s Generic Clearance for Customer Satisfaction Surveys



Revised February 2009

ATTACHMENT 1:

Instructions for Completing Supporting Statement for DOI Generic Clearance Submission, OMB Approval Number 1040-0001

38480. Survey Title/Date Submitted to the Office of Policy Analysis (PPA): Insert title for the proposed survey. Insert date that the expedited approval package will be submitted to PPA. Reminder: Please submit the package through your bureau/office Information Collection Clearance Officer.

38481. Bureau/Office: Insert the name of the bureau/office conducting the survey.

38482. Abstract: Summarize the proposed study with an abstract not to exceed 150 words.

38483. Bureau/Office Point of Contact Information: Complete the bureau/office contact information. PPA will communicate with the point of contact listed here throughout the entire approval process.

38484. Principal Investigator (PI) Conducting the Survey: Complete information about the PI who will be conducting the survey, if different than Point of Contact listed in #4. Otherwise note: Same as #4.

38485. Name of Program Office Conducting Survey: Provide the name of the bureau program, office, or organizational unit conducting the survey.

38486. Description of Customers/Services Provided: Provide a brief description of the customers who will be surveyed, the services provided by the program conducting the survey, and how these services are provided to customers.

38487. Survey Dates: List the time period in which the survey will be conducted, including specific starting and ending dates. The starting date should be at least 45 days after the submission date. The request for expedited approval, and submission of a complete and accurate approval package, must be made at least 45 calendar days prior to the first day the PI wishes to administer the survey instrument to the public.

38488. Type of Information Collection Instrument: Check the type(s) of information collection instrument(s) that will be used. If other, please explain.

38489. Survey Development: Explain how the survey was developed. With whom did you consult during the development of the survey on content? On statistics? Did you pretest the survey? What actions did you take to improve the survey? What suggestions did you receive for improving the survey? Which of the six topic areas will be addressed? (Note: A description of any pre-testing and peer review of the methods and/or instrument is highly recommended.)

38490. Survey Methodology: Explain how the survey will be conducted. Provide a description of the survey methodology including: (a) How will the customers be sampled? (if fewer than all customers will be surveyed); (b) What percentage of customers asked to take the survey will respond, and (c) What actions are planned to increase the response rate? If statistics are generated, this description must be specific and include each of the following:

- The respondent universe,

- The sampling plan and all sampling procedures, including how individual respondents will be selected;

- How the instrument will be administered;

- Expected response rate and confidence levels; and

- Strategies for dealing with potential non-response bias.

Note: Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by the web.

12. Total Number of Initial Contacts/Expected Number of Respondents: Provide an estimated total number of initial contacts and the total number of expected respondents.

13. Estimated Time to Complete Initial Contact/Instrument: Estimate the time to complete the initial contact and the survey instrument (in minutes).

14. Total Burden Hours: Provide the total number of burden hours. The total burden hours should account for the amount of time required to instruct the respondents in completing the survey, and the amount of time required for the respondent to complete the survey.

15. Reporting Plan: Provide a brief description of the reporting plan for the data being collected. A copy of all survey reports must be archived with PPA. Please note this in the reporting plan.

16. Justification, Purpose and Use: Provide a brief justification for the survey, its purpose, goals, and utility to managers. Specifically, describe how data will be tabulated and what statistical techniques will be used to generalize the results to the entire customer population. Describe how data from the survey will be used. Describe how you will acknowledge any limitations related to the data, particularly in cases where we obtain a lower than anticipated response rate. Note whether or not the survey is intended to measure a Government Performance and Results Act (GPRA) performance measure.

ATTACHMENT 2:

Approval Form for DOI Programmatic Clearance for Customer Satisfaction Surveys (OMB Control Number 1040-0001)

U.S. Department of the Interior Office of Policy Analysis (PPA)

PPA Tracking Number: (for PPA use only) CSS-5

			Date Submitted to PPA:	3/28/2011
1.	Survey Title:	National Park Service Visitor Satisfaction with Concessioners Survey
2.	Bureau:	National Park Service



3.

Abstract: (not to exceed 150 words) This research intends to measure visitor satisfaction with services provided by concessioners within the National Park Service system (NPS). The concessioner survey will be administered for three consecutive years to provide the NPS office specific data at the system level about the service quality provided by the park concessioners. The resultant information is beneficial to better meet the needs and desires of NPS visitors.

4.

Bureau/Office Point of Contact Information

First Name:

Kurt

Last Name:

Rausch

Title:

Senior Operations Analyst

Bureau/Office:

NPS- Commercial Services Program

Street Address:

1201 Eye Street- 11th Floor

City:

Washington

State:

DC

Zip code:

20005

Phone:

202-513-7202

Fax:

202-371-2090

Email:

[email protected]

5.

Principal Investigator (PI) Information

First Name:

Jennifer

Last Name:

Hoger Russell

Title:

Research Support Scientist

Bureau/Office:

University of Idaho- Park Studies Unit

Address:

P.O. Box 441139

City:

Moscow

State:

ID

Zip code:

83844-1139

Phone:

208-885-4806

Fax:

208-885-4261

Email:

[email protected]

6.

Name of Program or Office Conducting Survey:

National Park Service Commercial Services Program

7.

Description of Customers/ Services Provided:

Customers will be any visitor purchasing a good or service provided by a concessioner at an NPS site selected to participate in this study and for the specific functional group of services used. For the purposes of this study, concessioners were limited to retail, food, lodging, guide services, and marinas.

8.

Survey Dates

(mm/dd/yyyy)

to

(mm/dd/yyyy)

04/15/2011

04/14/2014

9.

Type of Information Collection Instrument (Check ALL that Apply)

_X_Intercept

__Telephone

__Mail

__Web-based

Focus Groups

_X_Comment Cards

__Other

Explain:

10. Survey Development: (Who assisted in survey content development statistics? Was the survey pretested? How were improvements integrated? Which of the six topic areas will be addressed?) The survey instruments were developed by the University of Idaho Park Studies Unit in consultation with Visual Input Systems Analysts, Inc. and the Commercial Services Program of the National Park Service. Content of the surveys was directed by the needs of the NPS Commercial Services Program for information regarding customer satisfaction that has previously never been investigated. All survey instruments were evaluated by the UI-PSU statistician Dr. Lena Lee. Survey instruments were pretested using the UI-PSU staff and students from the Conservation Social Science Department at the University of Idaho. The results of the pilot showed people distributing themselves across the possible response choices offered and none of the questions skipped. Survey questions will cover the following approved topic areas: Delivery, quality and value of products, information, and services: The questions in this category will allow the respondent to provide information concerning their level of satisfaction with the services and products offered by the concessionaires within the survey sample. Management practices: the respondent will be able to provide feedback concerning ways the concessionaire could improve the retail experience in the survey sample. Mission management: The questions in this category will ask visitors to provide feedback concerning their satisfaction data with the services provided by the concessionaires in the survey sample. Interaction with DOI personnel and contractors: The questions in this category will focus on obtaining customer feedback regarding customer satisfaction concerning visitor interactions with concessionaire within the survey sample. The respondents will be asked to rate their level of satisfaction with the staff, merchandise and quality of their experience. General demographics: The questions in this category are limited to gender, age and home zip code.
11. Survey Methodology: (Use as much space as needed; if necessary include additional explanation on separate page).
Respondent Universe	All visitors, 18 years of age or older, who have actually used the concessioner service at the survey location and on the survey day are potential respondents. However, visitors may not complete a survey within more than one concessioner service category.
Sampling Plan/Procedure	This is a national study and every NPS location, which offers a concession service, is included in the pool from which the sample of participating parks was drawn. To ensure a representative range of concessioner services, five types of concessioner services (retail, dining, lodging, guided tours and marinas) were selected by the NPS to be included in the study. Since some NPS locations contain services in all five categories whereas others may contain only one service the NPS Commercial Services Program office in Washington, D.C. provided a list for this project of all NPS units and which service(s) the unit provides. NPS locations will be stratified based upon the five concessioner services offered at each location. Within each group, a random sample of ten locations will be chosen. Because some locations offer multiple concession services, some locations will be randomly selected to be surveyed in more than one category. For each functional group an NPS site will be selected and each site will be responsible for choosing one concessioner location. Each location will be responsible for collecting 56 surveys during their assigned a survey month. Within this month, it will be the NPS Survey coordinator’s responsibility to manage the survey administration by choosing four survey days (evenly distributed between weekdays and weekends) and a two-hour period on each survey day to distribute surveys. A procedures manual will be provided to each NPS location to ensure that sampling and survey administration is consistent.
Instrument Administration	The surveyor will stand outside the concessioner location and, using an objective method for randomization, approach exiting visitors. A script will be provided so all surveyors follow the same process to identify visitors who have actually used the service, inform them of the survey, obtain verbal consent and ensure that all visitors realize their participation is voluntary and anonymous. (See the attached script)
Expected Response Rate Confidence Levels:	Visitors will be asked to complete the survey instrument on site and deposit it into a locked box. This method has been used to administer and collect surveys in the National Parks with a response rate of 80-85%. Using the same method we expect similar response rates. Distribution numbers were calculated using a 95% confidence level with a +/- 5% sampling error.
Strategies for dealing with potential non-response bias	Surveyors will be provided with a record sheet and trained to complete the following information for each visitor they approach: whether the visitor accepts or refuses to complete a survey, gender, zip code and overall satisfaction with the concessioner service experience. This collected information can be used to compare the visitors who refused to take the instrument survey, to those who took the instrument and completed it, and to those who took the instrument but did not return it, to determine if significant differences exist.
Description of any pre-testing and peer review of the methods and/or instrument (recommended)	To reduce measurement error, a small pilot study (margin of error = 18%), was done with the completed instruments to evaluate question wording and questionnaire construction. The results of the pilot showed people distributing themselves across the possible response choices offered and none of the questions skipped. Concessioners Survey Instruction Manual: We will provide each NPS Concessioner Survey Coordinator (an NPS employee) with a copy of the Concessioner Survey Instructions Manual (see attachment 1). The coordinator will be responsible for choosing the managing the survey schedule and protocol at each site. Understanding and following the guidelines in the manual is imperative to the success of the concessioner survey. A pilot study of the manual was done to ensure that different types of people would consistently attain correct results using the manual. Groups of three were given jobs of coordinator, surveyor and visitor. The coordinator was given an example park location with the information necessary to choose a concessioner location, survey days, and times. The surveyor was given the manual to read and provided additional instructions as needed by the coordinator. The visitor was asked to present questions and respond both positively and negatively to the surveyor interview questions. After each group, additions and changes were made to the manual based on the comments or problems of individuals. Then the pilot was repeated. After the fourth pilot group, people were able to follow the manual and consistently attain the desired result. Methodology of the survey mimics 2 successful surveys currently conducted by the Park Studies Unit for the National Park Service (OMB Control Number 1024-0216).

12.	Total Number of Initial Contacts: 70 Expected Number of Respondents: 56	70 contacts per park × 80% (for refusals) = 56 possible responses
13.	Estimated Time to Complete Initial Contact: 3 mins x 70 contacts =210 minutes Estimated Time to Complete Instrument (mins.): 5 mins x 56 responses = 280 minutes	Total time per individual Parks Initial Contact: 3 hours and 30 Minutes Time to Compete Survey: 4. Hours and 40 Minutes
14.	Estimated Total Burden Hours: 490 minutes X 50 Parks = 408 hours
15. Reporting Plan: We will provide the Commercial Services Program with a system wide report annually and each park a summary of their results by category annually. At the end of three years we will provide a summary overview for the Commercial Services Program. Additionally all reports will be posted on our website psu.uidaho.edu

16. Justification, Purpose, and Use:
Survey Justification and Purpose	The NPS Commercial Services Program contracted with the Park Studies Unit (PSU), based at the University of Idaho’s Department of Conservation Social Science, to develop a concessions-specific survey to evaluate user experiences with concessioner services. This Concessioner Survey is a three year collection that will provide specific data, usable at the system and park level about the quality of services provided by park concessioners.
Survey Goals	To assess visitor satisfaction with the services provided by the contracted concessioners needed to make decisions to better serve the public.
Utility to Managers	NPS Commercial Services Program will have quantifiable data regarding visitor satisfaction with services provided by contracted concessioners.
How will the results of the survey be analyzed and used?	Results of the surveys will be electronically scanned and frequencies reported for each response category. Concessions managers will be able this information to highlight strengths and weaknesses with the services provided to visitors who use the service.
How will the data be tabulated? What Statistical Techniques will be used to generalize the results to the entire customer population? How will limitations on use of data be handled? If the survey results in a lower than anticipated response rate, how will you address this when reporting the results? (Use as much space as needed; if necessary include additional explanation on separate page). Survey instruments will be scanned and tabulated electronically. Frequencies will be reported for each response category. Results will be generalize-able to the entire customer population because the number of surveys distributed and expected was calculated using the Dillman et al., 2009, p. 207 calculation for a population ranging from 8,000 to 1,000,000,000. This study assumes a 75% response rate for conservative estimates; however past experience with NPS surveys using this method have received 80- 85% response rates. If lower than expected returns are encountered, language describing this will be included in the reports and cautions to the use of the data will be explained.
Is this survey intended to measure a Government Performance and Results Act (GPRA) performance measure? If so, please include an excerpt from the appropriate document. (Use as much space as needed; if necessary include additional explanation on separate page). The Government Performance and Results Act (GPRA) require federal agencies to document their performance in annual performance plans and performance reports. This Act authorizes the National Park Service to collect information to evaluate the customer satisfaction and performance of external partnerships and organizational effectiveness. This information collection can be used as a GPRA measure to support NPS’ programmatic requirements, mission, and goals to be a provider of the best possible services for park visitors and partners.

ATTACHMENT 3

Checklist for Submitting a Request to Use DOI Programmatic Clearance for Customer Satisfaction Surveys

X All questions in the survey instrument are within the scope of one of the DOI Programmatic Clearance for Customer Satisfaction Surveys topic areas.



X The approval package is being submitted to the Office of Policy Analysis at least 45 days prior to the first day the PI wishes to administer the survey to the public.



X A qualified statistician has reviewed and approved your request.



X Your bureau/office Information Collection Clearance Officer has reviewed and approved the approval package.



The approval package includes:

X A completed Information Form

X A signed Certification Form

X A copy of the survey instrument

X Other supporting materials, such as:

• Cover letters to accompany mail-back questionnaires

• Introductory scripts for initial contact of respondents

• Necessary Paperwork Reduction Act compliance language

• Follow-up letters/reminders sent to respondents



The survey methodology presented on the Information Form includes a specific description of:

X The respondent universe

X The sampling plan and all sampling procedures, including how respondents will be selected

X How the instrument will be administered

X Expected response rate and confidence levels

X Strategies for dealing with potential non-response bias

X A description of any pre-testing and peer review of the methods and/or the instrument is highly recommended.



X The burden hours reported on the Information Form include the number of burden hours associated with the initial contact of all individuals in the sample (i.e., including refusals), if applicable, and the number of burden hours associated with individuals expected to complete the survey instrument.

X The package is properly formatted (Word) and submitted to the Office of Policy Analysis electronically. 

ATTACHMENT 4

CERTIFICATION FORM FOR SUBMISSION UNDER OMB CONTROL NUMBER 1040-0001

This form should only be used if you are submitting a collection of information for approval under the DOI Programmatic Clearance for Customer Satisfaction Surveys.

If the collection does not satisfy the requirements of the Programmatic Clearance, you should follow the regular PRA clearance procedures described in 5 CFR 1320.

Bureau/Office Subgroup or Program National Park Service Commercial Services Program
Title (Please be specific) National Park Service Visitor Satisfaction with Concessioners Survey
Burden Hour		Initial Contact: =210 minutes Time to Complete Instrument = 280 minutes		Estimate Number of Respondents			Initial Contact:70 Survey Completion:56 (times 50 Parks)
Total Burden Hours		408 hours		Hours/Min per Response			Initial Contact: 3 mins Survey Completion: 5 mins (times 50 Parks)
Bureau/Office Contact (who can best answer questions about content of the submission):
Name	Jennifer Hoger Russell		Phone		208-885-4806
Certification: The collection of information requested by this submission meets the requirements of OMB control number 1040-0001
Bureau/Office Qualified Statistician Dr. Lena Le,						DATE March 23, 2011
Bureau/Office Information Collection Clearance Officer Phadrea Ponds						DATE March 23, 2011
Office of Policy Analysis Don Bieniewicz, DOI Information Collection Coordinator/Clearance Officer						DATE April 21, 2011
OMB, Office of Information and Regulatory Affairs (OIRA)						DATE

1