Justification for CSS-15



U.S. Department of the Interior

Office of Policy Analysis



Justification for a Survey under the Department of the Interior’s Programmatic Clearance for Customer Satisfaction Surveys



Revised March 2012

Instructions for Completing Justification for DOI Programmatic Clearance Submission, OMB Control Number 1040-0001



52152. Survey Title/Date Submitted to the Office of Policy Analysis (PPA): Insert title for the proposed survey. Insert date that the expedited approval package will be submitted to PPA. Reminder: Please submit the package through your bureau/office Information Collection Clearance Officer.

52153. Bureau/Office: Insert the name of the bureau/office conducting the survey.

52154. Abstract: Summarize the proposed study with an abstract not to exceed 150 words.

52155. Bureau/Office Point of Contact Information: Complete the bureau/office contact information. PPA will communicate with the point of contact listed here throughout the entire approval process.

52156. Principal Investigator (PI) Conducting the Survey: Complete information about the PI who will be conducting the survey, if different than Point of Contact listed in #4. Otherwise note: Same as #4.

52157. Name of Program Office Conducting Survey: Provide the name of the bureau program, office, or organizational unit conducting the survey.

52158. Description of Customers and Services Provided: Provide a brief description of the customers who will be surveyed, the services provided by the program conducting the survey, and how these services are provided to customers.

52159. Survey Dates: List the time period in which the survey will be conducted, including specific starting and ending dates. The starting date should be at least 45 days after the submission date. The request for expedited approval, and submission of a complete and accurate approval package, must be made at least 45 calendar days prior to the first day the PI wishes to administer the survey instrument to the public.

52160. Type of Information Collection Instrument: Check the type(s) of information collection instrument(s) that will be used. If other, please explain.

52161. Survey Development: Explain how the survey was developed. With whom did you consult during the development of the survey on content? On statistics? Did you pretest the survey? What actions did you take to improve the survey? What suggestions did you receive for improving the survey? Which of the six topic areas will be addressed? (Note: A description of any pre-testing and peer review of the methods and/or instrument is highly recommended.)

52162. Survey Methodology: Explain how the survey will be conducted. Provide a description of the survey methodology including: (a) How will the customers be sampled? (if fewer than all customers will be surveyed); (b) What percentage of customers asked to take the survey will respond, and (c) What actions are planned to increase the response rate? If statistics are generated, this description must be specific and include each of the following:

- The respondent universe,

- The sampling plan and all sampling procedures;

- How the instrument will be administered;

- Expected response rate and confidence levels; and

- Strategies for dealing with potential non-response bias.

Note: Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by the web site.

12. Total Number of Initial Contacts and Expected Number of Respondents: Provide an estimated total number of initial contacts and the total number of expected respondents.

13. Estimated Time to Complete Initial Contact and Time to Complete Survey Instrument: Estimate the time to complete the initial contact and the time to complete the survey instrument (in minutes).

14. Total Burden Hours: Provide the total number of burden hours. The total burden hours should account for the amount of time required to instruct the respondents in completing the survey, and the amount of time required for the respondent to complete the survey.

15. Reporting Plan: Provide a brief description of the reporting plan for the data being collected.

16. Justification, Purpose and Use: Provide a brief justification for the survey, its purpose, goals, and utility to managers. Specifically, describe how data will be tabulated and what statistical techniques will be used to generalize the results to the entire customer population. Describe how data from the survey will be used. Describe how you will acknowledge any limitations related to the data, particularly in cases where we obtain a lower than anticipated response rate. Note whether or not the survey is intended to measure a Government Performance and Results Act (GPRA) performance measure.

Justification for Submission under DOI Programmatic Clearance for Customer Satisfaction Surveys (OMB Control Number 1040-0001)

U.S. Department of the Interior Office of Policy Analysis (PPA)

PPA Tracking Number: (for PPA use only) CSS-15

			Date Submitted to PPA:	12/17/2014
1.	Survey Title:	BLM Arcata Field Office Visitor Satisfaction Survey
2.	Bureau:	Arcata Field Office, Bureau of Land Management



3.

Abstract: (not to exceed 150 words) The BLM Arcata Field Office requests permission for a visitor survey at four high-visitation recreation sites in the field office area. This survey will provide critical information regarding customer/visitor satisfaction with the BLM’s recreation services, facilities, rule/policy enforcement, and management at these highly valued sites. In addition, the BLM Arcata Field Office is initiating a revision of its Resource Management Plan (RMP) that will guide management of the public lands in northwest California for the next two decades. This work will provide critical adaptive management information to the BLM regarding management of public lands. This survey would be conducted in partnership with a principal investigator (PI) and students from Humboldt State University (HSU) through an existing assistance agreement. The PI for this project has extensive experience with conducting and analyzing visitor use surveys, including a recent survey project on BLM Arcata Field Office lands.

4.

Bureau/Office Point of Contact Information

First Name:

Benjamin

Last Name:

Blom

Title:

Acting Arcata Assistant Field Manager

Bureau/Office:

BLM Arcata Field Office

Street Address:

1695 Heindon Road

City:

Arcata

State:

CA

Zip code:

95521

Phone:

(707) 825-2310

Fax:

(707) 825-2301

Email:

[email protected]

5.

Principal Investigator (PI) Information

First Name:

Steven

Last Name:

Martin

Title:

Chair and Professor, Department of Environmental Science and Management

Bureau/Office:

Humboldt State University

Address:

1 Harpst Street

City:

Arcata

State:

CA

Zip code:

95521

Phone:

(707) 826-4147

Fax:

(707) 826-4145

Email:

[email protected]

6.

Name of Program or Office Conducting Survey:

BLM Arcata Field Office

7.

Description of Customers and Services Provided:

Visitors to public lands in the BLM Arcata Field Office

8.

Survey Dates

(mm/dd/yyyy)

to

(mm/dd/yyyy)

04/01/2015

07/31/2015

9.

Type of Information Collection Instrument (Check ALL that Apply)

_x_Intercept

__Telephone

_x_Mail

_x_Web-based

__Focus Groups

__Comment Cards

__Other

Explain:

10. Survey Development: (Who assisted in survey content development statistics? Was the survey pretested? How were improvements integrated? Which of the six topic areas will be addressed?) The survey was developed by BLM staff at the Arcata Field Office in partnership with the principal investigator, Steve Martin of Humboldt State University (HSU). This survey was based on prior surveys conducted by Dr. Martin in partnership with the BLM and other federal/state agencies. Additional assistance was received from Tim Casey, professor of recreation management at Colorado Mesa University. Topic areas 1, 2, 3, 4, 5, and 6 will be addressed with this survey.
11. Survey Methodology: (Use as much space as needed; if necessary include additional explanation on separate page).
Respondent Universe		Visitors, 18 years and older, to the following recreation sites in the BLM Arcata Field Office: Ma’lel Dunes Cooperative Management Area, Samoa Dunes Recreation Area, Mike Thompson Wildlife Area/South Spit Humboldt Bay, Lost Coast Headlands.
Sampling Plan/Procedure		The Principal Investigator, Dr. Steven Martin of Humboldt State University (HSU), will lead sampling efforts for this study. Visitors to identified recreation sites (identified above under Respondent Universe) will be contacted on-site by Dr. Martin and/or a trained student assistant. All contacted visitors (respondents) will be asked to participate in a short on-site survey. Following this short on-site survey, respondents will be asked to participate in a mailback survey. If the respondent chooses to participate in the mailback survey, the HSU survey administrator will hand a clipboard to the respondent for him/her to enter their mailing information so that the survey can be mailed directly to them. Respondents will also have the option to have the survey sent electronically by providing their email address. Past experience with this methodology strongly suggests that face-to-face contact during the on-site survey increases the response rate of the follow-up survey. Each of the four sites will be sampled during a high usage part of the recreation season (late Spring through July) and for a set period of time (e.g. two months with a proportional number of weekdays and weekends). Daily start and end times will be flexible to reflect differing patterns of use at different sites. The PI will maintain all survey responses. The completion of both the on-site and mailback surveys will take approximately 35 minutes to complete.
Instrument Administration		A survey administrator will be stationed at key intercept points for each of four sites. Only one individual per group will be asked to participate in the survey. Individuals chosen for surveying will be informed that the BLM is conducting a visitor use study. Using a script as a guide (see survey instrument), the survey technician will explain that the purpose of the survey is to measure overall satisfaction of visitors and results will guide future management of these public lands. Visitors will be told that participation is voluntary and responses will be anonymous. Those who agree to participate will be asked to answer the questions contained within the on-site survey instrument. Afterwards, the participant will be asked to participate in the mailback survey. If they agree to participate, the participant will be handed a clipboard and pencil to enter their mailing and/or email address. The PI will then send this mailback survey to the email or mailing address provided.
Expected Response Rate and Confidence Levels		The BLM and the PI hope to receive completed mailback surveys from 800 visitors (200 from each of four sites). A similar survey conducted on nearby BLM lands had an on-site response rate of 92% and a mailback response of 71%. Therefore, the PI will attempt to contact 1,240 visitors (310 at each site). High past and expected response rates may be attributed to the personal contact from a survey administrator and the potential influence of survey results on the BLM’s management of four highly valued recreation sites. For those analyses done for all four sites combined, on the total (est.) sample of 800 respondents, and based on an estimated total visitor population of 50,000, the confidence interval is 3-4% at a confidence level of .95. For those analyses done on each individual recreation site, on a total (est.) sample of 200 returned surveys, and based on an estimated total visitor population of 10,000, the confidence interval is 6-7% at a confidence level of .95.
Strategies for dealing with potential non-response bias		By collecting some demographic information from participants on site during the on-site survey, the PI will be able to discern if a non-response bias exists when analyzing data from the mailback survey. Also, by providing options for both web-based and mail surveys, we expect to increase response rates in the sampled population.
Description of any pre-testing and peer review of the methods and/or instrument (recommended)		All questions used in this survey have been vetted and used in prior visitor surveys on BLM lands.
12.	Total Number of Initial Contacts and Expected Number of Respondents		1240 contacts/1140 respondents (on-site survey), 1140 contacts/809 respondents (mailback)
13.	Estimated Time to Complete Initial Contact and Time to Complete Instrument		2 minutes (on-site initial contact), 5 minutes (on-site survey), 2 minutes (mailback initial contact), 28 minutes (mailback survey)
14.	Total Burden Hours Contacts Respondents ----------------- Total		Contacts – 41 hours (on-site) + 38 hours (mailback) Respondents - 95 hours (on-site survey)+ 378 hours (mailback survey) Total – 552 hours
15. Reporting Plan: A final visitor survey report will be completed by March 31, 2016. This report will describe all components of the survey, including methodology, results, and discussion. An example of this can be found at http://www.blm.gov/style/medialib/blm/ca/pdf/arcata/headwaters.Par.68049.File.dat/Headwaters%202012%20HSU%20final%20report_508%20compliant.pdf.

16. Justification, Purpose, and Use:
Survey Justification and Purpose	This survey will provide critical information regarding customer/visitor satisfaction with the BLM’s recreation services, facilities, rule/policy enforcement, and management at several highly valued recreation sites. This work will provide critical adaptive management information to the BLM regarding management of public lands.
Survey Goals	Assess satisfaction of visitors to high-visitation areas within the public lands managed by the BLM Arcata Field Office.
Utility to Managers	This work will provide critical adaptive management information to the BLM regarding management of public lands.
How will the results of the survey be analyzed and used?	Data will be analyzed by the PI, in partnership with BLM staff. Results may be used in the development of a revised resource management plan for the BLM Arcata Field Office.
How will the data be tabulated? What Statistical Techniques will be used to generalize the results to the entire customer population? How will limitations on use of data be handled? If the survey results in a lower than anticipated response rate, how will you address this when reporting the results? (Use as much space as needed; if necessary include additional explanation on separate page). The data will be entered into a statistical package database (SPSS) for analysis. Means, standard deviations, and 95% confidence intervals, along with frequency distributions, will be most commonly used. Data reduction techniques such as cluster analysis and factor analysis will also likely be used. A lower than anticipated response rate would be handled by using 95% confidence intervals, which take into account the sample size.
Is this survey intended to measure a Government Performance and Results Act (GPRA) performance measure? If so, please include an excerpt from the appropriate document. (Use as much space as needed; if necessary include additional explanation on separate page). This survey is intended to measure the following GPRA performance measures: Percent of visitors satisfied with the quality of their experience Percent satisfaction among visitors served by facilitated programs These measures were excerpted from the US DOI 2014/2015 Annual Performance Plan & 20013 Report (APP&R).

Checklist for Submitting a Request to Use DOI Programmatic Clearance for Customer Satisfaction Surveys

X All questions in the survey instrument are within the scope of one of the DOI Programmatic Clearance for Customer Satisfaction Surveys topic areas.



X The approval package is being submitted to the Office of Policy Analysis at least 45 days prior to the first day the PI wishes to administer the survey to the public.



X A qualified statistician has reviewed and approved your request.



X Your bureau/office Information Collection Clearance Officer has reviewed and approved the approval package.



The approval package includes:

X A completed Justification

x A signed Certification Form

x A copy of the survey instrument

x Other supporting materials, such as:

• Cover letters to accompany mail-back questionnaires

• Introductory scripts for initial contact of respondents

• Necessary Paperwork Reduction Act compliance language

• Follow-up letters/reminders sent to respondents



The survey methodology presented in the Justification includes a specific description of:

X The respondent universe

X The sampling plan and all sampling procedures, including how respondents will be selected

X How the instrument will be administered

X Expected response rate and confidence levels

X Strategies for dealing with potential non-response bias

X A description of any pre-testing and peer review of the methods and/or the instrument is highly recommended.



X The burden hours reported in the Justification include the number of burden hours associated with the initial contact of all individuals in the sample (i.e., including refusals), if applicable, and the number of burden hours associated with individuals expected to complete the survey instrument.

X The package is properly formatted (Word) and submitted to the Office of Policy Analysis electronically. 

1