Justification for Part. Sat. Survey -- EDMAP program



Justification for Submission under DOI Programmatic Clearance for Customer Satisfaction Surveys (OMB Control Number 1040-0001)

U.S. Department of the Interior Office of Policy Analysis (PPA)

PPA Tracking Number: (for PPA use only) CSS-13

			Date Submitted to PPA:	10/9/2012
1.	Survey Title:	Participant Satisfaction Survey – EDMAP program
2.	Bureau:	U.S. Geological Survey



3.

Abstract: (not to exceed 150 words) The National Cooperative Geologic Mapping Program (NCGMP) is the primary source of funds for the production of geologic maps in the United States and provides accurate geologic maps and 3-dimensional framework models that help to sustain the quality of life and economic vitality of the Nation and to mitigate natural hazards. EDMAP is the component of the NCGMP that trains the next generation of geologic mappers. Geology professors, who are skilled in geologic mapping, request EDMAP funding to support undergraduate and graduate students in a 1-year mentored geologic mapping project. The EDMAP Participant Satisfaction Survey provides a structured mechanism to gather feedback from participating students on the quality of their EDMAP experience.

4.

Bureau/Office Point of Contact Information

First Name:

Stephen

Last Name:

Gillespie

Title:

Economist

Bureau/Office:

USGS / Office of the Director

Street Address:

12201 Sunrise Valley Drive

City:

Reston

State:

VA

Zip code:

20192

Phone:

703-648-5705

Fax:

703-648-5068

Email:

[email protected]

5.

Principal Investigator (PI) Information

First Name:

Same as #4

Last Name:

Title:

Bureau/Office:

Address:

City:

State:

Zip code:

Phone:

Fax:

Email:

6.

Name of Program or Office Conducting Survey:

National Cooperative Geologic Mapping Program

7.

Description of Customers and Services Provided:

Customers: Students who joined the EDMAP program 3 years previously and subsequently completed the program. Services provided: Funding support

8.

Survey Dates

(mm/dd/yyyy)

to

(mm/dd/yyyy)

04/01/2013

05/31/2013

9.

Type of Information Collection Instrument (Check ALL that Apply)

__Intercept

__Telephone

__Mail

_X_Web-based

__Focus Groups

__Comment Cards

__Other

Explain:

10. Survey Development: (Who assisted in survey content development statistics? Was the survey pretested? How were improvements integrated? Which of the six topic areas will be addressed?) The questionnaire was developed as a cooperative effort of EDMAP program management and Stephen Gillespie, an economist in the USGS Office of the Director. Similar questions were asked to past participants in the EDMAP program. As a result of this prior testing, the question concerning satisfaction with guidance was simplified. The topic areas addressed are: 1. Delivery, quality, and value of products, information, and services 2. Administrative practices 6. General demographics
11. Survey Methodology: (Use as much space as needed; if necessary include additional explanation on separate page).
Respondent Universe		All students who entered the EDMAP program 3 years previous, and subsequently completed the program.
Sampling Plan/Procedure		Complete census of all EDMAP participants.
Instrument Administration		Each respondent is sent an email with a link to the on-line questionnaire. The link includes a unique identifier.
Expected Response Rate and Confidence Levels		The expected response rate is 70%. Since the survey is a census, the aggregate quantitative results contain no sampling error. On the basis of the results of the pretesting, it is anticipated that non-sampling error will be low. The anticipated precision of the quantitative results is significantly higher than that required (plus or minus 10 percentage points) for the purposes to which they will be applied.
Strategies for dealing with potential non-response bias		1. Non-respondents to the email will be contacted by telephone where possible. This subsequent phone call significantly reduces non-response rates. 2. Aggregate opinions of those who responded to the email will be compared to the aggregate opinions of those who responded only after a phone call. Without the reminder, most of the telephone respondents would have remained non-response. The persons who responded only after a telephone reminder can be thought of as being intermediate between the email responders and the remaining non-response. The extent to which the telephone responders systematically differ in opinions from the email responders suggests the possible level and direction of non-response bias. 3. Quantitative results will be compared to quantitative results from prior surveys. Outlier values in a time series will flag potentially unreliable data.
Description of any pre-testing and peer review of the methods and/or instrument (recommended)		No additional pre-testing is planned prior to each year’s survey.
12.	Total Number of Initial Contacts and Expected Number of Respondents		Around 35 initial contacts per year. Around 25 respondents per year.
13.	Estimated Time to Complete Initial Contact and Time to Complete Instrument		Initial contact = 1 minute Complete questionnaire = 10 minutes
14.	Total Burden Hours Contacts Respondents ----------------- Total		Initial contact: 35 persons @ 1 minute = about ½ hour Complete questionnaire: 25 persons @ 10 minutes = about 4 hours Total burden = about 5 hours
15. Reporting Plan: A written report will be provided to the EDMAP program office. The report will tabulate all responses to the survey, while preserving the confidentiality of the respondents. In addition, for those respondents who request further contact with EDMAP HQ, the program office will be provided with a data sheet detailing that respondent’s responses.

16. Justification, Purpose, and Use:
Survey Justification and Purpose	The purpose of the survey is to give EDMAP participants a structured mechanism for providing feedback to program managers on their EDMAP experience, and to quantify participant satisfaction with relevant aspects of that experience.
Survey Goals	The goal of the survey is the development of a quantitative measure of program achievement, and the collection of participant feedback to support program improvement.
Utility to Managers	The survey results will allow program managers to assess the success of the EDMAP program, and to identify specific actions to improve the administration of the program.
How will the results of the survey be analyzed and used?	The aggregate percentage of participants who are satisfied with an aspect of the program is an important metric for assessing how well the program is functioning in that area. Respondent comments (and later followup conversations) provide anecdotal information on what the program is doing well and where improvements can be made.
How will the data be tabulated? What Statistical Techniques will be used to generalize the results to the entire customer population? How will limitations on use of data be handled? If the survey results in a lower than anticipated response rate, how will you address this when reporting the results? (Use as much space as needed; if necessary include additional explanation on separate page). The quantitative data are relatively sparse. Simple frequency distributions is the only statistical technique used. If non-response bias appears likely, the report provided to EDMAP management will discuss the probable size and direction of the bias, so that quantitative results can be interpreted appropriately. Program improvements are more likely to arise from the anecdotal comments than from the quantitative results, and lower than expected response rates do not illegitimize anecdotal comments.
Is this survey intended to measure a Government Performance and Results Act (GPRA) performance measure? If so, please include an excerpt from the appropriate document. (Use as much space as needed; if necessary include additional explanation on separate page). The survey results are used to generate the metric: “percent of EDMAP students that work on subsequent degrees or obtain a job in a geosciences field.”

Checklist for Submitting a Request to Use DOI Programmatic Clearance for Customer Satisfaction Surveys

X All questions in the survey instrument are within the scope of one of the DOI Programmatic Clearance for Customer Satisfaction Surveys topic areas.



X The approval package is being submitted to the Office of Policy Analysis at least 45 days prior to the first day the PI wishes to administer the survey to the public.



X A qualified statistician has reviewed and approved your request.



X Your bureau/office Information Collection Clearance Officer has reviewed and approved the approval package.



The approval package includes:

X A completed Justification

X A signed Certification Form

X A copy of the survey instrument

X Other supporting materials, such as:

• Cover letters to accompany mail-back questionnaires

• Introductory scripts for initial contact of respondents

• Necessary Paperwork Reduction Act compliance language

• Follow-up letters/reminders sent to respondents



The survey methodology presented in the Justification includes a specific description of:

X The respondent universe

X The sampling plan and all sampling procedures, including how respondents will be selected

X How the instrument will be administered

X Expected response rate and confidence levels

X Strategies for dealing with potential non-response bias

X A description of any pre-testing and peer review of the methods and/or the instrument is highly recommended.



X The burden hours reported in the Justification include the number of burden hours associated with the initial contact of all individuals in the sample (i.e., including refusals), if applicable, and the number of burden hours associated with individuals expected to complete the survey instrument.

X The package is properly formatted (Word) and submitted to the Office of Policy Analysis electronically. 

CERTIFICATION FORM FOR SUBMISSION UNDER OMB CONTROL NUMBER 1040-0001

This form should only be used if you are submitting a collection of information for approval under the DOI Programmatic Clearance for Customer Satisfaction Surveys.

If the collection does not satisfy the requirements of the Programmatic Clearance, you should follow the regular PRA clearance procedures described in 5 CFR 1320.

Bureau/Office Subgroup or Program US Geologic Survey / National Cooperative Geologic Mapping Program
Title (Please be specific) Participant Satisfaction Survey – EDMAP Program
Estimated Number Contacts Respondents		35 25	Time per Response Contacts Respondents				1 minute 10 minutes
			Total Burden Hours Contacts Respondents ----------------- Total				½ hour 4 hours ------------- 5 hours
Bureau/Office Contact (who can best answer questions about content of the submission):
Name	Stephen Gillespie			Phone	703-648-5705
Certification: The collection of information requested by this submission meets the requirements of OMB control number 1040-0001
Bureau/Office Qualified Statistician Stephen Gillespie						DATE August 02, 2012
Bureau/Office Information Collection Clearance Officer Shari Baloch						DATE October 9, 2012
Office of Policy Analysis Donald Bieniewicz						DATE October 9, 2012
OMB, Office of Information and Regulatory Affairs (OIRA)						DATE

15