Supporting Statement B for National GAP Evaluation Revised 4-26-2011

Supporting Statement B

National Gap Analysis Program Evaluation

OMB Control Number: 1028-NEW

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Potential Respondent Universe and Sampling Selection Method

The nature of the population of GAP users required that we use sampling methods designed to identify individuals in a hidden population. A portion of the GAP users are known to GAP, but there are in fact, unknown users of GAP data sets whose perspective should be included in a thorough evaluation of GAP. Because there are no existing lists of GAP users this population was considered to be hidden, we used a snowball sampling method revealed to identify a sufficient number of users for our survey.

We began the process by creating a database of authors who published articles in volumes 1-14 of the GAP Bulletin and we solicited names of potential users from the offices that develop state and regional GAP data. The lists included: subscribers to the GAP Bulletin, names from individuals participating in a study regarding State Wildlife Action Plans, and names of individuals who had downloaded GAP data from a site maintained by a state or a university offering state level GAP data. This list included the name and email address for each individual.

We used the list to email each individual. In the email message we alerted them to an upcoming survey and asked if they could provide the names and email addresses of three other individuals they knew used GAP data. When we received new names, we added those names continued the modified snowball method of emailing and and asking if they could provide the names and email addresses of three other individuals who were users of GAP data. We engaged in this process until we believed we could not get more new names.

Expected Response Rate

Because the individuals in the sample are highly trained and use GAP data we believe that they will be engaged in the survey topic, therefore we anticipate a high response rate. Additionally, we will adhere to follow-up procedures for web surveys outlined in Dillman’s Total Design Method (2007). As a result, we anticipate a 70% response rate.

Table 1. Entities in the identified respondent universe for the survey.

Organization	Number of Entities	Average Expected Survey respondents per entity	Total Survey Respondents
U.S. States	50	7	350
U.S. Territories	2	5	10
Tribal Governments	3	1	3
NGO’s and private industry	206	1	206
TOTAL			569

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for sample selection:

Because this sample of 813 nonfederal respondents is not random and the population cannot be determined a priori with an adequate level of accuracy, we will sample 100% of the respondents we identified as GAP users. The sample for this collection will not be stratified.

Estimation procedure and degree of accuracy:

The sample for this collection includes 813 nonfederal users. A 70% response rate from the sample of 813 individuals will be sufficient to characterize the sample. The results will not be generalized to the entire population, but will represent the broad sample of users we have identified and provide information concerning the current and general usability of the data. This information is needed because the USGS must determine ways to improve the use of GAP data.

Because the population of GAP users cannot be determined with a high degree of accuracy, there will be some limitations in the results. The results will give an indication of the uses and limitations of GAP data across multiple sectors of GAP users. We acknowledge that any subsequent reports and interpretation of the data will carefully describe the purpose, intent, and limitations of the study.

Periodic data collection cycles:

This is a one-time survey and no periodic data collection will occur.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

We will use an on-line survey as an alternative to other survey modes such as mail or telephone. This method has become increasingly accepted as a strategy to decrease costs, increase the speed of data collection, and increase response rates with the hope of decreasing the amount of non-response error (Couper, 2000; Dillman, 2007). We feel that due to the highly technical nature of the respondents, they will be more likely to respond to a web-version versus a paper survey option. We predict that the response rate of at least 70% will be met. To maximize the response rate, we will send a letter from the USGS National GAP Program Office. This letter will explain the purpose and the need for the information from the program’s perspective and will come from a highly respected member of the GAP community. The second letter will be the initial email invitation that will contain a link to the survey. E-mail reminders will be sent out 7 and then 10 days later to all non-respondents (excluding those who wrote back asking to be removed from the survey). If the overall response rate is acceptable (above 70 percent) we will not follow-up with non-respondents.

In the event that our response rate is below 70% we will employ non-response methods by conducting a follow-up survey of all non-respondents. This survey will be a shorter version of the original survey. We will send one e-mail message with a link to the follow-up survey (a copy of the follow-up survey and e-mail message is attached in ROCIS).

We do not intend to generalize to the larger population of all GAP users. The data from this study is specifically intended to inform the USGS Gap program about the users of GAP data. We recognize that the respondents will not represent all GAP users however, the survey questions and responses have been designed to highlight the issues that are relevant to the USGS Gap Program that will provide the meaningful information currently unavailable regarding the use of GAP data.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The initial questions for this collection were subject to internal USGS review as well as extensive reviews by its external partners affiliated with GAP. An online version of the survey was pretested with USGS employees to estimate response times. Individuals were also asked to estimate the time burden of the survey. Comments and suggestions provided by reviewers and pre-test participants were evaluated and used to revise the survey instrument where appropriate. In particular, wording changes were made to questions to improve the clarity of the questions for an audience experienced with geospatial information systems (GIS) and to make the wording more consistent with language used by the GAP.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical consultants:

Dr. Jean Opsomer

Colorado State University

Department of Statistics

Professor, Statistics

Office Phone: (970) 491-3841

Email: [email protected]

Collection and analysis agency:

Policy Analysis and Science Assistance Branch

Fort Collins Science Center

U.S. Geological Survey

2150 Centre Ave, Building C

Fort Collins, CO

Phone: 970-226-9315