1018-refuge eval Supporting Statement B with rev

Supporting Statement B for

Paperwork Reduction Act Submission

OMB Control Number 1018-XXXX

National Wildlife Refuge System:

Indian Tribe, State Agency, and Local Partner Surveys

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved.

The survey populations are framed in institutional terms (State agencies, Indian tribes, and local nongovernmental organizations) that collaborate in some fashion with National Wildlife Refuge System (NWRS) refuges. For the purposes of these surveys, State agencies, Indian tribes, and local organizations are defined as those organizations that work directly with the NWRS and/or individual national wildlife refuges on a specific issue, service, or project.

We will ask approximately 550 people to participate in these surveys. We expect approximately 440 respondents, or about 80 percent of those asked to participate. The final number of survey participants (sampling frame) will depend on the number of NWRS partners identified. We will use five inputs to assemble the sampling frame:

(a) List of all volunteer “Friends Groups.”

(b) List of grant recipients (grants funded by the NWRS and awarded through a Cooperative Agreement with the National Fish and Wildlife Foundation);

(c) Active local partner groups.

(d) List of State Fish and Wildlife Agencies.

(e) List of Indian tribes that actively partner with the Refuge System.

Organizations	Sampling Frame
Volunteer Groups Local Conservation Organizations State Fish and Wildlife Agencies Fish and Game Agencies Indian Tribes Schools	Friends Groups Grant Recipients Active Local Partner Groups

The surveys being conducted will not be representative or be generalizable to the community of overall NWRS partners. The surveys, however, will provide additional information on the role and views of NWRS partners in regard to the performance of the NWRS. This data will be used, in combination with other analysis, to help provide an assessment on the performance of the NWRS and, in particular, on the NWRS’ performance in accomplishing Strategic Objective Goal number 6, under its current strategic plan -- SOG6: Facilitate partnerships and cooperative projects to engage other conservation agencies, volunteers, friends, and partners in the refuge system mission.

Other information being collected and analyzed as part of the independent evaluation includes the following: information from interviews with partners and refuge staff; a review of the NWRS performance reporting information system – RAPP (Refuge Annual Performance Plan); a data quality review of the refuge system’s performance data reporting system; a review of internal and external literature, including various assessment studies; a survey of refuge managers; and observations from field visits to 18 refuges.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The surveys will be self-administered and web-based. Respondents will complete survey questionnaires online. Once a completed questionnaire is submitted, response data will automatically populate an existing data file. This method allows for efficient administration of the survey instruments and greatly facilitates the management and eventual analyses of the survey data. Both open-ended and close-ended questions will be used for the questionnaires.

The surveys will be administered to one individual – the director or his/her designee – from each organization selected for the sample. It is intended that all FWS local partners will be provided an opportunity to complete the survey.

The surveys will be “open” (i.e., online and available for completion) for a 1- to 2-week period.

The surveys are not statistically representative surveys and will not be generalizable to the overall population of NWRS partners. The sample selection for partners includes all active partner groups that are classified as refuge support organizations; i.e. refuge Friends groups. In terms of the survey of State fish and wildlife agencies, representatives from all 50 States will be surveyed. In terms of tribes, a selection of survey participants was made based on the five tribes that are most active with the refuge system within each of the Service’s eight geographic regions. There is no sample stratification for any of the surveys as each survey treats the survey respondents as a single entity/survey cell.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Based on previous similar surveys for other agencies, we expect the response rate to be at least 80 percent. Organizations/groups that partner with refuges are inherently interested in conservation and most are willing to participate in surveys that will help inform management of related issues. The NWRS has not previously undertaken a survey of this nature. However, when MSI previously undertook a Visitor Satisfaction Survey for the refuge system in 2002 the response rate was 79.6%.

At least 1 week prior to commencement, we will send the following e-mail to viable respondents to make them aware of the upcoming surveys:

“Dear Partner of the National Wildlife Refuge System: The National Wildlife Refuge System (NWRS) is preparing to gather input from partners on how effective the NWRS is at accomplishing its mission. You and/or your organization have been identified as a recent partner of the NWRS, and we would appreciate your input into this evaluation. Within a week you will receive an email from me asking you to participate in the survey and giving you the online URL to access the survey. I hope you can take a few minutes to complete this survey and help us understand how we can improve the NWRS. Sincerely, Geoff Haskett, Chief of the National Wildlife Refuge System”

We will send the following e-mail once the survey periods are open:

“Dear Partners of the National Wildlife Refuge System: The National Wildlife Refuge System (NWRS) of the U.S. Fish and Wildlife Service is seeking input from our partners and friends in wildlife and natural resource conservation.

“We have contracted with an independent firm, Management Systems International (MSI), to conduct an independent evaluation of the overall effectiveness of the NWRS. MSI needs your opinion of the quality of partnership that exists between our two organizations and your views of how effective we are at achieving our objectives.

“The weblink below takes you to an online survey through which you can provide this valuable input. Your feedback is very important to us because it will help us identify our strengths and weakness, thereby allowing us to better serve you and other stakeholders in the future. I hope you can find a few minutes to compete the survey and help make the NWRS a more effective organization. Geoff Haskett, Chief of the National Wildlife Refuge System”

All responses to this survey will be collected online. It is our understanding that all NWRS partners and State agencies have e-mail accounts, and the tribal contacts provided by NWRS all have e-mail addresses. If the e-mail addresses are incorrect, there will be no response from that particular organization. Since the surveys are not intended to be a representative or generalizable this will not affect the survey’s utility. No password will be necessary to access the survey.

To increase response rates, we will send the following e-mail to survey participants mid-way through the survey time period reminding participants to complete the survey:

“This is a reminder to request that you complete the Partner Survey being conducted to assess the effectiveness of the National Wildlife Refuge System. If you have not already completed the survey, we would appreciate you doing so. The information from the survey will be used in combination with other data to determine the effectiveness of the NWRS and to identify opportunities to strengthen performance. Your feedback is very much appreciated. Geoff Haskett, Chief of the National Wildlife Refuge System”

The open nature of the surveys allows participants to answer when they have the time. The surveys being online also facilitates ease of response as participants need not keep track of physical materials.

Approximately between 80 and 90 percent of the data points collected on the surveys will be close ended for ease both at the answering and analysis phases. The remaining percentage of questions will be open-ended and will ask participants to indicate why certain selections were made and provide them an opportunity to make general comments. All open-ended questions are optional for respondents to answer.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The contractor has used similar survey questionnaires and associated sampling procedures for other agencies. These surveys achieved high response rates and provided useful and usable information for informing managers about planning and management issues. The previous surveys constitute important tests of procedures and methods that we will use in the proposed surveys.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Automated software will record and tabulate responses. Any further cross-tab analysis will be conducted by Management Systems International staff.

Keith Brown will be consulted on aspects of design and statistical aspects.

Keith Brown

Technical Director

Management Systems International

Washington, DC 20024

(202)-484-7170

4