Supplemental and Part B Questions

Supplemental Questions for DOC/NOAA Customer Survey Clearance (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

The customer feedback survey is being conducted by the Economic and Social Sciences Research Program (ESSRP), Alaska Fisheries Sciences Center, National Marine Fisheries Service (NMFS). The lead staff member for the survey is Dr. Brian Garber-Yonts (phone: (206) 526-6301; email: [email protected])). The survey questionnaire has been developed based on NOAA’s generic customer satisfaction questionnaire (OMB Control Number 0648-0342), selected and adapted for application to ESSRP’s Economic Status Report publications and associated users. The primary mission of ESSRP is to provide economic and sociocultural information that will assist NMFS in meeting its stewardship responsibilities. Activities in support of this mission include: (1) collecting economic and sociocultural data relevant for the conservation and management of living marine resources; (2) developing models to use that data both to monitor changes in economic and sociocultural indicators and to estimate the economic and sociocultural impacts of alternative management measures; (3) preparing reports and publications; (4) participating on NPFMC, NMFS, and inter-agency working groups; (5) preparing and reviewing research proposals and programs; (6) preparing analyses of proposed management measures; (7) assisting Alaska Regional Office and NPFMC staff in preparing regulatory analyses; and (8) providing data summaries. 

Two of ESSRP’s primary products are the annual Economic Status Reports, published as supplements to the annual Stock Assessment and Fishery Evaluation (SAFE) Reports for Groundfish Fisheries Off Alaska and King and Tanner Crab Fisheries of the Bering Sea and Aleutian Islands Regions, respectively. The Groundfish Economic Status Reports have been published annually since 1991 and provide the primary vehicle for dissemination of statistical information regarding economic status and trends in the Alaska groundfish fisheries managed under the North Pacific Fisheries Management Council’s Fishery Management Plan for Groundfish. In recent years, ESSRP has been developing the Economic Status Report for crab stocks managed under the Council’s FMP for BSAI crab stocks as well. Both documents are intended for distribution as appendices to the FMP SAFE Reports, produced annually for the Council.

Similar to the FMP SAFE reports, the Economic Status Reports are intended for use by direct users of the report documents and support website: a broad array of individuals, for the most part representing public and private institutions concerned with fishery management. This includes NMFS, Alaska Department of Fish and Game, NPFMC staff, other public agency staff involved in production of regulatory and environmental review analyses for NPFMC, NMFS Alaska Region, and at other levels in federal and state fishery management; representatives of the fishing industry and non-governmental organizations working in the Council arena and elsewhere within federal fisheries management; and academic researchers as well as members of the general public.

NMFS Alaska Fisheries Sciences Center socioeconomics research staff implementing the survey do not have any independent source on the size or composition of this population. Both Economic Status Report documents are published principally as electronic documents for distribution with the FMP SAFE Reports, and are available in the form of downloadable Adobe PDF files on the ESSRP website. In addition to pdf documents formatted for print reproduction, ESSRP has begun development of online tools for dissemination and delivery of information from the reports in graphical and tabular formats to provide greater utility to users.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? Statistics? What suggestions did you get about improving the survey?

The survey questionnaire has been developed based on NOAA’s generic customer satisfaction questionnaire (OMB Control Number 0648-0342), selected and adapted for application to ESSRP’s Economic Status Report publications and associated users. To aid in the development of both data and analytical content of the Economic Status Reports as well as improved electronic access to report content, it was determined by ESSRP staff that feedback from users of the reports should be obtained to identify unmet information needs relevant to users and falling within the scope of the reports to help prioritize items for development. Because the reports are distributed through a variety of venues, ESSRP does not currently have any means of gauging the population of report users, and information on user demographics and other characteristics is therefore also of interest.

The two primary purposes of the survey are to gain basic information regarding the distribution of report user-groups (by affiliation and avidity), and solicit suggestions and priorities for report development and improvement. The appropriate generic questions (from both the quantitative/categorical response and open-ended sets of questions) were identified and modified as needed to specifically address the Economic Status Reports and associated users. Survey revisions from the generic questions were developed internally, with discussion and contributions from all ESSRP staff. No formal pretesting has been conducted.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

The target population for the survey is comprised entirely of direct users of the Economic Status Report documents, and the 2011 edition of the report documents will be used as the initial means of contacting users and soliciting participation in the survey. We do not have any independent source on the size or composition of this population. However, we expect that direct users in a given year number 100 or fewer individuals.

All report users will be asked to participate in the survey. An invitation to the online version of the survey and a link to the survey URL will be presented on the ESSRP webpage for the Report, which is the principal means of accessing the report document for most users. The invitation text and survey URL, as well as a print-formatted copy of the survey, will be reproduced as part of the front matter of the Economic Status Report pdf document, with instructions for mail submission as an alternative to online survey participation. It is expected, however, that only a negligible fraction of report users have access only to printed form of the reports, and that virtually all of the users access the reports electronically (either by downloading from the ESSRP or other websites, or receiving the pdf by email) and have sufficient internet support to participate in the survey online, making this the preferred mode option.

We do not currently have sufficient basis for forecasting a response rate for the survey. Ex-post estimation of response rate will be based on the fraction of unique visitors to the report website, as measured by Google Analytics data captured by website administrators, that complete all or part of the survey. Given that the survey is addressed to report users, who inherently possess an expressed interest in the report and it’s contents, and the brevity and ease of access to the survey, we anticipate that greater than 50% of report users will participate in the survey.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

As noted above, the two primary purposes of the survey are to gain basic information regarding the distribution of report user-groups (by affiliation and avidity), and solicit suggestions and priorities for report development and improvement. This information is to be used internally by ESSRP scientists and report authors, in conjunction with direct outreach to key users within the NPFMC community (staff analysts, SSC members, and others), in developing the report content over time. Despite the small number of direct users of the Economic Status Reports estimated above, the reports are known to be used in the preparation of a wide variety of decision support analyses by researchers and analysts within NMFS and other fishery management agencies, as well private and academic analysts. As such, the potential

beneficiaries of improvements in the Economic Status Reports span a much larger population than direct users themselves.

The survey does not employ sampling, and analysis of survey results will not be principally based on inferential statistics. As noted above, ex-post estimation of response rate will be based on Economic Status Report website page view statistics accumulated automatically, relative to the number of completed survey responses. This will allow us to assess to a certain degree the presence of a systematic difference in response rate by top-level ISP domain from which the respective users access the reports (.gov, .edu, .com, etc). Interpretation of quantitative results from the survey will use this information to adjust for any realized bias. Responses to open-ended questions soliciting suggestions for report improvement will be coded for content, and coding results incorporated into quantitative analysis of categorical response data. Novel responses to open-ended questions suggesting potential improvements in the utility of the report, while not representative, may prove to be valuable contributions and will be evaluated individually as well.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The respondent universe for the Alaska Groundfish and BSAI Crab SAFE Economic Status Report User Feedback Surveys is comprised of the entire population of direct users of the report documents and support website: broad array of individuals, for the most part representing public and private institutions concerned with fishery management. This includes NMFS, Alaska Department of Fish and Game, NPFMC staff, other public agency staff involved in production of regulatory and environmental review analyses for NPFMC, NMFS Alaska Region, and at other levels in federal and state fishery management; representatives of the fishing industry and non-governmental organizations working in the Council arena and elsewhere within federal fisheries management; and academic researchers as well as members of the general public. NMFS Alaska Fisheries Sciences Center socioeconomics research staff implementing the survey do not have any independent source on the size or composition of this population. However, we expect that direct users in a given year number 100 or fewer individuals.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The respondent universe for the Alaska Groundfish and BSAI Crab SAFE Economic Status Report User Feedback Surveys is comprised of the entire population of direct users of the report documents and support website; the data collection represents a census of the population, and sample selection/stratification will not be employed.

Data produced by the survey is comprised of categorical responses to likert-scale satisfaction questions, binary responses to questions regarding respondent affiliation status, frequency category of report use (avidity), binary responses to use/nonuse by respondent of individual report content categories, and open-ended text responses. Summary descriptive statistics of total and proportional rate by response category will be calculated for the population as well as by subpopulation user groups (by affiliation and avidity). Interpretation of these results will address realized differences in response rate by respondent group relative to total survey frame as described above, and statistical results to correct for bias as appropriate. Open-

ended response text will be analyzed for content and coded as appropriate. If coded responses provide sufficient basis for quantitative analysis, code values will be incorporated into the statistical summarization.

As noted above, the two primary purposes of the survey are to gain basic information regarding the distribution of report user-groups (by affiliation and avidity), and solicit suggestions and priorities for report development and improvement. The benefit of this information does not depend on a high degree of statistical accuracy.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

All Economic Status Report users will be asked to participate in the survey. An invitation to the online version of the survey and a link to the survey URL will be presented on the ESSRP webpage for the Report, which is the principal means of accessing the report document for most users. The invitation text and survey URL, as well as a print-formatted copy of the survey, will be reproduced as part of the front matter of the Economic Status Report pdf document as well, with instructions for mail submission as an alternative to online survey participation. It is expected, however, that only a negligible fraction of report users have access only to printed form of the reports, and that virtually all of the users access the reports electronically (either by downloading from the ESSRP or other websites, or receiving the pdf by email) and have sufficient internet support to participate in the survey online, making this the preferred mode option.

Given that the survey is addressed to report users, who inherently possess an expressed interest in the report and it’s contents, and the brevity and ease of access to the survey, we anticipate that greater than 50% of report users will participate in the survey. In public presentations of Economic Status Reports, (e.g., to NPFMC’s Scientific and Statistical Committee, FMP teams, etc), audience feedback will be solicited and directed to the online survey, and survey participation will be promoted in other available venues as appropriate.

The survey frame represents a census of the population. As such, the survey does not employ sampling, and analysis of survey results will not be principally based on inferential statistics. Ex-post estimation of response rate will be based on the fraction of unique visitors to the report website, as measured by Google Analytics data captured by website administrators, that complete all or part of the survey. This will allow us to assess to a certain degree the presence of a systematic difference in response rate by top-level ISP domain from which the respective users access the reports (.gov, .edu, .com, etc). Interpretation of quantitative results from the survey will use this information to adjust for any realized non-response bias.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The survey is using questions from NOAA’s generic customer satisfaction questionnaire, selected and adapted for application to ESSRP’s Economic Status Report publications and associated users. Adaptation of the generic questions was completed and reviewed by ESSRP staff with extensive experience in survey design and development. Online implementation of the survey will be tested to ensure compatibility with the full range of common software platforms.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

All survey design, administration, and analysis has been and/or will be performed internally by AFSC/Economic and Social Sciences Research Program scientists. The principal contact is Dr. Brian Garber-Yonts, 206 526-6301. Staff consulted in the development of the survey include:

Dr. Ron Felthoven, 206 526-4114

Dr. Stephen Kasperski, 206 526-4727

Dr. Amber Himes-Cornell, 206 526-4221

Dr. Ben Fissel, 206 526-4226