NOS - Community Rating System Supporting Statement Part B

SUPPLEMENTAL QUESTIONS PART B

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

DOC/NOAA Customer Surveys

NOAA Products and Services for Community Rating System Communities

OMB Control No. 0648-0342

COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

NOAA has designed the survey as a way of assessing trade-offs between information and response rates/burden to assess how future survey efforts should be performed. Specifically, NOAA has identified a minimum set of information it would like to collect under this survey. However, additional information from the respondents would also be valuable to assist NOAA in assessing how well its products and services and working. Thus, NOAA identified two areas of additional information that would be valuable:

• Number of NOAA products and services to include in the survey

• The extent of information collected on how the NOAA products and services contributed to earning CRS points

Number of NOAA products and services

NOAA and ERG identified 40 different NOAA products or services that could be used by CRS communities to improve their CRS scores. The project team categorized each product or service as being “high” (16 products and services), “medium” (12), or “low” (12) priority for this work. Next, the project team determined that the “low” priority products or services do not need to be included in the survey. NOAA then decided that 60 percent of the respondents should see just the “high” priority products and services and the remaining 40 percent would see both the “high” and “medium” priority categories. This approach allows us to determine if including the “medium” priority products and services significantly reduces the response rate. If so, future surveys may want to focus on the just the “high” priority products and services.

Detailed information on products and services

Understanding how the communities use the NOAA products and services is important to helping NOAA to either improve its existing products and services or to develop new products and services to fill potential gaps. The survey will ask respondents about their use of the products and services to earn CRS points. There are three potential levels of information that can be collected:

• In general, the extent to which the product or service was useful to earning points. This is the most basic level and provides little detail in how the communities use the NOAA products or services to earn points. NOAA and ERG have identified areas of the CRS Coordinators Manual where we think each product or service can assist CRS communities, but this link is based on NOAA and ERG’s best professional judgement. Nevertheless, asking respondents whether the product or service was “useful for earning points” is the least burdensome approach to assessing usefulness of the NOAA products and resources. This is the minimal level of information we envision collecting under this survey in terms of usefulness. We are proposing to collect this level of information from all respondents using a five-point scale (“not at all useful”, “slightly useful”, “moderately useful”, “very useful”, or “extremely useful”).¹

• The extent to which the NOAA product or service was useful for earning points under the four different Series in the CRS Coordinator’s Manual. The CRS Coordinator’s Manual is organized into sections (referred to as “Series”) and communities can earn points under four of those areas: Public Information Activities (300 Series), Mapping and Regulations (400 Series), Flood Damage Reduction Activities (500 Series), and Warning and Response (600 Series). Respondents who are asked this level of information would be asked to assess the extent to which NOAA products and services they rated as “very useful” or “extremely useful” in the overall usefulness question (prior bullet) were useful at earning points in each of the four series. We are proposing to ask this of 30 percent of all respondents.

• The extent to which the NOAA product or service was useful for earning points under the different sections within in the CRS Coordinator’s Manual. Within each of the Series in the CRS Coordinator Manual, there are “sections” that define specific elements where communities can earn points. There are 19 total sections² on the CRS Coordinator’s Manual where points can be earned. Respondents who are asked this level of information would be asked to assess the extent to which NOAA products and services they rated as “very useful” or “extremely useful” in the overall usefulness question (first bullet above) were useful at earning points in each of the 19 sections. We are proposing to ask this of 30 percent of all respondents.

Sample Allocation Among Distinct Groups

Table 1 provides the allocation of the (assumed) 800 coastal communities across the six distinct groups defined by the decisions made above. Specifically, there are two “priority levels” defining which products or services to include and three levels for “point usefulness”.

Table 1. Sample Allocation

Group	NOAA Products and Services Included in Survey	Points Usefulness Question	Sample Allocation [a]	Assumed Response Rate	Estimated Number of Respondents
A	Just High	General only [b]	192	60%	115
B	Just High	Series Level	144	55%	79
C	Just High	Section Level	144	50%	72
D	High and Medium	General only [b]	128	50%	64
E	High and Medium	Series Level	96	45%	43
F	High and Medium	Section Level	96	40%	38
Totals	-	-	800	51.4%	411

[a] Total respondents are assumed to be 800 coastal communities. First, 60 percent (480 communities) are assigned to “just high” for the number of NOAA products and services included and 40 percent (320 communities) to the “high and medium” category. Next, for the point contribution question, we assigned 40 percent to the “general question only” group, 30 percent to the “series level” question, and 30 percent to the “section level” question.

[b] It should be noted that all respondents are asked the “general” question; this group is only asked that question.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The survey will be implemented as a web-based survey that is sent to potential respondents via email. ERG will code the survey in its online survey platform licensed from Qualtrics. FEMA has agreed to provide NOAA with its most current list of CRS Coordinators that includes email addresses.

The survey is not using statistical methods and no statistical methodology is being employed.

There are no unusual problems requiring specialized sampling.

This is a one-time data collection effort. However, based on the results of the analysis to compare response rates and completion times, NOAA will assess how to implement this survey in the future.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

NOAA expects a 40-60 percent response rate (see Table 1) from this survey and will take the following steps to increase response:

• ERG will be following good survey design practices in this project. This includes sending a well-designed survey and sending reminders to increase response.

• As noted above, one aspect of the design is to assess whether collecting additional information would lead to lower response rates. NOAA will have its contractor for this work analyze the response rates associated with additional data being collected to determine the best approach for future data collections.

• Finally, many respondents may not be familiar with some of the NOAA products and services in the survey. Thus, OCM has determined that an appropriate incentive for the respondents is to provide a PDF that contains information on how to access the NOAA products and services mentioned in the survey upon completion of the survey. This accomplishes two things. Frist, respondents can be assured additional information they can use if they compete the survey. Second, providing the respondent an assurance they will be provided details on the products and services reduces the incentive for respondent to leave a survey mid-task to access a specific product or service mentioned.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

NOAA’s contractor, ERG, will perform a cognitive test of the survey with nine or fewer CRS Coordinators to assess the extent to which the survey will be easy to complete. ERG will send the survey to the nine (or fewer) respondents, ask them to complete it, and then perform a follow-up interview with the nine respondents that involves reviewing the survey with the respondent to assess issues in its design. ERG will then revise the survey following those nine interviews.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

NOAA has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the survey instrument and implement the survey. ERG’s project manager for this work is Lou Nadeau (781-674-7316; [email protected]).

1 As will be described in the next two bullets, 40 percent of respondents would only be asked this question in relation to the usefulness of earning points. The remaining 60 percent are equally allocated among the approaches in the next two bullets (30 percent each). Nevertheless, all respondents would be asked the general usefulness questions since we use that question to screen which NOAA products or services we ask about in the alternative approaches (see next bullets).

2 The sections fall within the Series described in the prior bullet.