0648-NWS 2025_ Supporting Statement Part B

SUPPORTING STATEMENT Part B

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Generic Clearance For NWS Risk Communication, Evaluation, and Feedback

OMB Control No. 0648-XXXX

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Data collection and estimation methods and procedures would vary among the individual collections the National Oceanic and Atmospheric Administration (NOAA) National Weather Service (NWS) would conduct under the requested generic clearance. Therefore, the information provided below for each of the five items in Part B cannot be very specific. A description of the plans for selecting respondents for each individual study will be provided to OMB at the time each instrument is submitted.

The appropriate respondent universe will be determined based on the goal of the specific study. For instance, a collection exploring public response to tsunami alerts may target members of the general public a certain distance from the coastline. Other collections will likely target NWS partners and stakeholders rather than the general public, since they serve as primary users and intermediaries in many situations. For example, a respondent universe could be US emergency managers. The sample within that universe would be selected in a way that ensures diversity in length of service, geographic location, available resources, and other characteristics.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Data collection procedures conducted under this clearance will be varied based on the goal of the individual collections, from foundational and exploratory (which will be more contextual and qualitative) to those that assess generalizability of a known phenomenon. Across studies, this will most likely include, ethnographies, participant observations, unstructured interviews, semi-structured interviews, structured interviews, focus groups, social network analysis, surveys, and survey-based experiments.

Within each study, the type and quality of the data collection and analysis procedures will be decided based on the specific goals of the individual collections and current resource capacities, which will be described to OMB for each collection. For example, work conducted in testbeds is constrained by physical space limitations and travel budget constraints, which will favor smaller sample and qualitative approaches. Post-event studies will face challenges reaching disaster survivors. A representative sample may be cost-prohibitive in some cases requiring a non-probability sampling strategy. Each application submitted to OMB will describe these constraints and, correspondingly, the limitations of the project including the types of conclusions and applications that will be appropriate based on the collection. It will not be appropriate to draw generalizable conclusions from small sample methods, but small sample methods could be appropriately used to inform subsequent studies and measure development for later quantitative work.

For qualitative work, best practices for data collection and analysis will include but are not limited to:

1. Ensuring a diverse sample of participants that accurately reflects the broader community being studied.

2. Where relevant, utilizing a mixed methods approach to ensure triangulation of findings.

3. Leveraging a rigorous deductive (theory-guided) or inductive (data-driven) approach for qualitative analysis depending on the situation.

4. Ensuring the sample size allows for sufficient thematic saturation.

For quantitative work, data collection and analysis will include consideration of response rates, item nonresponse rates, frequency distributions of data items to include cross tabulations, probability weighting, and reliability estimates. More specific information about data collection procedures will be contained in the description provided to OMB at the time each instrument is submitted. Generally speaking, quantitative studies will:

1. Employ cognitive testing of survey instruments where relevant and realistic.

2. When needed, will use weighting adjustments to ensure that the final dataset is reflective of the overall study population.

3. Ensure that the sample size selected is sufficient to include representation from the universe of relevant populations and allows for reliable statistical analysis.

4. Choose a universally accepted margin of error (between 4% and 8%) at a reasonable confidence interval (95%).

5. Determine and describe the best statistical test depending on whether the analysis is descriptive or inferential, whether the data are parametric or nonparametric, and whether the analysis is univariate or multivariate.

6. Calculate reasonable response rates to ensure the required number of completed responses is acquired.

7. Use multiple estimate procedures depending on the goals and strategy of analysis - point estimates may be used to estimate the population value for certain survey findings, whereas in other cases, an interval estimate may be used to identify a confidence interval or range of values.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Standard survey procedures, including multiple contact attempts distributed among different days and times, will be utilized to achieve sufficiently high levels of response. For interviews and focus groups, existing networks will be utilized where possible. For instance, the NWS Weather Forecast Offices (WFO) are deeply embedded in their local area and have established relationships with partners and stakeholders. Also, reminder phone calls and/or letters to participants will be used to encourage them to keep their appointments. More specific information will be contained in the description provided to OMB at the time each instrument is submitted.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Internal cognitive testing will be employed as a first step in identifying burdens and improving utility. For surveys, an external pre-test will also be used to ensure data are coming in correctly and that no changes need to be made to the coded survey. For focus groups and interviews, the facilitators and interviewers will conduct practice sessions with federal employees and/or nine or less non-federal individuals to ensure questions are relevant and understandable.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Advice on statistical aspects of each individual instrument will be sought from subject matter expertise as the research proceeds. Depending on the nature of the research, NOAA staff from different program areas will have the primary responsibility for data collection and analysis. Each specific research project will also determine whether the data will be collected by NOAA or through a contractor. Contact persons for questions regarding data collection and statistical aspects of the design will be provided to OMB at the time each instrument is submitted.