Supporting Statement PART B 1660-NEW nw51 1-22-10

B. Collections of Information Employing Statistical Methods.



1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The potential respondent universe is anyone living in an EPZ of selection in the United States. Specifically, the universe for a nuclear power plant’s alert and notification survey consists of all residential, non-institutional households within the EPZ selected. While this area is generally a 10-mile-radius circle, with the nuclear plant as the center point, it may sometimes include areas extending beyond 10 miles. The longitudinal and latitudinal coordinates of the nuclear power plant, along with a complete description of the size and shape of the EPZ, are used to identify the universe of households within the EPZ.

There are currently 64 commercial nuclear plants in the US, and therefore, 64 EPZ’s, the 10-mile areas around each of them.  There are also 6 new ones over the course of the next 6-7 years. These plants receive their licenses to operate from the NRC (Nuclear Regulatory Commission). This collection is asking for approval to sample an average of one EPZ per year. It is not expected that more than one survey of an EPZ would occur in a year and that there will be years where no survey may occur. No sample will exceed 384 respondents in the largest EPZ when estimating at a 5% precision at a 95% confidence level. When the samples are drawn, they are of one EPZ at a time and the EPZ’s are selected for the study in order of programmatic dependencies. The next EPZ to undergo programmatic changes and adopt a new system will be selected next.  The order of selection is dependent on programmatic changes that have yet to occur. 

Immediately following activation of a nuclear power plant’s alert and notification system as a part of a demonstration that the system meets the requirements of 44 CFR 3540.9(a), FEMA will conduct a telephone survey of a systematic sample of residents within that nuclear power plant’s emergency planning zone (EPZ). The purpose of this survey is to estimate the proportion of households within the EPZ alerted by a nuclear power plant’s alert and notification system.

The regulations require the telephone survey to be completed within one hour of the sounding of a power plants’ alert and notification siren system. As this data is collected immediately after the alert is sounded, we expect a high response rate of 80% or higher.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection and estimation procedure:

When drawing a sample for the telephone survey, the first task is to obtain a complete set of detailed street maps covering the survey area. These maps are usually obtained from county planning commissions or local governmental agencies. Next, a list of names and addresses, the sample, is obtained for residents of households within the EPZ. Given the longitude and latitude of the plant, along with details of the size and shape of the EPZ, computerized mapping techniques are used to obtain this information from a data base for households within the EPZ.

Depending on the characteristics of the area, households are selected via census tracts, block groups, enumeration districts, zip codes, or other geographic delimiters.

A base list of households, geographically ordered, is compiled, comprising all available addresses and telephone numbers identified with in the EPZ. From this base list, a sample of households is selected in a systematic (nth name) fashion, ensuring that the sample proportionately represents the entire EPZ (i.e., EPZ areas are represented in proportion to the number of households contained therein).

Although the techniques used to collect the universal sample are highly accurate, sometimes households on the list are located just outside the EPZ. Households determined by this check to be outside the EPZ are removed from the sample. The first step in preparing the sample for interviewing is to check each address on the aforementioned street maps to verify that it is, in fact, located with the EPZ.

For households located within the EPZ, hard copy sample labels are printed and affixed to sample call report forms. These sample pieces are then systematically (nth name) divided into mini-samples. Each of these mini-samples (commonly referred to as replicate groups) is proportionate and representative of the entire (universal) sample. After the mini-samples have been selected, they are shuffled into random order to ensure that the specific calling sequence does not introduce any bias into the results. Immediately following the alert and notification system activation, these sample pieces re release to interviewers in replicate groups, and the interviewers begin telephoning households within the EPZ. Since the exact number of sample pieces need to complete interviewing cannot be precisely determined in advance, the release of the sample in randomly shuffled replicate groups ensures that the final set of completed interviews is representative of the entire surveyed EPZ.

As interviewing progresses, interviewers key all responses into computer terminals, and running totals are kept on completed interviews for those households that were alerted and those households that were not alerted.

The target number of completed interviews varies from survey to survey, depending on the alerted/not alerted ratio. This target number is established to ensure that a sufficient number of interviews are conducted to provide survey results with 5% precision at a 95% confidence level. The computer program monitors the alerted/not alerted ration and performs a sample size calculation after each recorded interview. The number of interviews required is continuously updated and displayed to supervisory staff, allowing the sample to be released properly and the interviewing process to be terminated when the (required number of) interviews have been completed. The interviews are usually completed within one hour of the alert and notification system activation.

After the telephone survey has been terminated, all completed interviews are once again checked against maps to see if the surveyed households are located within the EPZ. During interviewing, respondents are asked to provide their address and the closest cross street or intersection to their house. Since respondents sometimes provide new addresses or a more exact location, these addresses are checked against street maps a second time. In addition, this second check identifies anyone who has moved or whose address has changed from the one obtained with the original sample. Before final tabulations are run, any households detected to be outside of the EPZ are removed from the universe of surveyed households.

SAMPLE SIZE DETERMINATION

The number of households that need to be surveyed is determined based upon the need to obtain a sample size sufficient to obtain a 95% confidence interval with precision (half-width) of 0.05 for the estimate of the proportion alerted. The exact number of households to be surveyed can be derived from the following statistical considerations. For relatively large sample sizes (n ≥ 30), taken without replacement from a population (N), the sampling distribution for proportions (e.g., the proportion of the population alerted) is nearly a normal distribution, the mean of which is the proportion (p) of the population alerted and the variance of which is

If P is the observed sample proportion, then for a particular confidence level with confidence coefficient z_c,

)

Thus, for this confidence level, the actual proportion of the population alerted satisfies the following inequalities:

Thus, the precision (W) is simply given by

This equation can be solved to determine the sample size (n) required to yield a given precision (W) with a given observed sample proportion (P) as follows:

Although this expression for n can be used directly, it is customary to make several approximations. First, since the term in N in the denominator (the finite population term) is positive definite for all reasonable values of W (0 < W < 0.5), omitting this term will result in an approximation to n that is slightly larger than its true value. This is an acceptable practice in sizing the sample since a larger sample gives greater precision.

A second approximation that can be made is to neglect the terms in W² within the bracket in the numerator. Analysis demonstrates that this underestimates n when P < 1/2– 1/ 4 or P > 1/2– 1/ 4 and overestimates n for P between those two values. For the case of interest (a 95% confidence interval with precision of 0.05), this approximation provides an overestimation of n when a sample size greater than 191 is required. Since the sampling plan calls for a minimum sample size of 250, regardless of the value of P, this approximation is acceptable because it also yields an estimate of n larger than the true value. Therefore, for the purposes of the pilot test and subsequent surveys, the following approximate equation can be used to determine whether a sample size larger than 250 is required:

Or using 1.96 for z_c and 0.05 for W,

N = 1536.64 P(l - P)

Data from the pilot test can be used to illustrate the effects of these approximations. In the pilot test, the population of tone alert households from which the sample was to be drawn (N) was approximately 4,500 and the observed proportion alerted (P) was 0.675. This yields 311 as the exact result for n.

Neglecting the finite population term yields an estimate of 334 for n and the simplified final approximation estimates n as 338. Thus, the final simplified approximation overestimates the required sample size by 27 in this case.

• Degree of accuracy needed for the purpose described in the justification,: This target number is established to ensure that a sufficient number of interviews are conducted to provide survey results with 5% precision at a 95% confidence level.

• The NRC regulations require a 94% degree of accuracy is needed for the purpose of ensuring the power plant sirens are heard with 10 miles of the power plant.

• Unusual problems requiring specialized sampling procedures:

There are no expected unusual problems requiring specialized sampling procedures beyond those described above.

The telephone survey provides prompt assessment of an alert and notification system's effectiveness following an activation and permits monitoring of responses during the sampling process in order to obtain enough responses to achieve statistically valid survey results. Special sampling provisions are considered in unique situations where large portions of the population within a nuclear power plant's EPZ may not have home telephones (e.g.,where religious beliefs prevent telephones). Other means of efficiently obtaining public survey information are considered on a case-by-case basis.

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden: There is no use of periodic (other than annual) data collection cycles to reduce burden.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

The regulations require the telephone survey to be completed within one hour of the sounding of a power plants’ alert and notification siren system. As this data is collected within one hour of the alert sounding, we do not feel there will be a low response rate and do not provide incentives for respondents to participate. Simply giving their input to the community regarding whether or not they heard the alarm is seen as their contribution and, for some, providing their input into the community is a personal incentive in and of itself.

We do not take extra methods to maximize response rates and to deal with issues of non-response. It is expected that these measures will help to maintain sufficiently high response rates suitable to analysis, but in the event of response rates falling below 80%, a non-response analysis will be performed on the group(s) in question. These analyses will be conducted by using the “SPSS Analysis of Missing Data” module of the general SPSS software package and the findings of the analysis will be addressed accordingly.

To maximize the validity of the data collected we will do the following:

a) Verify that we are speaking with the head of the households (first question when respondent answers phone).

b) Verify that we have contacted the correct household (Question 1 under “Introduction”).

c) Verify that address is correct (Question 6).

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Pilot Test

At the beginning of each survey collection period, a pilot test is conducted on no more than 10 persons to discover any potential problems with the survey instrument or process. For quality assurance purposes, data from the pilot is reviewed by us and improvements are made to the survey process as deemed necessary.

In addition, in 1985, a pilot test was conducted on this data. This was before the Paperwork Reduction Act of 1995 was issued, and therefore, was not a violation of the PRA. In the pilot test, the population of tone alert households from which the sample was to be drawn (N) was approximately 4,500 and the observed proportion alerted (P) was determined to be 0.675.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

FEMA-Information Resources Management Branch, IC-Records Management

Person 1: Nicole Bouchet
Records Management Division
Office of Management
Federal Emergency Management Agency
Attention: OM-RM
500 C Street, SW
Washington, DC 20472
Office: (202) 646-2814
Fax: (202) 646-3347

Email: [email protected]

FEMA – Radiological Emergency Preparedness Branch, Policy Section

Person 2: Deborah Mauldin

Radiological Emergency Preparedness

Technological Hazards Division

Federal Emergency Management Agency

Attention: TH-REP

500 C Street, SW

Washngton, DC 20472

Office: (202) 212-2127

Email: [email protected]

FEMA – Radiological Emergency Preparedness Branch, Policy Section

Person 3: Russell Gates

Radiological Emergency Preparedness

Technological Hazards Division

Federal Emergency Management Agency

Attention: TH-REP

500 C Street, SW

Washngton, DC 20472

Office: (202) 212-2196

Email: [email protected]

8