Supporting Statement B SafetyCodes Survey

Supporting Statement

U.S. DEPARTMENT OF ENERGY HYDROGEN PROGRAM KNOWLEDGE AND OPINIONS ASSESSMENT OF SAFETY AND CODES OFFICIALS

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent universe and selection methods

Potential respondent universe

The respondent universe for the survey of safety, codes, and standards officials includes the following groups:

• International Code Council (ICC)

• National Fire Protection Association (NFPA)

• National Association of State Fire Marshalls (NASFM)

• International Association of State Fire Chiefs (IAFC)

Data/respondent selection methods

The targeted sample for these officials includes all 50 state offices for each of the four groups. An advance letter will be sent to a named person in each organization. Sampling will be accomplished by calling the appropriate office and requesting the person to whom the letter was addressed. The designated respondent (i.e., the addressee of the letter) may assign someone else in the office to represent the agency if need be.

Expected response rate

One of the populations surveyed in 2004 was state and local government officials. The respondents for that survey were identified in advance, and an advance letter was sent to the pre-identified respondents. The response rate for the 2004 state and local government survey was 96%. That survey population is similar to the safety and codes population in two respects: (1) officials will be identified by their office responsibilities and (2) an advance letter will be sent to the pre-identified individuals. Because of this similarity, it is expected that the response rates will be similar to the response rates of the 2004 survey of government officials (i.e., over 90%).

2. Procedure for collection of information

Statistical methodology for stratification and sample selection

Safety and codes officials compose a well-defined target population and will be explicitly identified.

Estimation procedure

Estimates and other statistical analyses will be computed using the Statistical Analysis System (SAS). The analyses will properly account for the stratification as well as sampling weights. (Although a response rate greater than 90% is expected, sampling weights will be used, if necessary, to adjust for nonresponse.) In addition to standard summary statistics, exploratory analyses (e.g., contingency tables and correlations) will be performed to investigate possible relationships between responses to the various questions (e.g., performance on technical questions vs opinion questions, or differences attributable to responder age, sex, region, etc.).

Degree of accuracy needed for the purpose described in the justification

The 2004 data and analysis report, including estimates with standard errors and confidence intervals, is at http://www1.eere.energy.gov/hydrogenandfuelcells/hydrogen_publications.html (see links under “General/2004 Hydrogen Knowledge and Opinions Survey”). For example, the average score on the technical questions for the general public was 32.8% with 95% confidence limits of 31.3 and 34.4. The corresponding average score for state and local officials was 65.8% with 95% confidence limits of 65.4 and 66.2. These and many other statistical results have been used in the Hydrogen Education Program. They have been considered adequate but not excessive in statistical precision. Similar statistical analyses will be adequate for the safety, codes, and standards survey.

Unusual problems requiring specialized sampling procedures

None.

Use of periodic (less frequent than annual) data collection cycles to reduce burden

This survey is to be repeated every three or four years; it is not an annual survey.

3. Response rates and the issue of non-response

All surveys conducted in this information collection will employ CATI technology. All interviews will be conducted by professionals trained in interviewing techniques. Although a CATI interface will be used to facilitate sampling and dialing, potential respondents will interact only with the interviewers themselves, not the CATI interface (for example, respondents will not be asked to hold or push touch-tone keys).

Maximizing response rates

Procedures in this study for maximizing response rates will exceed usual standards for CATI surveys. Advance letters will be sent to a named person in each organization to be surveyed. Interviewers will make a minimum of 15 attempts to reach eligible respondents. Each call attempt will be a minimum of five rings. The CATI software will cycle the attempts to maximize coverage of the population. Lines that are busy will be called back a minimum of five times at 10-minute intervals. If the line is still busy after the fifth attempt, the number will be attempted again on different calling occasion(s). If the line is still busy after the fifth calling occasion, the CATI system will attempt to contact the phone company to ascertain whether the number is actually in service.

The surveys will be conducted to accommodate the special feature that relatively large proportions of “don’t knows” and “no opinions” are expected. (Changes in the proportions of people having an opinion on these questions are of particular interest.) Respondents will be assured that relatively few people currently know the answers and that “don’t know” and “no opinion” are perfectly acceptable answers. This will help to minimize item non-response rates. Opinion questions will be mixed with technical questions to minimize possible frustration with technical (hydrogen science) questions.

Methods for dealing with non-response

Statistical methods. A primary objective of the surveys is to measure changes in understanding and awareness about hydrogen and fuel cell technologies. Thus, statistics collected during the survey will be compared with results of the survey when it is repeated in 2011. Although non-response bias is always a potential problem in CATI surveys, it is a less serious problem in this survey, because of the tendency for non-response biases to cancel out in differences. That is, the non-response bias in the baseline and in the future survey are expected to be similar. When cross-year differences in estimates are computed, bias will tend to subtract out. Although the response rate for this survey is expected to exceed 90%, another method for dealing with non-response, which will be employed if necessary, is adjustment of sampling weights for differential response rates in the various survey strata.

In addition to these statistical adjustments, the following CATI procedures will also be employed to reduce non-response.

Call backs. CATI surveys accurately handle large numbers of scheduled call back appointments. When a scheduled appointment time arrives, the CATI system finds the next available station and delivers the record as the next call. Ensuring that appointments are kept helps to maximize response rates (and to minimize imposition on study participants). The CATI system also allows for callbacks to reschedule interviews and to restart interrupted interviews (for example if a respondent wants to terminate an interview but to finish it later). Scheduled call backs can be either casual (general time) or definite (exact time) depending on the respondent’s request. The CATI system also automatically handles callbacks for no-answer, busy, and answering machines. Call backs for busy signals are retried at several minute intervals; callbacks for no-answer and answering machines are scheduled to ensure coverage at different times of day.

Refusal Conversion. Another step that will be taken to maximize response rates will be to route each initial refusal to special survey staff trained and experienced at converting initial refusals to responses. How initial refusals are handled is decided on the basis of each particular case and the experience by the special staff member. A frequently successful technique that will be used is switching from an English-speaking to a Spanish-speaking interviewer.

4. Pretesting

Many of the technical questions on the safety and codes survey are identical to questions posed to other populations in the 2004 survey. Thus, the attached survey questionnaire has been thoroughly pretested for the most part. A few new questions pertinent to this specific population were added and these new questions will be tested prior to the survey being fielded.

5. Name and Telephone

Oak Ridge National Laboratory (ORNL) will oversee the survey and conduct the data analysis. ORNL contact: Richard L. Schmoyer, PhD, Statistics (1980), 865-576-5327, [email protected].

4

83I Supporting Statement, Hydrogen Knowledge and Opinions Assessment – Part B Statistical Method