SS PartB-5-14-09

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

NIST will use statistical methods to generalize the results of the information collection to the universe of U.S. fire departments. We began this stage of selection by identifying an appropriate sampling frame. The 2007 National Directory of Fire Chiefs & EMS Administrators (2008) was obtained from the National Public Safety Information Bureau for use in departmental sampling. The sampling frame contains contact information as well as the total population served by the department. Volunteer and other ineligible departments and offices were removed prior to processing. Volunteer and other ineligible departments represented about 4/5 of all departments in the 2007 National Directory.

We then conducted an assessment of the approximately 6,200 departments that made up our sampling frame. Departments serving small communities (with populations ranging 2,000 to 6,200) represent a total of 2.2% of the U.S. population served by our non-volunteer departments, yet comprise a quarter of all departments in the frame. The cost associated with selecting and processing the low volume of data from these departments would not be as beneficial as that of sampling the larger departments (and we note that covering departments serving communities with as few as 9,000 population would adequately reflect small department activity). In consequence, we removed from our sampling frame all departments serving 6,200 and less population served. As a result, a total of 4,656 departments in the 50 states plus Washington, DC remained in sampling frame.

The sample of departments was determined in two steps – (a) identification of ‘self-representing’ departments, and (b) the selection of the remaining “non-self-representing” departments. Self-representing departments are those whose populations served (and thus annual fire/EMS event volume) is so large that they would appear in any random sample with certainty. To understand why self-representing departments are unavoidable in our probability-proportional-to-size (pps) sample, note that a sample of 494 departments would cover in aggregate the 227 million population served. Accordingly, each sampled department will “represent” (227 million)/494 or about 460,000 population. Thus, any department whose population served exceeds 460,000 would be sampled with certainty. It is conventional in survey sampling to use 75% of this ratio as a threshold for determining self-representing selections. We used this approach to identify 76 self-representing departments, all with ‘population served’ exceeding 350,000. The remaining 418 non-self-representing departments were stratified by population served and sampled with probabilities proportional to population served. Combined with the 79 self-representing departments, this yielded our total desired sample size of 494.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The principal analyses will utilize a two-tailed statistical test of hypothesis comparing either two levels of staffing (e.g., 2- vs. 3-person teams) or two levels of response time (e.g., less than vs. greater than 6 minute response time). Furthermore, we sought to detect a 10% relative difference using a difference of means test with a 5% significance level and 80% power.

For fire events our power analyses used the following outcomes

• property damage (in dollars);

• numbers of worker and civilian injury or death.

And power analyses for ALS events utilized:

• breathing rate;

• pulse;

• blood pressure (both systolic and diastolic).

The final stage of sampling involves the selection and entry of data for eligible events occurring at each sampled department during a pre-specified field period. Departments will be fielded sequentially in replicates (i.e., random subsamples) over a three- to four -month data period, starting with a release of a subsample of departments comprising about 10 percent of the total sample. A single sworn staff person from each department will be trained on identification and entry of eligible events. A web-based data entry protocol will be used. Ideally, we seek about 83 events sampled per department per event type (i.e., fire and EMS). The actual sample sizes from departments will depend on the department stratum (self-representing vs. non-self-representing), and the volume of eligible events.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Distribution of the opportunity for project participation to the universe of fire departments will be conducted by each of the project participants, which represent a broad, objective coalition of respected organizations, including the Commission on Fire Accreditation International, the International Association of Fire Chiefs, the International Association of Firefighters, NIST, and Worcester Polytechnic Institute. A significant literature review and project participation from nationally-renowned fire chiefs and tacticians during the first year of this project has identified all elements of community risk assessment in an effort to minimize specification error in the model. The significance of the coalition of project team members is expected to increase fire department response rates. In addition to training the respondents in the study objectives, data entry procedures, and intent of the data elements, the data collection will use reliable, objective measures of risk and mitigation strategies. Existing fire department measures of data elements will be preserved wherever possible to minimize reporting error or confusion during data entry.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

N/A

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Jason Averill, NIST, (301) 975-2585

Robert Santos, Urban Institute, (202) 261-5904

3