The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the Form OMB 83-1 is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The respondent universe shall be derived from the users of the Lessons Learned Information Sharing Portal (www.llis.gov). The user group is approximately 29,960 individuals representing all geographical localities. The users span across all functional and jurisdictional levels. These individuals constitute the best available representation of the national community of emergency response providers and State and local homeland security personnel as well as account from relative proportions of the subgroups of interest (such as law enforcement, fire, medical, and emergency planners). The sample size will be large enough to ensure that comparisons can be made across categories and that the entire sample is representative of the community at large. DHS anticipates disseminating approximately 1,500 surveys.
The previous collection had a 97% return rate and a total of 2,477 respondents
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection,
Estimation procedure,
Degree of accuracy needed for the purpose described in the justification,
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The actual distribution of the survey will occur one week after as respondents are invited to visit a dedicated website to fill out the survey. The online collection mechanism will facilitate the collection and standardization of the data and provide immediate feedback on the response rates in each of the subgroups of the sample population. Response rates will be monitored throughout the data collection phase of this process. Surveys collected by phone and other follow-up methods will be used to boost response rates when deemed necessary to maintain the proportionality that is required for the sample to be representative of the entire population and each of its subgroups. All data collection activities will maintain the integrity of the dataset and ensure that validity of the study. Individuals will be answering these questions less than once during a 12 month period.
Data analysis will utilize accepted hypothesis testing methodology at the 5% level of significance ( p=.05 ). Statistical tests (e.g., Chi-squared tests and t-tests) will be used to determine if typical responses differ across population subgroups. Estimates of the standard error and covariance (in the event that variables are not independent) will inform the choice of methods for comparisons. Analyses will also account for the reliability of the survey instrument. There are no usually problems that require specialized sampling procedures.
As a preliminary step to validate the contact information of the respondent pool and to potentially increase response rates in the survey, a letter of introduction will be sent out via email to the respondent population one week prior to survey distribution. The letter introduces the survey, explains its purpose, provides respondents with an estimated time commitment, assures confidentiality, and outlines the way the results of the survey will be used.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collection based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
The survey distribution will be reinforced with phone calls, and preexisting relationships with LLIS users will be leveraged to encourage participation. These follow-up strategies can also be tailored to compensate for uneven response rates across subgroups, and to ensure that the analysis provides meaningful results.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
General measurements of effectiveness and user-friendliness will be relatively easy to produce once all the survey responses have been collected; however, additional analysis is needed to be able to focus policy choices and to maximize the value of the survey. Data will be analyzed on several additional levels: first, sentiments within in the national community of responders and state and local homeland security officials; second, statistically significant differences in the responses of the different categories included in this study (fire, law enforcement, medical, and emergency planners); and third, differences that emerge with the various jurisdictional subgroups that have been identified (municipal, State, county, regional). The goal of the analysis is to provide a clear picture of the effectiveness and user-friendliness of DHS programs and to identify specific opportunities for improvement.
Data analysis will utilize accepted hypothesis testing methodology at the 5% level of significance ( p=.05 ). Statistical tests (e.g., Chi-squared tests and t-tests) will be used to determine if typical responses differ across population subgroups. Estimates of the standard error and covariance (in the event that variables are not independent) will inform the choice of methods for comparisons. Analyses will also account for the reliability of the survey instrument.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Individuals who were consulted include: Chris Cihlar (DFI) 202-416-0135, Kevin O’Prey (DFI) 202-494-3124,
Sharon Kushnir (DHS) 202-282-9680, and Dr. Phyllis McDonald (JHU) 410-312-4400
5 CFR 1320.8, AGENCY RESPONSIBILITIES
Such office shall ensure that each collection of information: is inventoried, displays a currently valid OMB control number, and, if appropriate, an expiration date; is reviewed by OMB in accordance with the clearance requirements of 44 U.S.C. 3507; and informs and provides reasonable notice to the potential persons to whom the collection of information is addressed of ---
the reasons the information is planned to be and/or has been collected;
the ways such information is planned to be and/or has been used to further the proper performance of the functions of the agency;
an estimate, to the extent practicable, of the average burden of the collections, (together with a request that the public direct to the agency any comments concerning the accuracy of this burden estimate and any suggestions for reducing this burden);
whether responses to the collection of information are voluntary, required to obtain or retain a benefit (citing authority), or mandatory (citing authority);
the nature and extent of confidentiality to be provided, if any (citing authority); and
the fact that an agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number.
An agency shall provide the information described above as follows:
In the case of forms, questionnaires, instructions, and other written collections of information sent or made available to potential respondents (except in an electronic format), such information can be included either on the form, questionnaire or other collection of information, as part of the instructions for such collection, or in a cover letter or memorandum that accompanies the collection of information.
In the case of forms, questionnaires, instructions, and other written collections of information sent or made available to potential respondents in an electronic format, such information can be included either in the instructions, near the title of the electronic collection instrument, or, for on-line applications, on the first screen viewed by the respondent.
In other cases, and where OMB determines in advance in writing that special circumstances exist, agencies may use other means to inform potential respondents.
| File Type | application/msword |
| Author | DHS |
| Last Modified By | DHS |
| File Modified | 2007-01-24 |
| File Created | 2007-01-24 |