Decision Science Generic Part B_revised

Justification –Part B of the Supporting Statement

Part B – This part must be completed for any information collection that includes surveys, research studies, or program evaluations in which the purpose is to make statistical generalizations beyond the particular respondents and to use the information for the description, estimation, or analysis of the characteristics of groups, segments, activities, or geographic areas. Surveys include both censuses (i.e., all members of the group of interest are asked to submit information) and sample surveys (i.e., only a subset of all members are asked to submit information). Research studies and program evaluations may involve the comparison of groups or testing of hypotheses about the impact or effectiveness of a program or the relation among variables, but the intention is to describe, estimate, or analyze the characteristics of groups or subgroups and draw inferences that can be generalized or applied beyond the particular observed sample. For more information, see OMB’s Standards and Guidelines for Statistical Surveys and Questions and Answers when Designing Surveys for Information Collections.¹

B1. Methods and Design.

The activities under this collection may involve samples of self-selected participants, as well as representative samples of affected community and industry members. In some cases, the use of convenience samples or quota samples may be made. Potential respondents will be selected to cover a broad range of individuals and entities related to specific aspects of decision science related to existing and emerging NIST mission and research goals such as cyber security, forensics, manufacturing, sustainability, hazard preparedness, emergency communication, disaster response, sustainability.

The specific sample planned for each individual collection and the method for soliciting participation will be described fully in each information collection request. In some cases, statistical methods will be employed to ensure that the sample reflects attributes of the population for which decisions are being analyzed. The number of individuals and entities participating in a given study will vary based on the nature of and needs addressed by the specific study.

In some cases, samples drawn from hard-to-reach populations will result in response rates under 80 %.

Under this collection certain protocols will be determined to be eligible for offering remuneration. Remuneration is justified due to:

(1) The complex study design of many collections requires ongoing participation of various respondents, each of whom is important to achieving study goals. In such studies, should attrition occur at a higher rate than expected, the study goals will not be met. Remuneration is a means to increase study participation and improve retention in longitudinal studies.

(2) There is a burden on the respondent to take time out of their workday in many cases. There will be equity in the use of remuneration: all respondents will be treated equally with regard to incentives as a means to compensate them for their time.

B2. Design of Data Collection Instruments.

Pretesting may be done with internal staff, a limited number of external colleagues, and/or individuals/entities familiar with the type of information sought. If the number of pretest respondents exceeds nine members of the public, NIST will submit the pretest instruments for OMB review under this generic clearance. Any pretesting requiring human subjects will be entered through the NIST Internal Review Board and the Research Protection Office.

B3. Collection of Data.

Data collection methods and procedures may vary and the specifics of these will be provided as supplemental documents for each information collection request focused upon decision science. NIST experts may use a variety of methodologies for these collections; and commercial software may be employed to automate decision science data collections and analyses of feedback. Some decision science collection instruments may also involve physical copies, in which case information collection instruments may be electronically disseminated and/or posted on target pages of a NIST web site. Telephone scripts, personal interviews, and focus groups may also be used and if applicable, will be provided as a part of the submission.

B3. Procedures for Quality Assurance and Quality Control and Evaluation.

Many of the information collections conducted under this generic will be completed by the participants themselves via an electronic platform, thus minimizing clerical errors on the part of federal staff conducting the information collection. Opportunities to amend or correct information will be noted within the public burden statement by showing participants who to contact to update information. In the event that a collection is done via an interview, or by some other means that does not involve an electronic platform for the participant to answer themselves, questions will be well articulated with ample time for response. As with electronic platforms, contact information will be given to ensure the participant may reach out if an amendment of information is necessary.

B4. Production of Estimates and Projections.

Estimates will generally by utilized for internal use of the agency in order to determine appropriate sample sizes. The use of any biased estimators will be identified and justified within individual collections under this generic clearance. Other statistical methodology concepts such as measuring sampling error / estimation error; weights applied to survey data to calculate target population estimates; and any development of model-based estimates will be noted in individual collections.

B5. Data Analysis.

Data collection plans will consider the minimum number of responses needed to provide accurate and reliable data and develop plans to obtain at least the minimum number of responses. Information collected under this generic clearance may not always yield generalizable quantitative findings; however, it can provide useful input concerning the context under which decision-making is conducted and considerations that are made by different stakeholder groups. Increased decision science understanding is critical to inform measurement science advancements that meet critical needs concerning topic areas germane to NIST’s core mission and emerging U.S. Federal priorities, including but not limited to community resilience planning, hazard preparedness, disaster response, cybersecurity, manufacturing, and sustainability. NIST may use generally accepted survey methods to minimize non-response, including, but not limited to tailoring the type, frequency, and nature of solicitation as well as offer remuneration to lessen burden and increase response rates and representativeness of samples.

B6. Review and Evaluation Procedures.

See above. This is a generic clearance which will be used for a myriad of decision science-based information collections. Statistical methodologies will range and include (but are not limited to) the responses in Questions 4 and 5.

B7. Data Dissemination.

Procedures for releasing and disseminating statistics and microdata (i.e., detailed responses for individual respondents) to the public may vary for individual information collections under this generic clearance but will be addressed in the supporting statements of the collections.

B8. Contact Person(s).

If statistical methods are used, NIST will obtain and provide information from the statisticians involved in the development, design, conduct, and analysis of usability data collections to be conducted, when appropriate. NIST will provide the name and contact information of the persons consulted in specific information collection requests submitted under this generic clearance.

1