0693.0033.SuppStmt.030213

SUPPORTING STATEMENT

U.S. Department of Commerce

National Institute of Standards and Technology

Generic Clearance for Program Evaluation Data Collections

OMB CONTROL NO. 0693-0033

A. JUSTIFICATION

This request is to extend the Office of Management and Budget (OMB) approval of the information collection.

1. Explain the circumstances that make the collection of information necessary.

In accordance with OMB's regulations at 5 CFR 1320 - implementing the Paperwork Reduction Act, the Government Performance and Results Modernization Act of 2010, and its’ mission

“to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life”

the National Institute of Standards and Technology (NIST), a non-regulatory agency of the Department of Commerce (DOC), proposes to conduct a number of data collections under this Generic Clearance for Program Evaluation Data. The information collections will be designed to evaluate current programs from a customer’s perspective that will provide customers with the opportunity to express their views on NIST programs. This data will assist NIST in improving the programs’ processes and services to its customers.

These types of data collection efforts will include but are not limited to questionnaires, interviews, focus groups, electronic methodologies, empirical studies, and video and audio data collections.

NIST will limit its inquiries to data collections that solicit strictly voluntary opinions and information, and will not—under this PRA clearance request—collect information that is required or regulated.

For each proposed request using the generic clearance; NIST will submit the actual instrument to OMB along with responses to the following questions:

1. Explain who will be surveyed and why the group is appropriate to survey.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

3. Explain how the survey will be conducted, how customers will be sampled of fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

4. Describe how the results will be analyzed and used to generalize the results to the entire customer population.

The collection instruments in ROCIS have been approved previously and are ongoing.

Below is a summary of lessons learned from each individual ongoing ICs:

NIST Small Business Innovation Research Program (SBIR) Commercialization Survey

FY07 Phase 2 and FY09 Phase 1 awardees were contacted and asked survey questions to determine the status of their projects and progress toward commercialization. All awardees, except one, participated in the information collection. The information provided by awardees improves the ability of the SBIR Program Office to better anticipate and address issues and problems that may face future awardees.

Summer Institute for Middle School Science Teachers

Focus groups and informal feedback of teachers who have completed the program have provided the basis for improvements to the annual program to date. The addition of the pre-/post- surveys to obtain more rigorous program assessment has indicated that the NIST Summer Institute is meeting many of its stated goals for participating teachers. These goals include: increased understanding of the subjects they teach; increased understanding of how scientific research is performed; materials and resources to implement what they learned at NIST in their classrooms; increased enthusiasm for science, and a network of scientists and engineers at NIST with whom to consult. The survey provided measures for professional impact, in probes on how the NIST Summer Institute met their needs as a professional development program. This was measured by a number of factors, from increasing teachers’ commitment to learning, increasing capacity to provide engaging activities for students, to increasing enthusiasm for science. Comparing the pre- and post-Institute survey responses, there were several statistically significant differences in participants’ sense of preparation and sense of efficacy in the classroom. The findings indicated that teachers increased their knowledge of current issues in scientific research, gained a greater understanding of everyday applications of science and technology, increased their knowledge of science-associated careers, and acquired greater understanding of fundamental science concepts. In addition, based on the survey, teachers were very positive in their descriptions of their students’ engagement with the NIST-based lesson, activities and materials that they brought back to the classroom. It is important to continue this more formal approach of evaluation, for a continuing rigorous program assessment with comparison measures.

Office of Weights and Measures Training Evaluation Survey

Responses from participants of training events obtained using this collection have shown that students highly value the training and generally are very satisfied with the quality of the course materials and instruction. Students are frequently motivated to provide constructive and detailed written ideas that are intended to improve the course for future students.

Data collected from the training surveys have been used to modify the duration (course length), technical content (scope and level of technical material), and format (balance between lecture and participant-centered exercises) of our classes to better meet participant needs and achieve the learning objectives for our classes.  Examples include developing in-depth and condensed versions of the same topic area; establishing prerequisites for some classes to better match participants’ abilities with course level; incorporating opportunities for students to share anecdotal examples to illustrate the application of subject matter; and increasing development and inclusion of additional “hands-on” exercises to supplement lecture-based instruction.

As an example of one key finding from the surveys, students in Retail Motor Fuel Dispenser training classes consistently express a need for more detailed information on audit trail and sealing features that is specific to the models of devices they encounter in the field. The Office intends to act on this finding by reinforcing this section of the training materials and by continuing to invite and encourage industry representatives to participate using hands-on demonstrations.

National Institute of Standards and Technology (NIST) Forensic Database Survey

To date, survey efforts have yielded 218 unique databases, reference materials, and standard reference collections used by forensic scientists in laboratories at the state and local level.

The survey at the federal level covered 165 contacts from 93 laboratories and locations at

33 agencies. The survey at the state level covered 72 contacts representing 158 laboratories and 47 states.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

In general, the data collected by NIST will be used to help plan the direction of future research in a variety of activities and to improve program management. The information collected will not be disseminated to the public but will be used to support research published in journals and conferences. All information collected will be stored anonymously and there will be no attribution to individuals in the analyzed data.

NIST has no plans to disseminate or release this information. If this situation changes, NIST will ensure that the information complies with the NIST Chief Information Officer (CIO) Information Quality Guidelines and Standards.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

Wherever possible, improved information technology will be used to reduce burden on NIST’s customers. In addition to traditional data collection methods, NIST will, whenever possible, offer electronic response via the NIST Internet web site and by fax.

The planned use of transactional and electronic web site surveys will substantially contribute to the number of projected responses and associated burden hours.

4. Describe efforts to identify duplication.

NIST established an internal review process that will examine each survey or data collection effort to be conducted under the generic clearance, to prevent internal duplication of effort and to ensure that appropriate data collection instruments are developed. By examining the three-year, NIST-wide requirements and combining those requirements into a generic clearance request, NIST is, to the greatest extent possible, centralizing the administration of its program evaluation data collections. This will provide for a more consistent and comprehensive approach. While there may be other surveys or data collections that become the subject of separate clearance

requests, NIST is confident that the procedures in place ensure that there will be no duplication of effort. Due to the nature of NIST’s unique mission and programs to further the mission, no similar data exists.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

Some small businesses or other small entities may be involved in these efforts, but NIST will keep the burden to them, as well as on any other business, organization, or individual at a minimum by asking for only the minimum information needed to evaluate programs on a strictly voluntary basis.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If these surveys and other data collections were collected less frequently, NIST will be unable to gauge the programs for present and future use. If the collections were not conducted, the following would not be possible: (1) the results from these data collections may lead to further investigations that could result in changes to, or enhancements of, the administration of programs, as well as identifying high-priority items for improvement; (2) the collection of economic data that will shed light on the productivity of NIST programs and their role in achieving NIST’s mission. There is no technical or legal obstacle to reducing this burden.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

Not applicable.

8. Provide a copy of the PRA Federal Register notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their

views on the availability of data, frequency of collection, the clarity of instructions and

recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

No comments were received as a result of the publication of the Federal Register Notice,

Vol. 77, No. 103, pg. 31584 on Tuesday, May 29, 2012.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

NIST will provide no payment or gift to respondents to any written, telephone, comment card, or other such survey. For focus groups professionally conducted by a contractor, NIST will, on a case-by-case basis, consider modest remuneration for the participant’s time and travel. In such cases, the remuneration will range from $50-$75 per individual. Remuneration for focus group participation is a recognized and standard industry practice, without which it would be difficult to achieve appropriate and adequate participation and response and to obtain reliable information.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

No assurances of confidentiality will be given.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

No sensitive data will be collected.

12. Provide an estimate in hours of the burden of the collection of information.

The request for the annual burden hours for the three-year period is 3,022.

13. Provide an estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection (excluding the value of the burden hours in

Question 12 above).

There are no known costs to the respondents.

14. Provide estimates of annualized cost to the Federal government.

It is difficult to estimate the costs in advance. The availability of funds and staff resources will impact NIST’s ability to conduct all the proposed activities. Consequently, it is uncertain of the number of activities that will be undertaken. Most of the surveys will be conducted and analyzed using in-house resources, though NIST has intent to contract some of the surveys to be conducted.

15. Explain the reasons for any program changes or adjustments.

Not Applicable.

16. For collections whose results will be published, outline the plans for tabulation and publication.

The results from these data collection activities are not intended for general publication, but may be disseminated to NIST staff, key policy and management officials, and stakeholders.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

All written and electronic material will display the expiration date for the OMB approval of the information collection. All written and electronic surveys (with the exception of transactional surveys in the form of business reply cards where space will not permit) will also display the following notification:

This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB). Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number. Public reporting burden for this collection is estimated to be XXAdd-Number of Minutes per ResponseXX minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed and completing and reviewing the collection of information. Send comments regarding this burden estimate or any aspect of this collection of information, including suggestions for reducing this burden, to the National Institute of Standards and Technology, Attn: XXAdd-Individual-Point-of-Contact_InformationXX.

18. Explain each exception to the certification statement.

NIST does not require any exceptions.

7