SUPPORTING STATEMENT
U.S. Department of Commerce
National Institute of Standards and Technology
Generic Clearance for Program Evaluation Data Collections
OMB CONTROL NO. 0693-0033
A. JUSTIFICATION
This is a request to extend, with changes, the Office of Management and Budget (OMB) approval.
1. Explain the circumstances that make the collection of information necessary.
In accordance with OMB's regulations at 5 CFR 1320 - implementing the Paperwork Reduction Act, the Government Performance and Results Modernization Act of 2010, and its’ mission “to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life” the National Institute of Standards and Technology (NIST), a non-regulatory agency of the Department of Commerce (DOC), proposes to conduct a number of data collections under this Generic Clearance for Program Evaluation Data. The individual information collections will be designed to evaluate current programs from a customer’s perspective that will provide customers with the opportunity to express their views on NIST programs. By collecting this type of information, NIST will be able to evaluate its performances as well as help determine the future direction of research to be conducted.
These types of data collection efforts will include but are not limited to questionnaires, interviews, focus groups, electronic methodologies, empirical studies, and video and audio data collections.
NIST will limit its inquiries to data collections that solicit strictly voluntary opinions and information and will not—under this PRA clearance request—collect information that is required (mandatory) or regulated.
For each proposed request using the generic clearance; NIST will submit the actual collection instrument and related documents (letters, emails to respondents, scripts, etc.) to OMB along with responses to the following questions (to be considered an abbreviated supporting statement):
1. Explain who will be surveyed and why the group is appropriate to survey.
2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.
3. Explain how the survey will be conducted, how customers will be sampled of fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.
4. Describe how the results will be analyzed and used to generalize the results to the entire customer population.
The following collections have been previously approved, are ongoing, and are part of this request.
NIST Small Business Innovation and Research Commercialization (SBIR) Survey
The survey results were used internally to provide additional insight on SBIR awardees that have completed the program. While no specific program changes were made as a result of the survey, the survey provided trends that have helped guide program improvements.
Outcome Survey for NIST Summer Institute for Middle School Science Teachers
The goal of the survey is to show how well/if the workshops meet the Institute goals as well as how relevant the topics covered are for the middle school curricula. The information gathered has help NIST improve the workshops offered and the presenters selected for the workshops.
Baldrige Examiner Training – Preparation Course Collection
The Baldrige Program trains 300-400 volunteer examiners each year to support the annual Malcolm Baldrige National Quality Award (Public Law 100-107). The training design leader, and all facilitators, receive the survey data so that the curriculum for the next year, as well as training approaches, might be improved. Recent improvement and changes due to feedback from survey data include a later start date for training each day and the addition of a third half day, an increased focus on key themes and scoring alignment, and an increased emphasis on how to write feedback-ready comments.
NIST Food and Nutrition Standard Reference Materials (SRM) Customer Questionnaire
Questionnaire results have been used to inform NIST program decisions regarding SRM development. Program changes have included a decision to reduce the number of assigned values on some SRMs based on these survey results (because it appeared that the values were not important to customers). Information was shared with NIST Office of Reference Materials and NIST is working to develop better data collection instruments for customers through sales processes.
Baldrige Executive Fellows – Post Program – Survey
The agenda/curriculum for each of the five Fellows on-site workshops are informed by receiving feedback from the Fellows. For recent workshops, changes made based on survey data included more breaks, a greater focus on recruiting manufacturing Fellows, a better understanding of expectations, and more clarity on the nature of the capstone projects. Suggestions for improving the fellowship from the survey are reviewed by Baldrige Fellows staff to prioritize and decide which to implement, with some suggestions to be implemented in coming years, as appropriate.
Baldrige Cybersecurity Excellence Builder Tool – Feedback – Information Collection
Data from the survey were used to improve the subsequent versions . Based on survey data, updates included more alignment to the next version of the Framework for Improving Critical Infrastructure Cybersecurity; emphasis on the cybersecurity supply network; and more clarification on the linkages among an organization’s context, processes, and results. From survey data, Baldrige staff also received a better understanding of who is using the tool and how to make it more user-friendly.
Baldrige Technical Editing Information Collection
Data from this survey are used to learn from technical editors who review feedback reports for the Malcolm Baldrige National Quality Award. By asking the technical editors how long they spent and what they found in the reports, The Baldridge Program can provide evidence-based inputs into the training for examiners writing the reports. Training and award process improvements are directly shared as inputs to the annual examiner training curriculum. For example, 2018 data showed that most technical editors had comments about the scoring alignment of the reports; this information was shared so that the training curriculum might focus more on scoring.
Baldrige Perception Survey
Data from this survey were used as inputs for the Baldrige Program leadership and advisory board, the Board of Overseers, who offer guidance on the improvement of the Malcolm Baldrige National Quality Award process and provide advice to the Baldrige Program and director of the National Institute of Standards and Technology (NIST). By understanding how the Baldrige Award is received by Baldrige Award recipients, the Overseers can better guide Baldrige Program leadership and the NIST Director. Data from this survey are also used to answer requests about the Baldrige Program and Award from the Secretary of Commerce and/or Congressional staffers.
Onboarding Program Outcome Evaluation
Feedback from the surveys is used to draft monthly reports for leadership on key outcomes of the agency’s Onboarding Program (ex. length of time to get badge, email, workstation). More specifically, NIST uses the data as programmatic feedback. Based on feedback, the NIST Onboarding Office makes programmatic changes dynamically and acts as a change-agent within the agency (ex. changes sequence/order of presentations, query about long delays in badging).
2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.
In general, the data collected will be used by NIST to aid and determine the planning and direction of future research for a variety of activities as well as to improve program management. The information collected will not be disseminated to the public but may be used to support research published in journals and conferences. All information collected will be stored anonymously and there will be no attribution to individuals in the analyzed data.
This information collection and dissemination will comply with the NIST Chief Information Officer (CIO) Information Quality Guidelines and Standards.
3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.
Wherever possible, improved information technology will be used to reduce burden on NIST’s customers. In addition to traditional data collection methods, NIST will, whenever possible, offer electronic response via the NIST Internet web site and by fax.
The planned use of transactional and electronic web site surveys will substantially contribute to the number of projected responses and associated burden hours.
4. Describe efforts to identify duplication.
NIST has an internal review process that will examine each survey or data collection effort to be conducted under this generic clearance, to prevent internal duplication of effort and to ensure that appropriate data collection instruments are being developed. NIST is confident that the procedures in place ensure that there will be no duplication. Due to the nature of NIST’s unique mission and programs to further the mission, no similar data exists.
5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.
Some small businesses and other small entities may be involved in these efforts. NIST will keep the burden to them, as well as on any other business, organization, or individual at a minimum by asking for only the minimum information needed to evaluate the NIST program(s) on a strictly voluntary basis.
6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.
If responses to these surveys and other data collections are not collected, NIST would be unable to gauge some of the programs for present and future use. If the collections are not conducted, then: (1) the results from these data collections may lead to further investigations that could result in changes to, or enhancements of, the administration of programs, as well as identifying high-priority items for improvement; (2) the collection of economic data may shed light on the productivity of NIST programs and their role in achieving NIST’s mission, would not be possible. There is no technical or legal obstacle to reducing this burden.
7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.
The data collection conducted under this generic clearance will be conducted in accordance with the guidelines in 5 CFR 1320.5.
8. Provide a copy of the PRA Federal Register notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their
views on the availability of data, frequency of collection, the clarity of instructions and
recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.
A 60-day Federal Register Notice soliciting public comment was published on March 5, 2019 (Vol. 84, Number 43, page 7880). No comments were received.
A 30-day Federal Register Notice soliciting public comment was published on June 3, 2019 (Vol. 84, Number 106, page 25534-25535).
9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.
NIST will not provide payment or gifts to respondents for any written, telephone, comment card, or responses to other such survey. In the case of focus groups, if respondents must leave their home or place of business to travel to a specific location, NIST will, on a case-by-case basis, consider a modest remuneration for the participant’s time and travel. In such cases, the remuneration may range from $25-$75 per individual, dependent on the data collection and the level and length of participation required. Remuneration for focus group participation is a recognized and standard industry practice, without which it would be difficult to achieve appropriate and adequate participation and response and to obtain reliable information.
10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.
Individuals will be assured of confidentiality to the extent permitted by law, including the Freedom of Information Act (FOIA).
The Privacy Act (5 U.S.C. § 552a €(3)) only requires a Privacy Act Statement when collecting personal information that will be placed or stored in a system of records. Each information collection under this generic clearance will be assessed for Personally Identifiable Information (PII) and how the information is stored and retrieved. If a collection is deemed to be part of a System of Records as defined by the Privacy Act, NIST will describe the appropriate System of Records Notice (SORN) and provide a Privacy Act Statement on the instrument.
11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.
No sensitive data will be collected.
12. Provide an estimate in hours of the burden of the collection of information.
The estimated totals for the three-year period of this collection is 40,000 Respondents and 20,000 burden hours.
13. Provide an estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection (excluding the value of the burden hours in
Question 12 above).
There are no costs to the respondents.
14. Provide estimates of annualized cost to the Federal government.
This is an umbrella submission. This Generic Clearance will involve 5 NIST employees that will devote a portion of their time, aside from normal duties, to plan, coordinate, administer, or monitor the overall types of individual information collections to be conducted. NIST estimates that it would involve an annual estimated total of 560 hours and an estimated cost of $125,000 per year.
NIST will describe individual cost estimates for each individual information collection request made. For example, if NIST determines that a Contractor will be involved in the collection efforts, those specific costs will be described.
15. Explain the reasons for any program changes or adjustments.
The Management and Organization (M&O) Office is responsible for overseeing the Paperwork Reduction Act (PRA) Program for NIST. M&O has been conducting extensive outreach and training to our NIST customers regarding PRA and Office of Management & Budget (OMB) requirements as well as the approval process. M&O's broad outreach efforts include discussions with NIST staff at all levels of the Agency in the use of information collections for their specific needs and guidance on how to proceed.
NIST is seeking approval to increase the annual respondents from 22,000 to 40,000 and annual burden hours from 10,000 to 20,000 under this renewal request. These increases of 18,000 additional respondents and 10,000 additional hours are requested to cover current and future information collections under this request for a three (3) year renewal and will ensure that NIST remains compliant with the PRA.
16. For collections whose results will be published, outline the plans for tabulation and publication.
The results from these data collection activities are not intended for general publication, but may be disseminated to NIST staff, key policy and management officials, and stakeholders.
17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.
All written and electronic material will display the OMB Control # and expiration date of the OMB approval for the information collection. All written and electronic surveys instruments (with the exception of transactional surveys in the form of business reply cards where space will not permit) will also display the following notification:
OMB Control #0693-0033
Expiration Date: 06/30/2019
This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB). Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number. Public reporting burden for this collection is estimated to be _______ minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed and completing and reviewing the collection of information. Send comments regarding this burden estimate or any aspect of this collection of information, including suggestions for reducing this burden, to the National Institute of Standards and Technology, Attn: XXAdd-Individual-Point-of-Contact_InformationXX.
18. Explain each exception to the certification statement.
NIST does not require any exceptions.
| File Type | application/msword |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |