Supporting Statement (3145-0136)

REQUEST FOR CLEARANCE

 

National Science Foundation

Directorate for Education and Human Resources

Division of Human Resource Development

Centers of Research Excellence in Science and Technology

Monitoring System (CREST)

Attachment A

Section A

Introduction

This request for Office of Management and Budget (OMB) review asks for a renewal of clearance of the monitoring data collection for the National Science Foundation (NSF) Division of Human Resource Development (HRD) Centers of Research Excellence in Science and Technology (CREST) program under the Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on March 31, 2011. The EHR Generic Clearance is for the collection of monitoring data on education and training activities supported by the NSF.

A.1. Circumstances Requiring the Collection of Data

The CREST program was initiated in 1987 to enhance the research capabilities of minority-serving institutions through the establishment of centers that effectively integrate education and research. CREST promotes the development of new knowledge, enhancements of the research productivity of individual faculty, and an expanded presence of students historically underrepresented in science, technology, engineering, and mathematics (STEM) disciplines. The program does this by:

 

·         Developing outstanding research centers that integrate education with research to produce new knowledge and increase the number of underrepresented minorities with Ph.D.s in STEM fields

·         Making substantial resources available to upgrade the research and education capabilities of the most productive minority institutions

·         Enabling faculty at these institutions to cooperate with other science and engineering activities

·         Building bridges for minority student career development through alliances with business, government laboratories, and other universities

 

Data collected from CREST centers through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary information for future impact assessment and evaluation activities.

A.2. Purposes and Uses of the Data

The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF's program, project and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.

The primary purpose of this collection is to provide data and information for effective program management and monitoring of program activities. This data collection activity is designed to track the extent to which CREST projects meet the objectives of the program. Within the HRD division, this information is used to administer and monitor the progress of the program. The findings are used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts.

In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort contributes to the formal evaluation of the program and provides regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.

“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”

These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.

The CREST program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the CREST monitoring system may be used to report to these committees on program activities. The materials developed for the September 2010 CREST COV included data about the program that had been collected in the CREST monitoring system over multiple years. In addition, NSF is required to measure the attainment of its program, project, and strategic goals, as identified by the President’s Accountable Government Initiative, by the Government Performance and Results Act (GPRA) Modernization Act of 2010, and by the NSF’s Strategic Plan. Data collected in the CREST monitoring system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.

Data are collected from CREST PIs using the CRESTWeb Data Collection System, an online system developed to facilitate electronic data collection. The three general categories of data that are collected are summarized below:

1.  Participant information.  Data are collected on the individual participants in the CREST program, who include principal investigators (PIs), faculty members, postdoctorates, students, other personnel, and contributors. These data include:

·        Contact information (name, address, phone number, e-mail address, etc.)

·        Demographic data (race, ethnicity, gender, disability, citizenship, etc.)

·         Academic information (academic level, year in school, academic rank, degree, field of study, etc.)

2. Project Implementation Data. Aggregate data are collected on the amount of financial support CREST awardees receive from various sources, including:

·        NSF CREST

·        Other NSF

·        Other Federal agencies

·        State government

·        Local government

·        Industry

·        University

·        Other sources

In addition, data are collected on number of students involved and on the CREST involvement of individual participants, including research thrust areas and financial support received.

3. Project accomplishment data.

Aggregate data, including title, year, thrust area, and a narrative description, are collected on each CREST center's: 

·        Proposals (CREST Centers/PIs are actively charged with seeking co-funding. These can include written responses to NSF-issued program solicitations and to other funding sources.)

·        Awards (Funding received as a result of a proposal submitted in response to a program announcement or solicitation)

·        Presentations (Events such as lectures or speeches that are presented to an audience)

·        Publications (Any research or technical papers, briefs, articles or other written products that were fostered or funded by CREST during the current reporting period)

·        Collaborative projects (A CREST initiative undertaken as a joint venture between a CREST-affiliated center and other outside entities. Collaborative projects can also be between two CREST centers.)

·        Patents (Patents awarded to patent applications previously filed with the U.S. Patent and Trademark Office or the European equivalent. In addition, patent applications are tracked, even if the patent application was rejected.)

·         Activities (Educational procedures designed to stimulate learning through firsthand experience)

A crosswalk of data elements can be found in appendix B.

A.3. Use of Information Technology To Reduce Burden

Like other tasks under this generic clearance request, CRESTWeb uses advanced electronic information technology to minimize data duplication and respondent burden.  EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. CRESTWeb data fields are marked with out-of-range indicators, warning respondents to check their data if they appear to be out-of-range. Web-based systems also employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden. Other efforts to reduce burden include automatically entering each center's contact information and the amount and dates of its award, and allowing users to import previously entered accomplishments to carry them forward to the current round of data collection.

The CRESTWeb system is compliant with Section 508 of the Rehabilitation Act.

A.4. Efforts To Identify Duplication

The CRESTWeb data collection does not duplicate other NSF efforts. Data collected under the CREST program are not available anywhere else, and questions are asked only if the information requested is not available elsewhere. Whenever possible, data are drawn from existing NSF databases, and NSF has examined its data collection requirements to ensure that the requested data are not available from other Federal sources. Changes to NSF’s FastLane system are monitored on a regular basis, and as new data elements are added to (or deleted from) the FastLane application and/or project reporting system, the instrument is modified accordingly.

A.5. Small Business

No information is to be collected from small businesses.

A.6. Consequences of Not Collecting the Information

Without this information, NSF would be restricted in managing and reporting on the activities of awards in the CREST program. Without this feedback, NSF would have no way of making systematic modifications to the CREST program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the CREST program. The information requested here is not available elsewhere.

 

Additionally, without this information NSF would find it difficult to meet agency data requests, as well as GPRA and OMB reporting requirements, and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.

A.7. Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6

The data collection will comply with 5 CFR 1320.6.

A.8. Consultation Outside the Agency

The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.

The PIs who will be using the CRESTWeb system were consulted in its design and planning. A pilot test was conducted before the system launched in 2002. Feedback from system users is also solicited at the annual CREST PI meeting, usually held in the spring, and user comments submitted during the collection period are taken into consideration for system improvements.

A.9. Payments or Gifts to Respondents

No payments or gifts will be provided to respondents.

A.10. Assurance of Confidentiality

Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.

The opening screen on CRESTWeb states the following:

The Federal Government has a continuing commitment to monitor its awards to identify and address any inequities based on gender, race, ethnicity, or disability of the principal investigators, co-principal investigators, trainees, or other participants.

Information from this data collection system will be retained by the National Science Foundation (NSF), a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may e used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs.  Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.

A.11. Questions of a Sensitive Nature

CREST requests data that are considered of a private nature, including the name, phone number, e-mail address, disability status, and citizenship of the PIs, faculty members, and students associated with the activities of a center. In addition, financial support information is collected for faculty members and students. These data are collected in order to monitor the sites' participant populations, to track the participants over time, and to assess the success of the program. Respondents have the option of not providing information that they consider privileged and may mark their gender, race, ethnicity, citizenship, and/or disability as "not reported." Individuals' data are provided only to CREST program staff and consultants conducting studies using the data as authorized by NSF. Any public reporting of the data is in aggregate form.

A.12 Estimates of Response Burden

A.12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden

The estimated average annual response burden is 1,170 hours. This figure was calculated using the total annual burden reported from the last three collection cycles. Respondents will be either PIs or program coordinators. One PI or program coordinator per award completes the questionnaire.

 

The estimated annual burden is calculated below.

 

A.12.2. Hour Burden Estimates by Each Form and Aggregate Hour Burdens

The CREST system consists of only one form. This form accounts for the entire annual response burden of 1,170 hours.

 

A.12.3. Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $46,798. The following table shows the annualized estimate of cost to PI/program coordinator respondents, who are generally university professors. This estimated hourly rate is based on a report in the April 16, 2010, edition of The Chronicle of Higher Education (2010). (“What Professors Earn.” The Chronicle of Higher Education, 56(31), A10, Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $83,511. When divided by the number of standard annual work hours (2,080), this calculates to approximately $40 per hour.

 

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

There is no overall annual cost burden to respondents or record keepers that results from CREST other than the time spent responding to the online questionnaire; the system screen shots are attached as appendix A.

 

It is usual and customary for individuals involved in implementing a CREST award to keep descriptive records. The information being requested is from records that are maintained as part of normal practice at higher education institutions. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs, other center personnel, and students who are the respondents to the CREST data collection task make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.

A.14. Estimates of Costs to the Federal Government

Computing the annualized cost to NSF for the CREST and HBCU-RISE data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the CREST data collection:

 

The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the CREST and HBCU-RISE data collection is $223,302.

A.15. Changes in Burden

The previously reported total annual estimated response burden for this collection was 1,971 hours for 27 respondents; the current request for 1,170 hours for 34 respondents is a decrease of 801 hours. Though there is an increase in the number of respondents because of an increase in the number of awards, we now have statistics to estimate burden hours for NSF’s HBCU-RISE initiative. This initiative focuses on Historically Black Colleges and Universities and provides awards with the same mission as CREST awards through the same program solicitation. The data collected and forms used by CREST and HBCU-RISE initiative respondents are identical, however the HBCU-RISE awards generally have less data to report than the CREST centers.

There have been no major changes in the instrument that would affect the burden. Minor adjustments to the data elements collected include requesting demographic data about PIs, data about conferences organized, additional questions about collaboration projects, and an NSF-funding source field in the Proposals/Awards section.

A.16. Plans for Publication, Analysis, and Schedule

This is a recurring study. Data collection is conducted annually, beginning in early to mid-February and ending in late April. The data are collected for internal review purposes and to monitor the CREST awardees, as well as for use in reporting to Congress and fulfilling NSF’s GPRA and other reporting requirements. Reports to NSF management and Congress dealing with the characteristics and performance of the CREST program include tables and charts generated from the database. In addition, CRESTWeb users are able to access tables that display summary information for data entered in the current and previous collections.

Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. ICF Macro, the contractor conducting this data collection on behalf of NSF, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF. After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF, and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information delivered by the collection that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.

Before the conclusion of the collection, both NSF and the funded CREST centers may use preliminary data to improve management and performance. For example, data generated by this study may appear as inputs to other internal and external NSF reports. At this time, NSF has no set timeline for publishing interim reports from this study.

A.17. Approval to Not Display Expiration Date

Not applicable

A.18 Exceptions to Item 19 of OMB Form 83-I

No exceptions apply.

Section B

Introduction

B.1. Respondent Universe and Sampling Methods

The sample size is the entire universe of projects. There will be an estimated average of 34 awards active each year, and data will be collected from each center. These projects are heterogeneous in terms of their research focus and their academic disciplines in the sciences, technology, engineering, and mathematics.

Population	Estimated Universe Size	Sample Size
Awardees	34	34

 

B.2. Information Collection Procedures/Limitations of the Study

This data collection uses a Web-based instrument. Each respondent will provide answers each year during the duration of their NSF funding and 1 year after NSF-funding has expired.

NSF understands the limitations of this data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the CREST system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.

B.2.1. Statistical Methodology for Stratification and Sample Selection

This collection is a census, so no sampling is required.

B.2.2. Estimation Procedure

Not applicable

B.2.3. Degree of Accuracy Needed for the Purpose Described in the Justification

Not applicable

B.2.4. Unusual Problems Requiring Specialized Sampling Procedures

Not applicable

B.2.5. Use of Periodic (Less Frequent Than Annual) Data Collection Cycles

Not applicable

B.3. Methods for Maximizing the Response Rate and Addressing Issues of Nonresponse

Past collections have had 100 percent response rates and NSF expects that rate to remain the same. Considerable effort is made for follow up for non-response and incomplete responses. Biweekly during the collection, the contractor sends e-mail messages and calls award sites that have not logged into the system, and notifies all award sites that are still entering data when the system closing date is one week away. Appendix C includes examples of the e-mail messages announcing the opening of the system and reminding awardees to log in and enter data. The collection is part of reporting required of awardees; principal investigators are responsible for ensuring that data are collected from other center personnel and students, and have access to status information on the Web site indicating which participants have not responded.

B.4. Tests of Procedures or Methods

This system has been in operation since 2002 and has been tested extensively. The system is routinely updated, based on user feedback. For example, after receiving feedback from system users, a check was added to verify that duplicate persons and accomplishments are not added to the CRESTWeb system. Also, the Proposal and Award sections were combined into a single section; this streamlined approach facilitates data collection and reduces user burden.

B.5. Names and Telephone Numbers of Individuals Consulted

Agency

Victor Santiago, National Science Foundation, (703) 292-4673

William Neufeld, National Science Foundation, (703) 292-5148

Contractors

ICF Macro will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.