REQUEST FOR CLEARANCE
NATIONAL SCIENCE FOUNDATION
DIRECTORATE FOR EDUCATION AND HUMAN RESOURCES
DIVISION OF GRADUATE EDUCATION
GRADUATE TEACHING FELLOWS IN K–12 EDUCATION
DISTANCE MONITORING SYSTEM
Attachment D
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance for the distance monitoring data collection for the National Science Foundation (NSF) Division of Graduate Education (DGE) Graduate Teaching Fellows in K–12 (GK-12) program under the Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on January 31, 2008. The EHR Generic Clearance includes collections of information about NSF's education and training (E&T) activities.
The GK-12 program (http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5472&from=fund) was initiated in 1999. The primary objective of the program is to provide fellowships to highly qualified graduate students in science, technology, engineering, and mathematics (STEM) disciplines to serve directly as resources in the nation's K-12 schools. GK-12 Fellows, selected by awardee institutions, work directly with cooperating teachers to demonstrate STEM concepts, connect elementary and secondary learning to the habits and skills required for future study, provide role models for future STEM professionals, enhance teachers' content knowledge and understanding of principles of science and mathematics, and assist in science and mathematics instruction.
By supporting the work of the Fellows, the program intends to enhance the level of interaction in STEM education among K–12 schools and higher education in ways positive for all; to increase the appreciation and capability of graduate students for high quality STEM teaching; and to enhance both appreciation and capacity for content-rich, inquiry-based approaches in K–12 schools.
The GK-12 program supports the NSF strategic outcome goal of cultivating "a world-class, broadly inclusive science and engineering workforce," and expanding "the scientific literacy of all citizens," labeled as 'Learning' on page 5 of the FY 2006-2011 Strategic Plan, http://www.nsf.gov/pubs/2006/nsf0648/NSF-06-48.pdf. In particular, the program will help promote NSF's Learning-related investment priorities to “build strong foundations and foster innovation to improve K-12 teaching, learning and evaluation in science and mathematic,” “develop methods to effectively bridge critical junctures in STEM education pathways,” and “prepare a diverse, globally engaged STEM workforce” (reference page 7 of the plan).
The GK-12 monitoring system consists of a Web-based data collection instrument that will be completed by the principal investigator (PI) of the award, with additional data submitted by GK-12 graduate fellows and by the cooperating teachers that work with them.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements, for measuring attainment of NSF's program, project and strategic goals as laid out in NSF’s Strategic Plan, and a baseline for future program evaluations.
The primary purpose of this collection is program management, also known as program monitoring. This data collection activity is designed to track the extent to which GK-12 projects meet the objectives of the program. Within the DGE division, this information is used to administer and monitor the progress of the program. The findings are used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts.
The GK-12 program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management, to provide advice for continuous improvement of NSF performance. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the GK-12 monitoring system can be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project and strategic goals by the President's Management agenda as represented by the Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART), the Government Performance and Results Act (GPRA) of 1993, and the NSF’s Strategic Plan. These data help NSF management examine their progress towards these goals and respond to these reporting requirements. In addition, Some of the GK-12 principal investigators (PIs) also use the data collected to report information at the school district and state level.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts, such as the sort of rigorous evaluations described in the May 2007 Report of the Academic Competitiveness Council, which was established by the Deficit Reduction Act of 2005 (P.L. 109-171) to serve as a multi-agency effort to identify federal STEM education programs and establish their effectiveness. The full ACC report can be accessed at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/index.html. DGE plans for monitoring system data to be used as a baseline for an evaluative research study to be conducted by Abt Associates.
Data are collected using the GK-12 Data Collection System, an online system developed to facilitate electronic data collection. The six categories of data that are collected are summarized below:
1. Award Data. Since GK-12 awards are made directly to academic institutions, specific data about each awardee are collected, including:
· Grantee institution
· Award title
· Award start and end dates
2. Award Effort Data. Data collected on specific award efforts include:
· Effort title
· Associated colleges and universities
· Schools
· Award discipline(s)
· Award design model
· Partner organization(s)
3. Additional Funding Source Data. The names and types of additional sources of funding and the amount of the additional funding are collected.
3. Associated Colleges and Universities Data. Contact information for institutions associated with the awardees is collected.
4. Collaborating School Data. Data collected about collaborating schools include:
· Contact information
· Descriptive data (grades served, setting/location, etc.)
· Dates of involvement in the fellowship
5. Class Data. Information on classes is collected, including:
· Class name
· Grade
· Subject
· Number of students
6. Participant Data. Contact information (including name, social security number (for Fellows only), phone number, fax number, and e-mail address), academic information, and demographic information (such as gender, race, ethnicity, citizenship, and disability status) are collected for:
· Award evaluators
· Principal investigators
· Fellow contacts
· Other participants
Respondents are GK-12 award principal investigators, who provide project and school information, and graduate fellows and cooperating teachers, who enter their own participant data, as well as some additional information about their experiences in the GK-12 program. Information on gender, race, ethnicity, disability (if any), academic discipline, and class is needed to monitor the GK-12 program's participant population. Names and social security numbers of Fellows are collected to permit tracking the program participants over time.
The GK-12 system follows the success of other Web-based data collection systems at NSF. The Web-based software facilitates respondents' data entry by ensuring more complete and correct data submissions and thus reducing the need for follow up. Unlike earlier systems, which provided software on disks, newer Web-based systems like GK-12 do not allow respondents to revise the field sizes, thus ensuring comparability of data across sites. Fields are also marked with out-of-range indicators, and respondents are warned to check their data if they appear to be out-of-range. GK-12 sites can view data submitted in previous collection cycles. Since most program participation is on a multiyear basis, this feature makes updating the previous year's data, particularly those on participants, far easier and less burdensome. The GK-12 system is compliant with Section 508 of the Rehabilitation Act.
EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. Web-based surveys employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
Data collected under the GK-12 program are not available anywhere else. The GK-12 data collection does not duplicate other NSF efforts.
Whenever possible, data are drawn from existing NSF databases. Survey questions are asked only if the information requested is not available elsewhere. NSF has examined its data collection requirements to ensure that the requested data are not available from other Federal sources.
Changes to NSF’s FastLane are monitored on a regular basis, and as new data elements are added to (or deleted from) the FastLane application and/or project reporting system, the survey is modified accordingly. The survey does not duplicate items available through FastLane.
No information is to be collected from small businesses.
The information requested here is not available elsewhere. Without this information, NSF would be restricted in describing the activities of the GK-12 program. Without this feedback, NSF would have no way of making systematic modifications to the GK-12 program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the GK-12 program. Finally, without this information NSF would find it difficult to meet GPRA and PART requirements and would be unable to comply fully with the congressional mandate that the Foundation monitor its STEM education programs.
The data collections will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register August 24, 2007, Volume 72, Number 164, page 48694. No comments were received.
The PIs who use the GK-12 data collection system were consulted in its design and planning. A pilot test was conducted with two awards in 2002. Feedback from system users is solicited annually at various GK-12 meetings (e.g., the annual orientation meeting for new awards and the annual Projects meeting). In addition, user comments submitted during the collection period are taken into consideration for system improvements.
No payments or gifts will be provided to respondents.
Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.
The opening screen on the GK-12 system states the following:
The Federal Government has a continuing commitment to monitor its awards to identify and address any inequities based on gender, race, ethnicity, or disability of the principal investigators, co-principal investigators, trainees, or other participants.
Information from this data collection system will be retained by the NSF, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested "for general use," will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grant and in response to Public Law 99-383 and 42 USC 1885c. The Social Security number (SSN) will be maintained in accordance with the requirements of the Privacy Act of 1974. Submission of the SSN is voluntary. It is used for survey quality control, program evaluation, and matching with other data sets maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998).
GK-12 collects data that are considered of a private nature, including the name, phone number, e-mail address, disability status, and citizenship of the PIs, award evaluators, cooperating teachers, Fellows, and other participants. In addition, social security numbers are collected for/from project Fellows. These data are collected in order to monitor the site's participant populations, to track the participants over time, and to assess the success of the program. Respondents have the option of not providing information that they consider privileged and may mark their gender, race, ethnicity, and/or disability as "not reported." For social security number, a value of 999-99-9999 may be entered. Individuals' data are provided only to GK-12 program staff and consultants conducting studies using the data as authorized by NSF. Any public reporting of the data is in aggregate form.
The estimated total number of annual respondents is 2280, with an average annual response burden of 6,120 hours. The Web-based collection is an annual activity of each award site of the GK-12 program. We estimate an annual average of 120 awards, with an average of 8 fellows and 10 teachers per award, resulting in an average of 120 PIs, 960 fellows, and 1200 teachers responding each year. The number of respondents was estimated using data from the current portfolio of GK-12 awards, and the average annual hour burden for all respondents was determined using the burden reported by respondents during the last collection cycle.
The estimated annual burden is calculated below.
|
Respondent Type |
Estimated Average Annual Number of Respondents |
Estimated Average Annual Burden Hours Per Respondent |
Estimated Annual Person Hour Total |
|
PIs/Program Coordinators |
120 |
23 |
2,760 |
|
Graduate Fellows |
960 |
1 |
960 |
|
Cooperating Teachers |
1200 |
2 |
2,400 |
Total |
2280 |
|
6,120 |
As mentioned above respondents will be project PIs, graduate fellows, and cooperating teachers. The estimated total annual response burden is 6,120 person-hours. There is a different Web-based form for each respondent. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Person Hours |
|
PI survey |
PI/Program Coordinator |
120 |
23 |
2,760 |
|
Fellows Survey |
Graduate Fellows |
960 |
1 |
960 |
|
Teachers Survey |
Cooperating Teachers |
1200 |
2 |
2,400 |
|
Total |
|
2280 |
|
6,120 |
The overall annualized cost to the respondents is estimated to be $168,360. The following table shows the annualized estimates of costs to respondents. The estimated hourly rate for PIs is based on a report in the April 20, 2007, edition of The Chronicle of Higher Education (2007. “What Professors Earn.” The Chronicle of Higher Education, 53(33), Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $76,639. The hourly rate for fellows is based on their annual stipend of $30,000. The rate for cooperating teachers was established by using the Bureau of Labor Statistic's May 2006 National Occupational Employment and Wage Estimates, which estimates the mean annual wage for those in Education, Training, and Library Occupations to be $45,320. Each of these average annual wages was then divided by the number of standard annual work hours (2,080) to determine an average hourly rate for each respondent type. Those rates and the total costs are indicated in the table below:
|
Respondent Type |
Number, Rate, and Burden |
Costs |
|
PIs/Program Coordinators |
(120 x $37/hour x 23 hours) |
$102,120 |
|
Graduate Fellows |
(960 x $14/hour x 1 hours) |
$13,440 |
|
Cooperating Teachers |
(1200 x $22/hour x 2 hours) |
$52,800 |
|
Total |
|
$168,360 |
There is no overall annual cost burden to respondents or record keepers that results from GK-12 other than the time spent responding to the online survey.
It is usual and customary for individuals involved in implementing a GK-12 award to keep descriptive records. The information being requested is from records that are maintained as part of normal practices of GK-12 projects, including graduate fellow training and collaboration with K-12 teachers. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs, Fellows, and cooperating teachers and/or other administrators who are the respondents to the GK-12 data collection task make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Netscape or Microsoft Explorer) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
Computing the annualized cost to NSF for the GK-12 data collection was done by taking the budgets for 3 years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the GK-12 data collection:
|
Operational Activities |
Cost Over 3 Years |
|
System Development (includes initial development of the database and Web-based application, and later changes requested by the program-e.g., increased reporting tools, additional validations) |
$614,300 |
|
System Maintenance, Updates, and Tech Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$270,000 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by HRD), and Followup activities (e.g., providing data to other consultants) |
$460,500 |
|
3-Year Total for All Operational Activities |
$1,344,800 |
The annualized cost was computed as one-third of the total 3-year costs; thus, the annualized cost to NSF for the GK-12 collection is $448,267.
The burden for this collection is expected to decrease from 3,510 respondents and 9,360 annual person hours to an annual average burden of 2,380 respondents and 6,320 hours. The previous burden was developed in early 2007, shortly before the first year of the annual GK-12 data collection that requested data from fellows and teachers as well as PIs. This initial burden was estimated using data on the GK-12 award portfolio and on the respondent burden in other, similar data collections. The new, lower burden was determined using data from the first year of this expanded data collection. It is based on the burden reported by respondents and reflects that there are fewer respondents that were estimated, and that it takes the fellows an average of one hour, not the projected two hours, to complete the form. This lower burden is based on respondent experience and is therefore more accurate.
There have been no changes in the instrument since it was previously cleared; the change in burden is due solely to more accurate data on the number and hourly burden of respondents.
This is a recurring study. Data collection is conducted annually, beginning in April and ending in June. The data are collected for internal review purposes, for monitoring the GK-12 Fellows, as well as for use in reporting to Congress. Reports to NSF management and Congress dealing with the characteristics and performance of the GK-12 program include tables and charts generated from the database. In addition, respondents are able to access tables that display summary information for data entered in the current and previous collections.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. Macro International Inc., the contractor conducting this third-party study on behalf of NSF, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF. After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF, i.e., NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information delivered by the study that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System ODS) or simply a page on the NSF Web site) in which to publish.
Before the conclusion of the study, both NSF and the funded GK-12 projects may use preliminary data to improve management and performance. For example, data generated by this study may appear as inputs to other internal and external NSF reports (e.g., the GPRA Annual Performance Plan). At this time, NSF has no set timeline for publishing interim reports from this study.
Not Applicable
No exceptions apply.
The sample size is the entire universe of respondents. There are currently an average of 2,280 PIs, Graduate Fellows, and Cooperating Teachers working on GK-12 awards and this number is expected to remain stable throughout the clearance period.
|
Population |
Estimated Universe Size |
Sample Size |
|
GK-12 PIs, Graduate Fellows, and Cooperating Teachers |
2,280 |
2,280 |
This data collection uses a Web-based survey. Each GK-12 project will provide project information each year during the duration of their NSF funding.
NSF understands the limitations of the this data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the GK-12 system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not Applicable
Not Applicable
Not Applicable
Not Applicable
The collection is part of reporting required of awardees; principal investigators (PIs) are responsible for ensuring that data are collected from other center personnel and students, and have access to status information on the Web site indicating which participants have not responded. Past collections have had 100 percent response rates from PIs and graduate fellows and NSF anticipates that the rate will remain the same. During the first year of the expanded data collection that included cooperating teachers, the response rate from this group was 63%. Unlike PIs and fellows, cooperating teachers are not supported by NSF and many are not available in the summer when data collection in completed. However, NSF is committed to improving this response rate and expects it to improve as teachers become more aware of the data collection in subsequent years. Also, NSF plans to begin data collection earlier in the season when school is still in session in a further effort to increase response rates. Considerable effort is made to follow up for non-response and incomplete responses with all respondent groups. This is done by sending e-mail every three weeks to award sites that have not logged into the system, by notifying all award sites still entering data when the system closing date is one week away, and by using additional email and phone follow-up as needed. Approximately 60% of award sites receive at least one of these follow-up e-mail reminders. In addition, PIs are encouraged to follow-up with fellows and teachers on their awards. Examples of the e-mail messages announcing the opening of the system and reminding awardees to log in and enter data are included in Appendix B.
This system has been operational since 2002 and was tested extensively. In addition, many of the items and response categories follow formats that are already in place at NSF. User feedback is also taken into consideration during system updates, to ensure that the system is continually improved.
Agency
Carol Stoel, National Science Foundation, (703) 292-8630
William Neufeld, National Science Foundation, (703) 292-5148
Contractors