| Number | Comment Source | Submitted Comment | Response |
| 1 | Council on Governmental Relations (COGR) | The Notice states that the purpose of the Biographical Sketch is to assess how well qualified the individual, team, or organization is to conduct the proposed activities. The purpose of Current and Pending (Other) Support (CPS) is to assess the capacity of the individual to carry out the research as proposed and to help identify any potential scientific and budgetary overlap/duplication, as well as overcommitment with the project being proposed. As implied in the Notice, the activities listed in the NSPM-33 Implementation Guidance Pre- and Post-award Disclosures Relating to the Biographical Sketch and Current and Pending (Other) Support (“Table”) should meet the stated objectives for the Biographical Sketch and CPS and have practical utility. Unfortunately, many of the activities listed in the Table can be interpreted broadly and creates confusion, leading to inconsistent responses. For example, the biographical sketch requires listing academic, professional, and institutional appointments and positions, but these terms can mean different things to different people, and no guidance or definitions are provided to clarify what these terms mean. For example, what is meant by an “institutional appointment”? Would this include service on university committees for university service (i.e., IRB, faculty advisor, parking committee)? Do professional positions include service on an editorial board? We also question the practical utility of some of the required data elements and how they contribute to the intended purpose of the forms, which is to assess the individual’s qualifications to conduct the proposed activities and to assess potential overlap or over-commitment. | Thank you for your comment. With regard to the issue of appointments, the NSPM-33 Definitions Appendix has been revised to provide specific definitions for Institutional, Professional and Academic Positions and Appointments. Additional information has been added to the Current and Pending (Other) Support section that deals with In-kind contributions, both in terms of descriptors as well as the categories of information requested. With regard to the Disclosure table, multiple research funding agencies has seen a significant reduction in questions regarding the types of activities to be reported upon issuance of the Disclosure Tables. The tables, however, will be modified to address changes made as a result of the public comment process. |
| 2 | Council on Governmental Relations (COGR) | We have similar questions about the purpose and utility of in-kind support, which is a relatively new area of disclosure, and one for which researchers have a significant number of questions. This support is often minor and of low value. It would be helpful if agencies could be more specific as to what aspects of in-kind support they believe are critical to the review or awarding of a research project. | Thank you for your comment. The instructions for the Current and Pending (Other) Support Common Form have been revised to state, "In this section, please disclose all in-kind contributions with an estimated dollar value of $5000 or more and that require a commitment of the individual’s time. An in-kind contribution is a non-cash contribution provided by an external entity that directly supports the individuals’ research and development efforts. An in-kind contribution may include but is not limited to: real property; laboratory space; equipment; data or data sets; supplies; other expendable property; goods and services; employees or student resources. In-kind contributions with an estimated value of less than $5000 need not be reported." |
| 3 | Council on Governmental Relations (COGR) | Finally, a relatively new item has been introduced related to reporting “start-up companies that are unrelated to intellectual property licensed by the applicant institution.” The relation this has to the stated goals of CPS to assess the capacity of the individual to carry out the research as proposed and to help identify any potential scientific and budgetary overlap/duplication is not clear. Rather, this would be addressed more appropriately under the recipient’s conflict of interest and conflict of commitment policies and not as part of a researcher’s grant application. | Thank you for your comment. The question/comment is unclear. We presume the comment is asking how a researcher’s involvement and commitment in a start-up company might impact the “capacity of the individual to carry out the research.” We note that start-up companies that are related to intellectual property licensed by the applicant institution are already reported to the institution. Thus, when a researcher becomes involved in a start-up company that has not already otherwise been reported, such an involvement, by definition, has the potential to impact that researcher’s capacity to perform other research. Therefore, start-up companies that are unrelated to intellectual property licensed by the applicant institution is reported to assess capacity and determine overlap/duplication. This is analogous to the reporting of paid consulting which is outside of that which the institution permits. These activities involve a time commitment and/or financial gain. The disclosure is meant to provide the federal funding agency with the necessary information to make that assessment. |
| 4 | Council on Governmental Relations (COGR) | Recommendations: We request that agencies: (1) critically assess and only require those data elements that truly serve the intended purposes of the forms, (2) provide information to the grantee community on the utility of the information needed for each type of activity and how it is necessary to meet the stated purpose and make grant decisions, and (3) provide definitions and examples for each data element. | Thank you for your comment. OSTP has been working hard to communicate with the research community on these points. We will seek to incorporate them into community briefings going forward. |
| 5 | Council on Governmental Relations (COGR) | The Notice estimates burden time as one hour for the Biographical Sketch and one hour for the CPS. COGR member institutions report that this is a significant underestimation of the actual time it takes to initially complete the form and update the information, considering the complexity of disclosure requirements. In a poll that COGR conducted during a recent webinar, the majority of respondents indicated it takes two hours or more to complete the biographical sketch for the first time, with almost half indicating it takes about an hour to update. An even larger majority indicated it takes two hours or more to complete the current and pending support for the first time, with almost half indicating it takes about an hour to update. We also collected anecdotal information from faculty, which shows that the initial completion of the CPS forms takes 4-6 hours. Due to limited resources, the cost and administrative burden may be significantly greater at emerging institutions. Another consideration is that the Notice only reflects the burden of CPS submissions at the time of proposal. Proposal or JIT (pre-award) submissions are not the only instances when CPS is collected. Should changes occur, agencies require updated CPS documents to be submitted with annual progress reports. Those updates take additional time, which is not reflected in the Notice. Recommendation: Recommendations: We recommend working with organizations like the Federal Demonstration Partnership to identify a more accurate estimation of burden and to understand pain points for researchers and opportunities to streamline. | Thank you for your comments. The proposed burden assessment will be revised based on feedback received. |
| 6 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. Define Terms. Noticeably absent throughout the documents are defined terms to ensure consistency of interpretation in the types of data provided. Definitions, qualifiers, and examples provide much-needed clarity for respondents. A great example defined in both documents is senior/key personnel which describes the qualifier of a key person (listed by the applicant/awardee organization and approved by the Federal research funding agency…). We recommend that the final document define essential (e.g., “titled,” “professional,” etc.) and vague terms (i.e., “in-kind”) and include qualifiers and examples. (See additional discussion associated with specific requirements below.) |
Thank you for your comment. With regard to the issue of appointments, the NSPM-33 Definitions Appendix has been revised to provide specific definitions for Institutional, Professional and Academic Positions and Appointments. Additional information has been added to the Current and Pending (Other) Support section that deals with In-kind contributions, both in terms of descriptors as well as the categories of information requested. It is not possible, however to develop a one size fits all definition of In-kind contributions given the variance in agency missions. |
| 7 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. In-kind contributions. It is still unclear exactly what is required for in-kind contributions. According to 42 U.S.C §§ 6605 and the CPS instructions (pg. 1), CPS includes in-kind contributions requiring a commitment of time and directly supporting the individual's research and development efforts, whereas the CPS instructions later state (pg. 4), In this section, please disclosure [sic] ALL in-kind contributions related to current and pending support. In-kind contributions include, but are not limited to, office/laboratory space, equipment, supplies, and employee or student resources. COGR requests confirmation that the requirement to report in-kind contributions is limited to those contributions that (a) have an associated commitment of time on the part of the senior/key person, and (b) directly support the senior/key person’s research and development efforts. We also request that this be made clear in the form itself. COGR encourages federal agencies to reassess the utility of disclosing in-kind contributions that do not have an associated commitment of time in the review for overlap and over-commitment. If, after this reassessment, the agencies continue to require reporting of all in-kind contributions, we note that in-kind contributions usually do not come with a specified “associated time commitment.” For example, researchers commonly receive cell lines or other reagents from colleagues at other institutions or from industry. These reagents may be in-kind contributions, but they do not require a specified time commitment. Considering they do not require a specified time commitment, it is arguable that these contributions would not be disclosable. Similarly, when a PI or senior/key person hosts postdoctoral scholars, students, or visiting scholars, the arrangement usually does not come with a specified time commitment. It would be helpful for the CPS to clarify that reporting is not required in such cases. COGR notes that NIH appears to require disclosure of in-kind contributions regardless of whether they require a time commitment. Lack of consistency creates confusion for researchers. Because most in-kind contributions do not involve time commitments, the format for reporting, with start/end dates, person months, and dollar values, usually is not applicable. For this reason, the format for in-kind should allow for the optional inclusion of start/end date, person months, and dollar values to address those situations in which these elements are not applicable. (See further comments on reporting format below.It is still unclear exactly what is required for in-kind contributions. ...the absence of any minimum financial thresholds to trigger the disclosure of in-kind contributions creates unnecessary burden, as does the lack of a limit on how far back in time the contribution was received (a welcome clarification previously provided by NIH). Finally, the absence of any minimum financial thresholds to trigger the disclosure of in-kind contributions creates unnecessary burden, as does the lack of a limit on how far back in time the contribution was received (a welcome clarification previously provided by NIH). COGR strongly encourages an assessment of the utility of this information in the review process. |
Thank you for your comments. The instructions for the Current and Pending (Other) Support Common Form have been revised to state, "In this section, please disclose all in-kind contributions with an estimated dollar value of $5000 or more and that require a commitment of the individual’s time. An in-kind contribution is a non-cash contribution provided by an external entity that directly supports the individuals’ research and development efforts. An in-kind contribution may include but is not limited to: real property; laboratory space; equipment; data or data sets; supplies; other expendable property; goods and services; employees or student resources. In-kind contributions with an estimated value of less than $5000 need not be reported." It is not possible, however to develop a one size fits all definition of In-kind contributions given the variance in agency missions. |
| 8 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. Appointments and Positions. The Proposed Instructions for Submission of the Biographical Sketch include instructions to list all the individual’s academic, professional, or institutional appointments and positions, beginning with the current appointment (including the associated organization and location). However, the subsequent paragraph states that for professional appointments, senior/key personnel must only identify all current domestic and foreign professional appointments. We would welcome confirmation that this means all past and present academic and institutional appointments must be listed, but only current professional appointments must be listed. Clarification on the differences between academic, institutional, and professional appointments (and examples thereof) would also be welcome, as would differentiation and explicit instruction on academic, institutional, and professional positions (again with examples). Finally, the Proposed Instructions state, “Appointments and positions include any titled academic, professional, or institutional position…” Clarification is requested regarding what is meant by “titled” and whether the instructions are intended to exclude the reporting of “untitled” positions. Defining “titled” with examples would be helpful. ...the requirement to list all of their academic, institutional, and professional appointments and positions, when combined with a limit on the number of pages allowed for the Biographical Sketch, does not provide them with sufficient space to adequately demonstrate their qualifications to carry out the proposed project; ...CPS specifies: Enter a summary of the in-kind contribution.... whether or not it has an associated time commitment, which is contrary to the Table, which specifies an associated time commitment. |
Thank you for your comment. With regard to the issue of appointments, the NSPM-33 Implementation Guidance Definitions Appendix has been revised to provide specific definitions for Institutional, Professional and Academic Positions and Appointments. |
| 9 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. Inconsistencies and Gaps. We note that there are inconsistencies or gaps between the common forms, the summary of data elements, and the Table. For example, the Proposed Instructions for Submission of the Biographical Sketch lists the Certification as an item that is not required. However, the Summary of Data Elements lists it as a required item. We assume the latter is true. Additionally, the Biographical Sketch instructions do not address page limits, and the CPS instructions specify that there is no page limit. Our members consistently hear from their investigators that the requirement to list all of their academic, institutional, and professional appointments and positions, when combined with a limit on the number of pages allowed for the Biographical Sketch, does not provide them with sufficient space to adequately demonstrate their qualifications to carry out the proposed project. Therefore, we ask that agencies not put a page limit on the Biographical Sketch or clarify that appointments and positions do not count against the page limit. We also note that the in-kind contributions section for Summary of In-Kind Contributions in the CPS specifies: Enter a summary of the in-kind contribution.... whether or not it has an associated time commitment, which is contrary to the Table, which specifies an associated time commitment. |
Thank you for your comments. Both the Biographical Sketch and Current and Pending (Other) Support Common Forms have been modified to address the inconsistencies noted. |
| 10 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. Format for Non-Project-Based Activities. There are several types of activities that are not project-based, but the proposed CPS reporting format attempts to fit them into a project-based format. These activities include in-kind, consulting, postdoctoral scholars, students, visiting scholars, sponsored travel, and startup companies based on non-organization-licensed IP. These activities often do not have a dollar value, start/end date, associated time commitment, etc. The forms should provide a format that fits the requested activity and/or provide flexibility to indicate “not applicable.” |
Thank you for your comments. Additional information has been added to the Current and Pending (Other) Support section that deals with In-kind contributions, both in terms of descriptors as well as the categories of information requested. |
| 11 | Council on Governmental Relations (COGR) | While we are pleased that the forms align with agency requirements for NIH and NSF, we had hoped that the common forms and corresponding instructions would provide clarity to longstanding areas in the disclosure requirements that remain ambiguous and in need of further clarification, definitions, qualifiers, and examples for simple interpretation as provided below. Certification. Both forms list the following certification statement: When the individual signs the certification on behalf of themselves, they are certifying that the information is current, accurate, and complete. This includes, but is not limited to, information related to domestic and foreign appointments and positions. Misrepresentations and/or omissions may be subject to prosecution and liability pursuant to, but not limited to, 18 U.S.C. §§ 287, 1001, 1031 and 31 U.S.C. §§ 3729-3733 and 3802. There are questions about the intent for the wording of the first part of the statement when the individual signs the certification on behalf of themselves. As worded, this seems to imply that someone other than the senior/key person could sign, which we do not believe is the intent. The form would be more straightforward if it stated, “I certify that the information is current, accurate, and complete…”. Similarly, the instructions could clearly state, “When the senior/key person signs, they are certifying…” The language also does not address unintentional omissions. NIH’s current language for other support (“false, fictitious, or fraudulent statements”) more clearly addresses this and focuses on intentional omissions. Also, to reduce administrative burden, we request that the agencies assess the number and types of certifications they impose on applicants. NIH, for example, will presumably require four levels of certifications (CPS form, Biosketch form, PI/Multi-PI certification per NIH GPS 2.3.7.6, and the institutional certification on the PHS 398 cover page). Consistency would also be welcome, considering the common forms require a certification that the information provided is current, accurate and complete (but not that it is true), whereas the institutional certification on the SF424 R&R cover page and the PI certification specified in NIH GPS 2.3.7.6 require certification that the information in the proposal is true, complete and accurate (but not that it is current). |
Thank you for your comments regarding the certification language. We concur with your comments and have modified the language accordingly. It is incumbent upon each Federal Research Funding Agency to provide any necessary guidance regarding the reporting of unintentional omissions in their implementation guidance. |
| 12 | Council on Governmental Relations (COGR) | Recommendations: As noted in this section, there is a need to be clear about reporting requirements and to have consistency across the various forms to reduce confusion. Confusion creates conditions for unintentional non-compliance in disclosure reporting, creates unnecessary administrative burden, and leads to multiple interpretations (including agency-specific or even program manager-specific interpretations). We request clarification in the areas mentioned above (in-kind, consulting, and appointments and positions) and recommend adding defined terms, qualifiers, and examples to minimize confusion and give researchers clear and explicit instructions on what items require disclosure. We request an assessment of the utility of reporting such a broad range of in-kind support and a reporting format more appropriate to report these kinds of support in a non-project format. We urge NSTC to consider testing the use of any new forms on a pilot basis to receive feedback and questions from the community before finalizing the forms. The FDP might be an ideal partner. This will go a long way toward providing agencies with the desired outcome while reducing misunderstandings and unnecessary burdens. |
Thank you for your comments. Additional information has been added to the Current and Pending (Other) Support section that deals with In-kind contributions, both in terms of descriptors as well as the categories of information requested. It is not possible, however to develop a one size fits all definition of In-kind contributions given the variance in agency missions. With regard to the issue of appointments, the NSPM-33 Definitions Appendix will be revised to provide specific definitions for Institutional, Professional and Academic Positions and Appointments. |
| 13 | Council on Governmental Relations (COGR) | As specified in the notice, variations among research agencies may be permitted as coordinated through NSTC or as cleared by OMB/OIRA. It is unclear at this time how agency-specific variations and deviations in the common disclosure forms will be handled (e.g., synergistic activities and personal statements). Agencies may request to supplement the forms or include agency-specific categories, which is counterproductive to creating true uniformity. We recognize the need for agency variations and appreciate that OSTP created a review and approval process to limit such variations to those absolutely necessary for an agency function. However, we are concerned that permitting agency-specific variations in the common disclosure forms will reduce our researchers’ ability to maintain one form that can be utilized across the various agencies and create true harmonization. It also increases the likelihood that researchers will need to maintain multiple disclosure forms and update each separately according to the agency-specific form, increasing the burden and opportunities for errors. A comprehensive form with clear and consistent requirements across all federal agencies is ideal for recipients to develop systems and processes that will assure complete and accurate disclosures in a streamlined, efficient manner. | Thank you for your comment. As the NSPM-33 Implementation Guidance states, any proposed changes or modifications to forms used by Federal research funding agencies will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across Federal research funding agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 14 | Council on Governmental Relations (COGR) | We also note that currently, the disclosure process varies across agencies, including timelines for when disclosures occur for initial submissions and updates. For example, at this time, only NIH specifies reporting CPS for just those projects likely to be awarded (e.g., at Just-In-Time (JIT)). This is a significant benefit to the recipient community since only a fraction of proposals are awarded, and therefore time is not spent on CPS for unfunded projects. Also, the requirements for updated information vary across the agencies as well. NSF and NIH require updates submitted in the annual progress report. However, the Department of Energy requires updated disclosures within 30 days of the change or on a timeline instructed by the program officer, which is a significant burden. Variances in reporting timelines increase the administrative burden and reduce the clarity of expectations for researchers to know when to report and update disclosures. | Thank you for your comment. As the NSPM-33 Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, Federal Research Funding agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. The NSPM-33 Implementation Guidance specifically states that "Research agencies may choose whether to require a "just-in-time" submission-meaning after the completion of the peer review but prior to funding-of R&D award application information." As such, it is a Federal Research Funding Agency decision regarding whether use of just-in-time is appropriate for that respective agency. |
| 15 | Council on Governmental Relations (COGR) | The Notice specifies that variations among research agencies will be limited and coordinated through the NSTC. Additionally, modification and/or supplementation of these common forms will require clearance by OMB/ OIRA under the PRA process. We appreciate that form variations will be managed through a review process and hope that NSTC will support only those changes that are necessary to meet programmatic requirements and have practical utility. To set clear expectations for the research community, changes to the forms should be as infrequent as possible and occur at a singular point with ample advance notice for the community to adopt and implement. | Thank you for your comment. As the NSPM-33 Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 16 | Council on Governmental Relations (COGR) | It appears that NIH and NSF will utilize SciENcv and leverage Digital Persistent Identifiers like ORCID (Open Researcher and Contributor ID). However, only NSF and the Department of Education are currently ready for users to move to SciENcv. Currently, no federal agency feeds proposal or award information to SciENcv or ORCID, leaving institutions responsible for populating these systems with Biographical Sketch and CPS information. As such, consideration should be given to allow institutions and researchers to comment on the practicality of the reporting system, provide a period to test, and ample opportunity to transition and implement. In regard to efficiency and ways to minimize the burden of the collection of information, it still appears that a driving indicator of the common forms is to elicit information indicative of whether the investigator is involved in a (malign) foreign talent recruitment program through the collection of a lengthy disclosure process, rather than asking the question outright. While it would be a very different approach, we think it could be valuable to consider asking more direct questions vs. expanding disclosure requirements. |
Thank you for your comments. The overarching goal of creation of Common Forms is to ensure the highest level of standardization possible across Federal research funding agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. The new requirements specified in CHIPS and Science will require senior/key persons to certify that they are not a party in a malign foreign talent recruitment program, as well as to disclose any foreign talent program participation. As such, this more direct approach will be taken in the future. |
| 17 | Council on Governmental Relations (COGR) | Recommendations: We recommend that agency-specific information be collected separately from the standardized disclosure forms and be explicitly limited to additional (not revised or altered standards) data elements. This will increase the likelihood that the common forms and associated data elements/definitions remain consistent across federal agencies and will promote clarity and accuracy of what must be disclosed. To help ensure coordination and consistency, we recommend that for the initial implementation, all agencies adopt the common forms during the same period and for each agency to make it clear how they plan to implement them. We recommend that the final forms, resources, training, and FAQs be posted and maintained by a single entity on a single site for a “one-stop-shop approach” (perhaps hosted by the NSF similar to the federal-wide Research Terms and Conditions). We also recommend that changes to the forms be reduced to a workable and predictable time frame, e.g., once a year, and apply to all agencies simultaneously. We request sufficient advance notice to implement changes in the requirements, especially those that require changes in IT systems and business processes. We also recommend continued engagement with stakeholders for input throughout the review process. |
Thank you for your comment. This approach would burdensome for reviewers to separately consider the qualifications and capacity of senior/key personnel in multiple locations in the proposal. |
| 18 | Department of the Army Criminal Investigation Division | Senior/key personnel with professional appointments should be required to identify past domestic and foreign appointments as well as current. Recommendation: A set time frame for disclosure could be considered, or all past appointments could be required to be reported. | Thank you for your comment. The Biographical Sketch Common Form and associated instructions have been modified to specify parameters regarding professional appointments, With regard to professional appointments, senior/key persons must only identify all current domestic and foreign professional appointments outside of their primary organization. The purpose of creation of a Common Form must balance the needs of the Federal Research Funding Agencies with the administrative burden associated with responding to these requirements. |
| 19 | Department of the Army Criminal Investigation Division | The pending support section should better define “under consideration”, which would help clarify as well as encourage full disclosures that otherwise might not have received. The sources of support section should clarify on what is meant by “internal funds” and note if such funds are derived from foreign support/donations. The Statement of Potential Overlap should be updated to clarify if refers to any foreign or domestic pending proposal or award. | Thank you for your comment. The instructions in the Current and Pending (Other) Support Common Form have been modified to provide additional clarity. The instructions also clearly outline the fact that the reporting applies to both foreign and domestic. See language for immediate reference: Identify the entity (entities) that is providing the in-kind contribution. Include, for example, Federal, State, Tribal, territorial, local, foreign, public, or private foundations, non-profit organizations, industrial or other commercial organizations, or internal funds allocated toward specific projects. |
| 20 | Department of the Army Criminal Investigation Division | Recently completed support or support that has ended from foreign and domestic sources should be required for disclosure. This disclosure would fall in line with the disclosed related research publications that are required, provides additional info on sources of funding, and indicates if research was already attempted and funded by another US agency. The activity section labeled “Consulting that is considered part of an individual's appointment/agreement with their home organization and consistent with the proposing organization's ‘Outside Activities’ policies and procedures” should be updated to disclose all foreign consulting. |
Thank you for your comment. As stated on the Current and Pending (Other) Support Common Form, "Current and pending (other) support information is used to assess the capacity or any conflicts of commitment that may impact the ability of the individual to carry out the research effort as proposed. The information also helps assess any potential scientific and budgetary overlap/duplication with the project being proposed." With regard to the issue of consulting, the instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 21 | University of Tennessee | When possible, simplify instructions and provide explanations/definitions at the point of entry. For instance, where “In-Kind” will be entered, include hovering definition of “In-Kind”. | Thank you for your comment. As new systems are developed to support electronic development of the Common Forms, we believe that language clarity, user interface, and help tools will be important components of systems development. |
| 22 | University of Tennessee | Make forms fillable | Thank you for your comment. As new systems are developed to support electronic development of the Common Forms, we believe that language clarity, user interface, and help tools will be important components of systems development. |
| 23 | University of Tennessee | Include tables so that fields do not need to be manually repeated for each entry. | Thank you for your comment. As new systems are developed to support electronic development of the Common Forms, we believe that language clarity, user interface, and help tools will be important components of systems development. |
| 24 | University of Tennessee | Do not require duplicate entry of information. Allow automatic generation of the Biographical Sketch and Current & Pending (Other) Support from (1) the Excel file or (2) Science. | Thank you for your comment. As new systems are developed to support electronic development of the Common Forms, we believe that language clarity, user interface, and help tools will be important components of systems development. |
| 25 | University of Tennessee | Clarify to users how these forms will interact with ORCID and SciENcv. | Thank you for your comment. The long-term vision is for services like ORCID and SciENcv to integrate seamlessly with the Common Forms. At this juncture, Federal research funding agencies will make decisions about the extent of integration with these services, while we continue to move towards possibilities for digital standardization. |
| 26 | Icahn School of Medicine at Mount Sinai | Please be aware that the Other Support implementation has required an enormous amount of effort both to create and maintain this document. At Mount Sinai, our primary funder is the National Institutes of Health (NIH); and so, my response is focused on the NIH. We need clear and thorough instructions and would appreciate any extra training materials from theFederal Funding agency as well. Please review our FAQs on Biosketch and Other Support FAQs ( https://icahn.mssm.edu/files/ISMMS/Assets/Research/GCO/NIHBiosketch_OS_FAQs.pdf) We currently have 82 questions and answers on the Other Support page alone. Approximately 50 of them are not repeats of information that is part of the Other Support instructions. If the instructions were clear and thorough, we would not need so many FAQs to clarify. Clear and thorough instructions are a necessity for such a complex document. For example, there should either be definitions of basic terms in the Other Support instructions document itself or links to them in the relevant source documentation. What are Projects/Proposals? What are In Kind Contributions? How does the NIH define Gifts? The NIH requests information on person month effort for each project but how do we calculate that? Is it a snapshot in time and active at the moment we submit? Is it that the JIT grant starts on a certain date and the NIH would like the proposed effort starting on that date? We have also created training sessions to educate our research community on Other Support. It is my understanding there is no comparable federal training. It would be very helpful to have a training session like GCO 401 reviewing the basics of preparing the page. GCO 401: Basics of Preparing an NIH Other Support (OS) Page GCO 402: Collecting Information for the NIH Other Support (OS) Page GCO 403: Using Mount Sinai's Other Support Template GCO 404: Upcoming Changes to the NIH Biosketch and Other Support Page (Slide Set) For applications due on or after May 25, 2021 Video (last updated 5/5/2021 I Slide Set ( last updated: 8/24/2021) Updates: GCO's Response to NIH Implementation Plan (last updated 8/24/2021) |
Thank you for your comment. Your comment has been forward to NIH for their consideration. |
| 27 | Michael Ferguson (submitting as an individual) | Person-Month(s) Per Year Committed to the Project: This important field needs clarification. The field sub-heading refers to “committed” effort while the instructions ask for “how much time the individual anticipates is necessary … .” Those are different things. Effort is a field that award recipients struggle with on other support because the original commitment can differ from the amount that is spent on a project. Most sponsors, including Federal ones, generally allow for a fluctuation of even annualized effort depending on needs so long as there is no minimum threshold in the NoA or terms and conditions. NIH helped this issue in their FAQs when advising to provide “actual” effort on a given project when completing other support Of course, that’s something that can only be provided for current year activities. | Thank you for your comments. The instructions for the Current and Pending (Other) Support Common Form have been modified to change "committed" to "devoted". |
| 28 | Michael Ferguson (submitting as an individual) | Statement of Potential Overlap: It’s curious that the advice is to enter “N/A” if there is no potential overlap instead of categorically stating “none.” N/A suggests a field that can be omitted as opposed to a response that asserts a negative status. Suggest that “None” be used when an investigator has no potential overlap as opposed to “N/A.” | Thank you for your comment. We concur with your recommendation and the instructions have been modified to address the issue. |
| 29 | Michael Ferguson (submitting as an individual) | Statement of Potential Overlap: Need clearly worded guidance that assists investigators in understanding what the agency wants here as opposed to inviting generic statements. There are two key areas that should be addressed: a) Since the majority of pending applications are unknown in terms of possible funding status, what is the agency reasonably looking for when it comes to commitment overlap? There is nothing wrong or problematic with a given investigator having, for example, a total of 24 person months of commitments on pending proposals. It’s only when a proposal becomes an award that effort cannot exceed 12 person months. Commitment overlap, therefore, should be measured by consideration of all active projects plus any pending projects for which funding is imminent, such as in a JIT stage. Pending applications where funding likelihood is unknown are not a factor in describing potential commitment overlap with the proposal or progress report for which the other support is being submitted. We have seen a considerable uptick in investigators providing unhelpful, generic statements, such as “If all pending proposals are awarded, adjustments in effort will be made to assure no over-commitment,” because they do not understand what the agencies are asking for in this section. I’m guessing that it is not broad general statements that can be copied and pasted into all other support. b)Out-year commitments on active grants can only be represented on the other support as the originally planned commitment. Therefore it is possible that if an agency adds up a future year’s effort it will exceed 12 person months. This should not be an issue for agency review of the current state of other support. Until a future year becomes a current year and the investigator knows what adjustments will be made on actual effort across projects, the out year on other support should be viewed as representational only. Agencies that have been kicking back other support for out year effort are prematurely asking for adjustment plans from investigators and institutions that cannot be reasonably known. Again, most awards and agencies allow for fluctuation of effort based on the actual needs of a given project. |
Thank you for your comments. Ultimately, decisions regarding conflicts of commitment only can be made once the applicant has provided a thorough picture of future possibilities for conflicts. We recommend that applicants provide information on all award applications that are pending so that Federal Research Funding Agencies can make informed decisions. |
| 30 | Michael Ferguson (submitting as an individual) | The Products section has a lengthy list of products but doesn’t state how these are to be organized/presented nor whether there remains a page limit to the overall bio sketch. There is no guidance for an investigator to know how to prepare this section | Thank you for your comment. The listing of products provided should be organized by the senior/key person in a way that best demonstrates their ability to carry out the research proposed. It is incumbent on each Federal Research Funding Agency to issue guidance regarding any specific limitations to the number of products permitted. |
| 31 | Michael Ferguson (submitting as an individual) | The Products section includes “website(s) and other internet site(s).” URLs have historically been disallowed except when linking to Federal sources, such as the link to one’s publications. URLs have also been historically disallowed because of security issues, cookies, and other tracking methods that could reveal agency visits and even the identity of agency reviewers of an application. Is this no longer a concern? | Thank you for your comment. In order to assess the qualifications of the individual or team to carry out the research as proposed, it is essential that inclusion of websites and other appropriate internet cites be permitted so that reviewers can access the referenced products. |
| 32 | Michael Ferguson (submitting as an individual) | Certification: what type of signature is acceptable, e.g., electronic only and verifiable by the institution, as opposed to wet, a stamp, or a dropped image of an investigator’s signature. Recommendation: Needs further guidance about what type of signature is acceptable. | Thank you for your comment. It is incumbent on each Federal Research Funding Agency to provide detailed instructions on the certification requirements that apply specifically to that Agency. |
| 33 | Department of Homeland Security | DHS is concerned about the implementation of a National Science Foundation (NSF) “common form” because it is not a government-wide approved form. The concern is based on the numerous other federal research programs within other federal agencies that leverage or modify a government-wide form for gathering grantees profile information through Grants.gov. The current government-wide Grants.gov Research and Related (R&R) SF-424 Forms Family must be updated in Grants.gov to reflect the Office of Science and Technology’s (OSTP) subcommittee recent work on data elements and Personal Identifiers and Disclosure requirements. An Application Form within the R&R forms family will provide the appropriate format for documenting disclosure information and assessing conflicts of interests. It focuses federal research programs stewardship of federal research in one form – a government-wide form managed by Grants.gov like all other government-wide forms. In the future, federal research programs within other federal agencies will need to be able to update one new Application Form and they will not be able to update the NSF “common form.” Most federal research programs in other federal agencies use Grants.gov forms which are approved government-wide. Federal agencies such as DHS pay a percentage of funding to update Grants.gov forms. While NSF mainly uses Research.gov, it does give their applicants the ability to use Grants.gov. |
Thank you for your comment. NSF and OSTP worked directly with OMB/OIRA in development of the Paperwork Reduction Act clearance package and use of the term "Common Form" at the request of OMB/OIRA. Detailed instructions will be provided by OMB/OIRA on agency use of the Common Forms prior to implementation. We concur with your comments regarding changes to Grants.gov and will be working with HHS on inclusion of these Common Forms in the SF 424 R&R. |
| 34 | Massachusetts Institute of Technology: Research Laboratory of Electronics | I prepare Current and Pending reports for PIs and I have a question about the Total Award Amount. Is it possible on the form to add clarification on what we should put here? Is it the total anticipated award amount or total obligated funds? Should we put the total amount for the project or just the portion that will go to the PI for which we are reporting? | Thank you for your comments. In response to your question, the correct entry should be "total anticipated award amount" or the total award amount. The anticipated amount is what is awarded to the recipient or any subrecipients identified on the project. . |
| 35 | University of Kansas | With the information currently available from agencies, KU believes that it is difficult to comment on whether the collected information is “necessary for the proper performance of the functions of the Agency” and has “practical utility,” since the goal of these common forms is to use them across multiple federal agencies and current practice shows federal agencies using disclosure information in a multitude of ways. While the instructions for both forms contain a general-purpose statement, this is too high-level to make utility determinations. The disclosure requirements outlined in “NSPM-33 Implementation Guidance Pre- and Post-award Disclosures Relating to the Biographical Sketch and Current and Pending Support” dated Sept. 1, 2022 (disclosure requirements outline) provides a good start to the harmonization goal for disclosure requirements, but we would request adding resources to help investigators better understand agency expectations and use. Our proposed additions include: examples/case studies for required disclosures, definitions of terms used in the forms and the disclosure requirements outline, and agencies-specific details on the use/necessity of the required information. | Thank you for your comment. With regard to consulting, the instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 36 | University of Kansas | We would also like to comments on the accuracy of the burden estimate of one hour to complete a biosketch and one hour to complete a CPS. Based on current activity, it takes significantly longer than one hour to initially complete a biosketch and a current and pending/other support document; there is no indication that the time commitment would be reduced in the newly proposed forms. A discussion with experienced staff who prepare these types of forms revealed that each of these forms average around 2 hours to populate and review for submission. This estimate of burden is consistent with the Council on Governmental Relations poll, which showed most polled individuals believed over 2 hours would be required for initial completion and that between one and two hours would be an accurate estimate of time for updating the same form. | Thank you for your comments. The proposed burden assessment will be revised based on feedback received. |
| 37 | University of Kansas | The overall CPS format raises concern because it assumes all activities can be reported in a project-based format, but many activities cannot, including in-kind commitment and sponsored travel and startup companies based on non-organization-licensed IP, to name just a few. This project-based format requires the start/end dates, person months, and dollar values for each type of in-kind contribution, but often time there may not be a known end date, direct effort allocation or known dollar value. If these fields cannot be made optional, please better clarify the expectation of what should be included to indicate that they are not applicable to the commitment. | Thank you for your comment. We fully recognize that not all activities are suitable for reporting as Proposals/Projects. In the referenced examples, all would be reportable as in-kind contributions. The required data elements for in-kind contributions have been revised to address the concerns raised, however, regarding the lack of information available. |
| 38 | University of Kansas | CPS instructions define “Pending” as “Pending – any proposal currently under consideration for funding (including this proposal) from whatever source irrespective of whether such support is provided through the proposing organization or is provided directly to the individual.” This instruction fails to realize that this form is also required at time of progress report, for which “this proposal” will no longer be pending. Please include instructions on how to format the form for submission with progress reports or provide a process to only require updates be submitted during progress report submission. | Thank you for your comment. Each Federal Research Funding Agency's implementation of the federal-wide Research Performance Progress Report provides instructions on how to update Current and Pending (Other) Support. |
| 39 | University of Kansas | Please clarify the requirement under “Total Award Amount” that requests “If the support is in a foreign country’s currency, convert to US dollars at time of submission.” Does this indicate the expectation that the value of pending proposals submitted in foreign currency should be recalculated at time of CPS submission or is the use of USD that was calculated at time of that proposal’s submission compliant? | Thank you for your comment. The "Total Award Amount" should be provided in U.S. dollars at the time of proposal submission, or as advised by the Federal Research Funding Agency. |
| 40 | University of Kansas | CPS in-kind contributions. Disclosure requires are still unclear for in-kind contributions. According to 42 U.S.C §§ 6605 and page one of the CPS instructions, disclosures should include in-kind contributions requiring a commitment of time and directly supporting the individual's research and development efforts, but page 4 of the CPS instructions state to disclose all in-kind contributions related to current and pending support. | Thank you for your comment. The instructions on pages 1 and 4 should align to include "directly supporting the individual's research and development efforts". The instructions will be modified to ensure consistency between the two sections. |
| 41 | University of Kansas | Please clarify if the requirement is to report in-kind contributions that have an associated commitment of time on the part of the senior/key person OR if the requirement is to report all in-kind commitments that directly support the senior/key person’s research and development efforts regardless of time commitment. The disclosure requirements outline specifies to only report an in-kind commitment with an associated time commitment. If the requirement is truly to report all in-kind commitments that directly support senior/key person’s research and development efforts, this list could be extensive and contain many entries that are under the $5,000 threshold considered “significant” for financial conflict of interest (42 CFR 50.603). Additionally, as noted above, it is not clear how this data would be useful for agencies. | Thank you for your comment. The Current and Pending (Other) Support instructions for in-kind contributions have been revised to state: "In this section, please disclose all in-kind contributions with an estimated dollar value of $5000 or more and that require a commitment of the individual’s time. An in-kind contribution is a non-cash contribution provided by an external entity that directly supports the individuals’ research and development efforts. An in-kind contribution may include but is not limited to: real property; laboratory space; equipment; data or data sets; supplies; other expendable property; goods and services; employees or student resources. In-kind contributions with an estimated value of less than $5000 need not be reported." |
| 42 | University of Kansas | The list of in-kind commitment examples (office/laboratory space, equipment, supplies, and employee or student resources) does not include investigator time. Is the expectation that any in-kind time commitment by key personnel should be included in the Proposal/Project section of the CPS under “Person-Month(s) (or Partial Person-Months) Per Year Committed to the Project,” or is it expected that a separate entry for this committed effort would be included in the in-kind section? The expectation for reporting this common type of in-kind commitment is not clearly indicated in the instructions. | Thank you for your comment. Contributions of senior/key personnel must be reported in the in-kind section. |
| 43 | University of Kansas | The bio sketch instructions do not address page limits, and the CPS instructions specify there is no page limit. Recommendation: Please clarify if there is a page limit, as this could impact the number of products to be included in the form. | Thank you for your comment. There is a correlation between page length and the product requirements. It is incumbent on each Federal Research Funding Agency to provide detailed instructions on any page limitations associated with the Biographical Sketch, as well as the number of products to be identified, that apply specifically to that Department/Agency. |
| 44 | University of Kansas | The second paragraph of the bio sketch instructions state " …approved by the Federal research funding agency who contribute in a substantive, meaningful way to the scientific development or execution of a research and development project proposed to be carried out with a research and development award.” KU’s understanding was that these forms would be required for all federal assistance agreements, even those for training. The wording in this specific sentence could be interpreted to indicate that this form is not required for all assistance agreements. | Thank you for your comments. The requirements apply to all Federal research and development financial assistance. Federal financial assistance that is outside the scope of research and development may be subject to other provisions, but not NSPM-33. |
| 45 | University of Kansas | Please provide definition and examples of terms for both the biosketch and CPS. While senior/key personnel is defined in both, other terms are left to interpretation, which reduces the consistency of the forms. Examples of terms that need definition and examples include: Primary Place of Performance: Examples of “virtual” projects are necessary, since- it would seem that if someone is performing work, they would have a physical location. Person-Month(s) (or Partial Person-Months) Per Year Committed to the Project: Much like NIH and NSF currently have FAQs and tutorials on this, please ensure consistent tutorial and examples of how to calculate this data point. |
Thank you for your comments. New definitions have been added to the NSPM-33 Implementation Guidance, as well as to the instructional guidance provided on both the Biographical Sketch and Current and Pending (Other) Support Common Forms. Additional help materials are planned for issuance after implementation of the Common Forms. |
| 46 | University of Kansas | The Certification statement on both forms also raises concerns and questions. “When the individual signs the certification on behalf of themselves, they are certifying that the information is current, accurate, and complete. This includes, but is not limited to, information related to domestic and foreign appointments and positions. Misrepresentations and/or omissions may be subject to prosecution and liability pursuant to, but not limited to, 18 U.S.C. §§ 287, 1001, 1031 and 31 U.S.C. §§ 3729-3733 and 3802” could be interpreted to allow delegation of signature. Please consider revising this statement to make it clear that each senior/key personnel must sign their own form. This certification does not address what type of signature (flattened electronic, wet, etc.) would be acceptable. While this might be intentional to allow for agencies to determine this, inconsistency in certification requirements can lead to lack of clarity and unintentional errors, so we would ask that the certification type be determined and made consistent. |
Thank you for your comment. As new systems are developed to support electronic development of the Common Forms, we believe that language clarity, user interface, and help tools will be important components of systems development. |
| 47 | University of Kansas | Finally, to comments on the forms’ minimization of burden related to collection of information, we would like to acknowledge that this is a fine initial step, but ongoing diligence from OSTP and their coordination of agency-specific variation, along with agency commitment to consistent applications of definitions, ways of certification, and requirements for these forms will be required to truly reduce burden and yield quality data for federal use. We recognize that more time and resources must be committed, and we look forward to partnering with our colleagues at other research institutes and the federal agencies to work toward these goals. We acknowledge that the NIH and NSF have already taken a first step in a positive direction for data collection by using SciENcv and leveraging Digital Persistent Identifiers. | Thank you for your comment. Continued diligence and coordination will be necessary as the forms are implemented. OSTP will continue to ensure this coordination through the Research Security Subcommittee. |
| 48 | University of California, Los Angeles | While in principle we support the requirement for PI’s to self-certify that the information provided in disclosure forms is complete and accurate, we question the practical utility of requiring separate signatures on both the Biosketch and Current and Pending Support (CPS) documents for all PIs and senior key personnel. We believe one certification covering all documents would serve the same purpose and reduce administrative burden. The burden associated with collecting additional signatures is compounded because (1) most federal funders have not yet transitioned to using SciENcv and (2) there is a lack of clarity, and potentially too much discretion, on the frequency of when updated disclosure documents will be required. We appreciate the NIH approach of only requiring disclosures at just-in-time and as part of the annual reporting process thereafter. In contrast, there is significant additional burden associated with more frequent reporting such as the DOE’s requirement that disclosures be updated within 30 days of any change. | Thank you for your comment. Implementation of a one signature approach would require re-instatement of PI/co-PI (and now all senior/key personnel) electronic signatures on the proposal. This process was removed decades ago, after significant negotiation with the research community. It is unclear that this approach is beneficial to applicants or the Federal research funding agencies. |
| 49 | University of California, Los Angeles | We question the practical utility of the proposed requirement to report in-kind contributions “currently under consideration” in CPS. In-kind reporting is already a challenging section to complete; reporting on hypothetical in-kind support seems a step too far, with the additional administrative burden to report outweighing any practical utility. We note that if in-kind support does transpire, it would be appropriately included as part of the annual disclosure update process. | Thank you for your comment. The requirement to report in-kind contribution information was specified in both NSPM-33 and the NSPM-33 Implementation Guidance. As such, this information will be included in the Current and Pending (Other) Support Common Form. |
| 50 | University of California, Los Angeles | UCLA is particularly concerned about the addition of a “virtual” place of performance option in CPS. If the information being collected relates to the location of the “primary organization” or “primary place of performance,” this should not be “virtual.” It would typically be the location of the organization through which the funds are accepted or where the PI is employed to do the work – not the PIs home office or other ‘virtual’ location. Allowing for a virtual place of performance could potentially trigger issues with our F&A rates, as well as potentially complicate IP ownership matters. | Thanks for your comment. The term virtual has been removed from the data element instructions. |
| 51 | University of California, Los Angeles | The Federal Notice estimates burden time as 1 hour for the Biographical Sketch and 1 hour for the CPS. We strongly disagree with these estimates. Anecdotal feedback from researchers, department personnel, and research administrators suggests that it takes 2-3 hours to create and edit each of these documents. To the extent that agency specific variations are permitted to the ‘common’ form, this burden will only increase. | Thank you for your comments. The proposed burden assessment will be revised based on feedback received. |
| 52 | University of California, Los Angeles | Permitting agency specific requirements, instructions, definitions, and data points creates confusion and frustration and having to maintain different forms for different sponsors increases the time required to maintain and update multiple forms. Additionally, requests to provide and update this information at different times during the grant process, depending on the federal agency, or even the whims of the program officer, has been difficult to manage. As noted above, UCLA supports the NIH JIT and annual RPPR reporting approach. Our hope is that these new forms will create a simpler, streamlined process for everyone involved but that will be dependent on a uniform approach. | Thank you for your comment. The NSPM-33 Implementation Guidance specifically states that "Research agencies may choose whether to require a "just-in-time" submission-meaning after the completion of the peer review but prior to funding-of R&D award application information." As such, it is a Federal Research Funding Agency decision regarding whether use of just-in-time is appropriate for that respective Agency. |
| 53 | University of California, Los Angeles | We also strongly support the recommendation from COGR that OSTP only permits agency specific changes to be requested during a set annual timeframe that is applied to all agencies simultaneously. | Thank you for your comment. As the NSPM-33 Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 54 | University of California, Los Angeles | We support COGR’s response pertaining to areas where additional clarification would be helpful. In addition, we have the following requests for clarification: Person months: If a project period is June 2023 through May 2024 (per the example provided), the instructions clarify that we should enter “2024” and enter the “corresponding person months”. Would that mean only the months that will be worked in calendar year 2024 should be reported? Or is the intent to report the months that will be worked for the period from June 2023 through May 2024? “Approved” senior/key personnel: The instructions note that disclosures will be required for senior/key personnel who are proposed by grantors “and approved by the Federal research funding agency,” however at proposal/JIT stage, we will not know who has been approved by funding agency. Additional clarification is requested. Source of Support: If funding is received from a pass-through entity, should the source of support list (a) the prime funder or (b) the passthrough entity? Consulting: We remain unclear on what consulting activities should be listed in the CPS and Biosketch. In particular, we heard at the October 2022 COGR meeting that there is a difference in reporting expectations between professional service consulting and research consulting, even if the research consulting is considered part of an individual’s appointment with their home organization. We request additional clarification on this point and consulting reporting requirements in general. |
Thank you for your comment. The instructions have been modified to delete the example related to the spanning of multiple calendar years. At the proposal stage, the applicant organization specifies the senior/key personnel for that application. Once the award is issued, the agency will identify in the notice of award, which individuals are senior/key for purposes of that project. The source of funding needs to be reported by the prime funder. With regard to consulting, the instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 55 | University of California, Los Angeles | We support COGR’s response pertaining to areas where additional clarification would be helpful. In addition, we have the following requests for clarification. Certification Dates We would also like clarification on whether there will be an “expiration date” on a signature – how recent must a signature be for these documents to be accepted? Recommendation: Reverse chronological order: In the Professional Preparation section of the Biosketch, we request that the instruction to report in “reverse chronological order” be simplified to more straightforward instruction, such as “starting with the most recent.” |
Thank you for your comment. The instructional guidance for both Common Forms has been updated to state that the signature must be the same calendar year of the proposal that is being submitted. |
| 56 | University of California, Los Angeles | We strongly urge the roll out of the common disclosure forms be delayed until such time, as federal agency proposal and award information can be feed into SciENcv or ORCID. This, coupled with the ability to collect electronic signatures in SciENcv, would greatly reduce administrative burden. As it stands, with a January 2023 manual roll out of the forms, we will have insufficient lead time to appropriately train PIs, departmental support staff and central research administrators. The lack of automated solutions places the data collection burden squarely with institutions without giving us sufficient lead time to respond to the new requirements. The burden of collecting information will be greatly reduced if the new forms are required only after appropriate automated, electronic, mechanical, or other technological collection techniques are readily available. | Thank you for your comment. The Common Form implementation process cannot be delayed until specific technical solutions are developed. However, we do acknowledge that the optimal path will be to use technical solutions and OSTP plans to support agencies in moving in such a direction. |
| 57 | University of California, Los Angeles | Timing of the rollout. We believe that the proposed forms will require substantial rework by the NIH funded researchers in particular (the proposed common forms more closely align with current NSF requirements than NIH). We understand the NIH intends to roll the manual version of these forms out in January 2023, at the same time as the new NIH Data Management Sharing requirements become effective. This will create significant administrative burden on researchers, departmental and central administrations. |
Thank you for your comments. Your feedback has been shared with NIH. |
| 58 | University of California, Los Angeles | We are concerned about how far in advance the final forms will be shared before we are required to implement them. Comments are required by October 31st, and there will need to be sufficient time to review feedback. However, we understand the NIH and NSF intend to issue the updated final forms in January 2023. This is insufficient lead time for institutions to support a successful roll out. We need time to update our processes to support the generation of accurate and timely data. As noted previously, it will limit our ability to review and train on the new requirements. This is compounded by the lack of adequate technological solutions currently available (e.g. SciENcv). |
Thank you for your comment. There will be a period of time prior to NIH and NSF adoption of the Common Forms. In the interim, please communicate any further concerns directly with the relevant Federal Research Funding Agency. |
| 59 | California State University, Pomona | The proposed collection of information is necessary for the proper performance of the functions of the Agency and will have a practical utility, in investigation instances and for agency internal consideration as well as assessment of COI. There is a burden associated with the collection of the proposed information, more so if it is not organized in a way that can serve other purposes (across the board post award management, research training/assessment requirements). Incorporating these requirements in current grant proposal submissions and online platforms that integrate with current federal agency submission platforms. In order to reduce burden of the collection of this information agencies must set clear expectations for institutional officials and require these are incorporated into currently established policies and procedures as not doing so will allow unintended flexibilities in applicability. | Thank you for your comment. We continue to work across the Federal Research Funding Agencies to ensure that these new harmonized data elements and instructions are integrated with all available reporting platforms and that they are integrated into agency policies and procedures. |
| 60 | Association of American Medical Colleges | We would further suggest refinement to these sections could create greater congruency with the section titled "Travel supported/paid by external entity for research activities with associated time commitment" if the text is adjusted in similar vein. | Thank you for your comment. This comment, however, is unclear and therefore cannot be addressed at this time. |
| 61 | Association of American Medical Colleges | Conversations with AAMC constituents flagged concerns that the estimate of time for completing both the Biographical sketch (Biosketch) and the Current and Pending (Other) Support form as set forth in the Federal Register did not reflect the actual effort or time commitment required of researchers. While NSF has estimated that it would take one hour to complete both the Biosketch and the form, input from AAMC constituents conveyed that it can take up to three hours to complete a Biosketch for the first time (with less than an hour to update it). Similarly, the Current and Pending (Other) Support form was reported to take over an hour to complete. Investigators also noted that the estimates should be expanded to account for the additional time needed when applying to or receiving funding from multiple federal agencies. Importantly, researchers and institutional administrators conveyed that estimating burden using only the time needed to complete these forms substantially underestimates the time and effort required to collect, review, and report those disclosures. The true cost and time burden includes building information systems, constantly updating the necessary infrastructure and coordinating processes among disparate offices to report to multiple agencies. This process is further complicated when a hospital is involved, when reporting effort around clinical research, or when storing confidential data. Finally, constantly changing requirements present one of the biggest challenges—each time an agency changes a component of the forms or requests unique information, it becomes increasingly difficult to automate a system to collect these data points and the burden subsequently increases. |
Thank you for your comments. The proposed burden assessment will be revised based on feedback received. |
| 62 | Association of American Medical Colleges | Institutions expressed to AAMC that current information transfer is manual, and that no systems were originally set up to have the type of information exchange needed to efficiently report on the disclosures being requested. It is thus very expensive and time-consuming to develop system integration across a variety of software platforms that currently store the requested information. AAMC has previously shared with NSF, OSTP, and the NSTC Research Security Subcommittee the potential of Convey, the AAMC-developed global disclosure system, to address some of these integration issues. The system was developed at the request of an Institute of Medicine (now National Academies) workgroup to fill an unmet need to reuse disclosure data for multiple purposes and for institutions to link existing systems through public APIs. Already in use by many journals, academic societies, and organizations, Convey can collect and store any of the requested information and can be adapted to the disclosure needs of the receiving organization, a key point when some agencies may request additional data beyond the common elements. Importantly, Convey can also connect disclosure information to digital persistent identifiers, through its integration with the ORCID iD. The AAMC would be glad to continue this discussion, as the implementation of NSPM-33 moves forward. |
Thank you for your comment. OSTP will continue to work across the Federal Research Funding Agencies to ensure that these new harmonized data elements and instructions are integrated with all available reporting platforms and that they are integrated into agency policies and procedure+E68s. |
| 63 | Association of Independent Research Institutes | AIRI takes seriously the need to ensure proper stewardship of federal funds through transparent reporting of other sources of research support. The Current and Pending (Other) Support (CPS) form requires, for each project/proposal listed, reporting of “the total award amount for the entire period of performance, including indirect costs.” Direct costs are the best reflection of the research dollars available to an individual to support their research efforts. Moreover, by using total costs as the criterion for assessing an individual’s research support, the CPS form disadvantages investigators from independent research institutions, which often have higher facilities and administrative (F&A) rates due to their unique business model and emphasis on federal funding, relative to universities and other grantees within the NIH and NSF ecosystems. Recommendation: We urge OSTP to allow individuals to report the breakdown of direct and indirect costs for each project/proposal on the CPS form. This simple modification will improve the clarity and utility of funding information as it relates to current and pending support and removes unintentional bias against research institutions with higher F&A rates. | Thank you for your comment. The amount to be disclosed is inclusive of both direct and indirect costs. |
| 64 | Association of Independent Research Institutes | With regard to administrative burden, the Federal Register Notice estimates a one-hour burden to complete the CP form. AIRI contends that this is an underestimate of the time required to complete the CPS form, given the complexity of the disclosure requirements and the need to continually monitor and update the list of projects/proposals and their corresponding details. In addition, administrative burden will likely be significantly increased at smaller research institutions like independent research institutions. Recommendation: AIRI urges OSTP to work with a range of stakeholders, including independent research institutions, to develop more accurate burden estimates. This could be implemented through engagement with the Federal Demonstration Partnership and other related organizations. | Thank you for your comments. The proposed burden assessment will be revised based on feedback received. |
| 65 | American Physiological Society | The requirements outlined on the common Biographical Sketch form closely recapitulate the NSF and NIH forms that researchers are already familiar with, making the transition to the new form easier. Recommendation: ...wish to have more clarity on which persistent identifiers (PID) are accepted. Specifically, will an ORCID ID be required, or may researchers use other forms of PID such as OpenID or ISNI? The form should clearly specify what is required for a valid PID. | Thank you for your comments. It will be incumbent on Federal Research Funding Agencies to include any relevant guidance on use of persistent identifiers in their implementation guidance on use of these Common Forms. |
| 66 | American Physiological Society | Although the Biographical Sketch and Current and Pending (Other) Support forms do not specify a process for correcting unintentional errors or inconsistencies, this is an important issue for researchers to understand. NSTC should ensure that once these forms are implemented, researchers have the resources necessary to accurately complete the forms, including clear guidance and instructions. | Thank you for your comment. The Implementation Guidance from OSTP will address the flexibility of Federal Research Funding Agencies to provide additional guidance on use of the Common Forms. Specific questions regarding each of the Common Forms will be addressed by NSF. |
| 67 | American Physiological Society | APS is supportive of efforts to reduce administrative burden for researchers, and the creation of common disclosure forms for use across different agencies is a positive step toward that goal. Once these forms are finalized, NSTC should direct agencies to use them in their final format, and not to create any additional agency-specific requirements, restrictions, or extra sections. NSTC should work with participating agencies to ensure that the format and use of these forms is consistent across all agencies so as to minimize confusion and maintain clear standards for researchers. Finally, NSTC should ensure that tools such as SciENcv that link to existing biosketch forms are compatible with the common disclosure forms. | Thank you for your comments. As the Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 68 | University of Michigan | U-M supports the common themes that the Council on Governmental Relations (COGR) has provided in its response to the request for comments on this topic. We look forward to further clarification being released. We appreciate your continued partnership. | Thank you for your comment. |
| 69 | Association of American Universities; Association of American Medical Colleges; Association of Public and Land-Grant Universities; COGR | In addition to these high-level comments, we also support the more detailed comments submitted by our colleagues at the Council on Governmental Relations (COGR) and the Association of American Medical Colleges (AAMC). Limit Agency Variation in Required Disclosure Data Elements and Instructions. The request for comments notes “agencies may develop agency- or program-specific data elements and instructions, if necessary, to meet programmatic requirements, although agencies will be instructed to minimize the degree to which they supplement the common forms.” While we appreciate the request that agencies minimize changes or additions to the data elements included in the common disclosure form, we urge that these common disclosure forms and instructions have limited variance among agencies. We ask that the federal research agencies not use this common disclosure form as a mere “floor” to which agencies are expected to add additional requirements. The allowance for such additions and inconsistencies in reporting formats across agencies will only create confusion and place increased and unnecessary compliance burdens on both researchers and institutions. Each additional, single agency request for a piece of information or a unique format in which that information must be provided increases the burdens on researchers and institutions and undermines the harmonization efforts that OSTP has worked hard to accomplish with this common form. Recommendation: Therefore, we recommend that any agency-specific departures from a final version of the common disclosure form be accompanied by a publication of a justification of the necessity of the additional information or a modification to the form in which that information should be provided. |
Thank you for your comment. As the Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 70 | Association of American Universities; Association of American Medical Colleges; Association of Public and Land-Grant Universities; COGR | In addition to these high-level comments, we also support the more detailed comments submitted by our colleagues at the Council on Governmental Relations (COGR) and the Association of American Medical Colleges (AAMC). Ensure a Transparent and Uniform Process for Updating the Common Forms Our associations request clarification on the process for future modifications to the common disclosure form whether through government-wide announcements or by individual agencies, noting that the RFI states “modification and/or supplementation of these common forms will require clearance by OMB/OIRA under the PRA process.” It is important to provide transparency on the factors considered to determine the utility of the requested information and how it meets the programmatic requirements for the agency. Currently agencies can update their forms at any time. Establishing a more consistent and uniform frequency will help the research community better manage expectations of when they can expect updates, develop systems and processes to better adhere to reporting requirements, limit potential reporting errors that could result from such changes, and reduce the administrative burden on institutions and individuals completing disclosure forms for multiple federal agencies. We also ask that OSTP establish a clear and consistent expectation on how often the respondent should update the information they provide to agencies to meet disclosure requirements. Currently, it differs across agencies, with some requesting at the time of application, just-in-time, at the time of award, annually, or within 30 days of a change (or as designated by the program officer). The inconsistency across agencies poses a challenge for researchers to effectively manage requirements and increases the administrative burden without sound reasoning for the frequency of updates. Recommendations: Our associations request clarification on the process for future modifications to the common disclosure form whether through government-wide announcements or by individual agencies. Additionally we ask that the federal government establish a clear and consistent frequency on how often OSTP and the federal research agencies will expect to issue updates to the forms. The NSTC should also periodically review variation across agency disclosure requirements to ensure conformance with the goal of common disclosure. We also ask that OSTP establish a clear and consistent expectation on how often the respondent should update the information they provide to agencies to meet disclosure requirements. Additional guidance should also be provided concerning who is ultimately responsible for making disclosure updates, specifically institutions or individual faculty. The common form, as much as possible, should also make clear the potential sanctions or consequences of non-compliance for both individual researchers and institutions. |
Thank you for your comment. It is unclear whether they are referring to updating of the Common Forms and associated instructions, or the updating of disclosure information, writ large. As the Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. The NSTC Research Security Subcommittee takes the issue of violations and penalties very seriously. |
| 71 | Association of American Universities; Association of American Medical Colleges; Association of Public and Land-Grant Universities; COGR | In addition to these high-level comments, we also support the more detailed comments submitted by our colleagues at the Council on Governmental Relations (COGR) and the Association of American Medical Colleges (AAMC). Clarify Definitions To Facilitate Compliance In creating a common disclosure form, we ask that the NSTC, working with the NSF and NIH, further clarify specific definitions relating to the types of information that must be reported, and ensure these definitions are consistently utilized across the research agencies for a clear and common interpretation. Currently there is continuing confusion on the specific meaning of some key terms where disclosure is required. These include terms such as “consulting” and “in-kind contributions.” The definitions for these and other important terms, such as “key/senior personnel,” should be clarified with qualifiers and examples and made consistent across the agencies. For example, stating that senior/key personnel are PIs/PDs across the various agencies. In addition to defining terms, we ask that the NSTC include qualifiers and provide examples to assist researchers in understanding what is required to disclose. For more specific terms and areas that need clearer definitions and explanations, we call your specific attention to the separate comments submitted by the COGR. |
Thank you for your comment. With regard to the issue of appointments, the NSPM-33 Definitions Appendix has been revised to provide specific definitions for Institutional, Professional and Academic Positions and Appointments. Additional information has been added to the Current and Pending (Other) Support section that deals with In-kind contributions, both in terms of descriptors as well as the categories of information requested. It is not possible, however to develop a one size fits all definition of In-kind contributions given the variance in agency missions. The consulting instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 72 | American Society for Biochemistry and Molecular Biology | Recommendation 1: Use existing semi-automated systems to generate the biosketch. Many principal investigators already have created biosketches for funding agencies. The OSTP should strongly recommend that agencies use a semi-automated systems to generate the new biosketch (one example is SciENcv). Doing so would significantly reduce the time principal investigators spend re-creating a biosketch in the new form. If a semi-automated system is not feasible, providing an option for principal investigators to submit their current NIH or NSF biosketch into a database and build off of the existing form is also a solution is preferred rather than having to create new submissions for each entry. Recommendation 2: Reduce requirements for the products section. As currently written, it is unclear if all products a principal investigator has ever contributed to must be included in the biosketch. If that is the requirement, we urge the OSTP to reconsider. Most researchers produce about 20 publications a year; filling out all publications, i.e., products, would take unnecessarily long to input. Federal agencies should adopt what is outlined in the NIH’s biosketch: PIs are asked to choose a select number of the most relevant products/papers. |
Recommendation 1: Your comment regarding the use of semi-automated systems is appreciated. Federal Reseach Funding Agency policies will determine the extent to which the use of semi-automated systems is permissible. The common standards will ensure that all Federal science funding agencies are using the same formats to reduce confusion and burden; in the future, we will work towards the possibility of streamlined technologies to support inputs into the common formats. Recommendation 2: Federal research funding agencies will include guidance on whether there are any limitations on the number of products that may be included in the products section of the Biographical Sketch. |
| 73 | American Society for Biochemistry and Molecular Biology | Recommendation 1: Clearly define “in-kind support” and clarify what is included in the definition. Federal agencies should define clearly “in kind support” and restrict what falls in this category to ensure that scientists are not spending hundreds of hours attempting to track down the origins and value of all the equipment and supplies in their labs. As the guidance is currently written, “in kind support” includes every piece of equipment and all supplies. It is not feasible for PIs to document and estimate the cost of each item in a single form. According to this guidance, every single plasmid, cell line, set of pipets, microfuge, etc. would need to be documented, resulting in hundreds of hours of administrative labor. This type of information has no bearing on the quality of a research project. In addition, principal investigators cannot estimate the financial value of everything in their laboratory. Recommendation 2: Clarify the “person-months” description. The section that explains “person-month(s) per year committed to the project” is unclear as written. Instead of aligning the person-months to the budget period, this should be changed to ask for the months during that budget period. Recommendation 3: Provide examples, including completed forms with references to types of support to give clear guidance to investigators. |
Thank you for your comment. The instructions for the Current and Pending (Other) Support Common Form have been modified to establish a threshold of $5000 or more for reporting of in-kind contributions. It is not possible to provide example Common Forms given the variance in the types of research supported by the Federal Research Funding Agencies. |
| 74 | APA Justice | We highly commend the White House Office of Science and Technology Policy (OSTP) for its August 2021 announcement that “[w]e have to assiduously avoid basing policies or processes on prejudice — including those that could fuel anti-Asian sentiments or xenophobia” and the January 2022 requirement that “[a]agencies must implement NSPM-33 provisions and related requirements in a nondiscriminatory manner that does not stigmatize or treat unfairly members of the research community, including members of ethnic or racial minority groups.” However, it is important for Federal agencies to recognize that emphasis on compliance is far less productive and effective than building partnership. There has been serious loss of public trust and confidence in recent years due to both perceived and factual profiling and discrimination against scientists and researchers of Asian descent, especially those of Chinese origin. While it is minimally necessary to create consistency within the Federal government, fear and chilling effects will continue if there is no commitment and clarity to prevent misuse or abuse of power by law enforcement and grant-making agencies. Evidence-based policymaking and transparency are critically important in our democracy, re-earn public trust, and heal the Asian American community. However, they are still significantly inadequate in the current implementation of NSPM-33. |
Thank you for your comment. We concur with the need for data and reporting on these critical issues. These issues will continue to be addressed as part of the process of the NSTC Subcommittee on Research Security and will have more to report in the coming months, especially alongside progress on NSF's Risk Assessment Center. |
| 75 | APA Justice | We highly commend the White House Office of Science and Technology Policy (OSTP) for its August 2021 announcement that “[w]e have to assiduously avoid basing policies or processes on prejudice — including those that could fuel anti-Asian sentiments or xenophobia” and the January 2022 requirement that “[a]agencies must implement NSPM-33 provisions and related requirements in a nondiscriminatory manner that does not stigmatize or treat unfairly members of the research community, including members of ethnic or racial minority groups.” However, it is important for Federal agencies to recognize that emphasis on compliance is far less productive and effective than building partnership. There has been serious loss of public trust and confidence in recent years due to both perceived and factual profiling and discrimination against scientists and researchers of Asian descent, especially those of Chinese origin. While it is minimally necessary to create consistency within the Federal government, fear and chilling effects will continue if there is no commitment and clarity to prevent misuse or abuse of power by law enforcement and grant-making agencies. Evidence-based policymaking and transparency are critically important in our democracy, re-earn public trust, and heal the Asian American community. However, they are still significantly inadequate in the current implementation of NSPM-33. |
Thank you for your comment. We concur with the need for data and reporting on these critical issues. These issues will continue to be addressed as part of the process of the NSTC Subcommittee on Research Security and will have more to report in the coming months, especially alongside progress on NSF's Risk Assessment Center. |
| 76 | APA Justice | As much as a researcher will try to be accurate with a standardized form, it will never be complete. A researcher or a group of researchers will be vulnerable to selective or arbitrary allegations of violation when they become profiling targets, as some have in recent years. | Thank you for your comment. The NSTC Research Security Subcommittee continues to work closely with our colleagues in security agencies to address issues pertaining to prejudice and discrimination. It is imperative that implementation of NSPM-33 be conducted in a non-discriminatory manner and all available steps are being taken to ensure this is central to the Biden-Harris Administration's approach to research security. |
| 77 | APA Justice | On September 30, 2022, Science Magazine reported that the National Science Foundation (NSF) will "soon begin crunching several large databases to see whether there are scientists who failed to disclose ties to foreign institutions in their grant applications.” Will some researchers be selectively blindsided without knowledge about how this process works with the incomplete standardized forms? | Thank you for your comment. This comment has been shared with NSF. NSF plans to issue a statement to the community regarding the Foundation's efforts in this area. |
| 78 | APA Justice | There is no historical data or a measurable benchmark on the magnitude of past and existing cases, investigations, or allegations, as well as their current state and disposition at the agency and aggregate levels. It is unclear how the performance and effectiveness of the new policies and standards can be measured without a baseline. | Thank you for your comment. We strongly agree for the need of more and better data. It is a priority to collect such data and to be transparent in our use of such data. |
| 79 | APA Justice | In her talk to the American Association for the Advancement of Science on October 21, OSTP Director Dr. Arita Prabhakar cited the integrated use of data science and behavioral science to address inequity issues. On October 4, 2022, OSTP issued a "Blueprint for an AI Bill of Rights." Taken together, they signal the increasingly important role of Big Data, Data Science, and Artificial Intelligence (AI) in the future of US science and technology. However, there has been little or no clarification about privacy, data quality, bias in algorithms, and unintended consequences that may have disparate impact and serious civil rights ramifications. | Thank you for your comment. The Blueprint for an AI Bill of Rights addresses many of the issues that are cited, including privacy, data quality, bias in algorithms, and unintended consequences. |
| 80 | APA Justice | On June 30, 2022, Science Magazine reported a possible workshop on factors affecting the classification of federally funded research. The meeting is expected to revisit a Cold War-era policy, namely National Security Decision Directive 189 (NSDD-189), that sets openness as the -gold standard and says any classification of fundamental research should be kept to a minimum It is unclear how this may impact the implementation of NSPM-33. We request the Biden Administration to reaffirm NSDD-189, as NSF and the National Science Board called for in 2018 and was done in 2010 under the Obama Administration in 2010 and in 2001 under the Bush Administration. | Thank you for your comment. This suggestion is being taken very seriously by the NSTC Research Security Subcommittee. |
| 81 | Katherine Ware, MIT | For the current and pending form - please make it sortable, and easier to update. Every time I fill one out for a Principal or Key person, I have to erase or move entries that have changed, or start over with a blank. I also cannot unclick the "pending" or "current" button, one button always stays clicked, so an entry is never truly blank. If the instructions are to have current first, then pending, make this easier to use, sort, and update. | Thank you for your comments. As new systems are developed to support electronical development of the Common Forms, useability will be an important component of new systems development. |
| 82 | University of Pennsylvania | The University of Pennsylvania (UPenn), like many other institutions, has proactively developed tools to assist our research community in accurately reporting the required information for Other/Current and Pending Support. At UPenn, this included working with a third-party vendor to create a system that aggregates information about the researcher’s activities from several different UPenn systems to create a single dashboard that provides accurate, up-to-date information (including amount of effort) for inclusion in the Other Support Form. The tool has been a key to our ability to ensure compliance with the reporting requirements for Other Support. The required use of SciENcv as the only acceptable system for the preparation of Current and Pending (Other) Support forms will require manual entry of numerous data fields and introduce the possibility of user error when transcribing information from our existing tool into the SciENcv system. It will increase the burden on our research community, both for investigators re-entering data from existing software tools and for research administrators who must review and certify the accuracy of such reports. An API directly into SciENcv for the Current and Pending (Other) Support information would allow institutions to send accurate data that is sourced from existing systems of record and researcher-entered data that has been reviewed within the institution’s existing dashboard tool directly into SciENcv. An API would ensure higher levels of accuracy in meeting federal requirements for transparency of Other Support provided by investigators, allowing agencies to make better informed funding decisions. We encourage NSF to ensure that such an API is available prior to the October 2023 effective date for the required use of SciENcv for Other Support reporting. If it is not feasible for an API to be ready by the October 2023 deadline, the option for users to upload a data file (e.g., csv) or other fillable template to populate in SciENcv would be extremely helpful. |
Thank you for your comment. As the Implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 83 | University of Maryland | Currently there is substantial confusion about the need to disclose consulting activity and much of this confusion stems from the table entitled Pre- and Post-award Disclosures Relating to the Biographical Sketch and Current and Pending Support. The Federal Register notice includes a link to this table, in the document titled Common Current and Pending (Other) Support Form, and thus the table is included in this request for comment. Before the forms can be reviewed effectively, we believe that it is essential to address the substance of the disclosure requirements. Recommendation: If the five-page Disclosure Table dated September 1, 2022 (“Disclosure Table”) is intended to become the official multi-agency policy on disclosure requirements, we strongly encourage that it is posted as a stand-alone Request for comments in the Federal Register. | Thank you for your comment. Multiple research funding agencies have seen a significant reduction in questions regarding the types of activities to be reported upon issuance of the Disclosure Tables. The tables, however, will be modified to address changes made as a result of the public comment process. With regard to consulting, the consulting instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 84 | University of Maryland | The five-page Disclosure Table is confusing and thus it does not meet the first “Objective” of the Guidance for Implementing NSPM-33 on National Security Strategy for the US Government - Supported Research and Development (“NSPM33 Guidance”), which reads: “Provide clarity regarding disclosure requirements…” (page 2). The September 1, 2022 disclosure table includes descriptions of activities that are unclear and, in some cases, contradictory. For example, one entry in the “Type of Activity” column states that, “Consulting that is considered part of an individual's appointment/agreement with their home organization and consistent with the proposing organization's "Outside Activities" policies and procedures” does not need to be disclosed. We are unclear how consulting could be both part of an individual’s appointment and subject to an Outside Activity policy simultaneously. Furthermore, while this example states that consulting that does not require disclosure, a prior entry in the “Type of Activity” column implies that consulting does require disclosure. That entry reads, “All projects (including this project) currently under consideration from whatever source, and all ongoing projects, irrespective of whether support is provided through the proposing organization, another organization or directly to the individual and regardless of whether or not they have monetary value” (emphasis in original) must be disclosed in the statement of Current and Pending Support. The lack of clarity in the Disclosure Table has the potential to lead to significant but unintentional oversights in what is reported in the statement of Current and Pending Support. In addition, it adds to the challenges that institutions face when trying to provide guidance to faculty and effective oversight of the proposal disclosure process. We recommend the following language for consideration as an alternative to the consulting entry above: “Consulting that involves any of the following must be disclosed in the statement of Current and Pending Support: (1) a foreign entity; (2) research; or (3) the transfer of technical information or intellectual property in the same or similar applications to the project/statement of work being proposed.” |
Thank you for your comment. Multiple research funding agencies has seen a significant reduction in questions regarding the types of activities to be reported upon issuance of the Disclosure Tables. The tables, however, will be modified to address changes made as a result of the public comment process. With regard to the specific issue of consulting, the Current and Pending (Other) Support Common Form instruction have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 85 | Michigan State University | One way to significantly reduce the burden is by recommend continued engagement with stakeholders for input throughout the review process. This would allow research active faculty to use the same form for federal proposals and project reports across agencies. Not only would this reduce burden but would likely reduce confusion of what needed to be disclosed, which in turn, would improve the accuracy of the information provided. If there are varying informational needs by agencies driven by statute, there could be a customizable section of the standard form developed subject to OSTP oversight and agreement that required. | Thank you for your comment. We recommend that the NSTC Research Security continue engagement with the research community at the OSTP level, and, not on an individual agency by agency basis. With regard to the issue of reporting, the Current and Pending (Other Support |
| 86 | Michigan State University | Please clarify, whether or not, in-kind contributions only need to be disclosed if they include a time commitment. The NSPM-33 Implementation Guidance Pre- and Post-award Disclosures Relating to the Biographical Sketch and Current and Pending (Other) Support table, indicates that in-kind resources only need to be disclosed if they have a time commitment, however, the instructions on the Current and Pending (Other) Support form indicates “whether or not it has an associated time commitment”. Also, the required fields for the in-kind section are too restrictive and confusing. For example, we do not have good examples of what is meant by “time commitment” for in-kind resources, such as equipment. The instructions for Person Month(s) in the in-kind section refer to “…how much time the individual anticipates is necessary to complete the scope of work on the proposed project or award,” which would not apply to some types of in-kind contributions. We would like to suggest capturing descriptive information on the in-kind contribution and allow for N/A responses in the Person Months and Dollar Value fields. | Thanks for your comment. If there is/will be a time commitment, then the estimated time commitment, or a reasonable estimate, should be provided, even if the specific amount of effort is unknown. If there is no associated time commitment, then the in-kind contribution does not need to be reported. The instructions have been modified for increased clarity. |
| 87 | University of North Carolina at Chapel Hill | While we understand flexibility for disclosure requirements is mandated, or desirable, among the different agencies, we would like to share a few specific concerns. Allowing an agency to create variances to require additional information is understandable but drains harmonization efforts and increases the administrative burden on researchers and institutions. To help ease this possible and unintentional administrative burden, we suggest: * any agency variations be handled through separate, agency-specific additional forms rather than new overall agency disclosure form. Such a process would allow for harmonized common forms for the biosketch and the current and pending (other) support documents, ones which are familiar to the researchers. Additionally, it would likely reduce the chance of data inaccuracies since the researcher would be accustomed to the common format to provide their core information, then include additional information if needed. * one central website where all approved, agency-specific additional forms are listed. * any agency variations be released on a set schedule (e.g., every s ix months or once a year); the schedule is standardized across all federal agencies to al+D4low for planning and consistent tracking. * collection of the current and pending other support common form should not be required across all agencies until a proposal has reached the stage of possible award across, using a mechanism, such as National Institutes of Health's (NIH) long standing Just-ln-Time process or the newly proposed National Science Foundation (NSF) process. |
Thank you for your comments and suggestions. This feedback will be seriously considered as we continue forward in the process of adopting and implementing the Common Forms. |
| 88 | University of North Carolina at Chapel Hill | Appointments: We recommend definitions and associated examples for the different types of academic, professional, and institutional appointments. For example, for someone with an " honorary" appointment at a foreign university,... Appointments: ...we suggest an example indicating if the appointment should be listed in the academic or institutional section. Websites: We recommend that the instructions directing researchers to list websites (URLs) are harmonized with current agency guidelines, such as the NIH guidelines that restrict URLs in submission paperwork. |
Thank you for your comments. The instructions for the Current and Pending (Other) Support Common Form have been revised to state, "In this section, please disclose all in-kind contributions with an estimated dollar value of $5000 or more and that require a commitment of the individual’s time. An in-kind contribution is a non-cash contribution provided by an external entity that directly supports the individuals’ research and development efforts. An in-kind contribution may include but is not limited to: real property; laboratory space; equipment; data or data sets; supplies; other expendable property; goods and services; employees or student resources. In-kind contributions with an estimated value of less than $5000 need not be reported." It is not possible, however to develop a one size fits all definition of In-kind contributions given the variance in agency missions. |
| 89 | University of North Carolina at Chapel Hill | All Projects: In the section titled, "All projects", the text includes a reference to In-Kind contributions. Researchers have expressed confusion regarding this section because of the in-kind reference. While it may be intended that reporting in this section is for research awards only, we recommend removing the In-Kind text "(e.g., even if the support received are in-kind contributions such as office/laboratory space ... " to decrease confusion ion. Maintaining a separation of in-kind text wi+D4ll make it easier for researchers to complete the forms successfully. | Thank you for your comment. The template has been revised to remove the phrase in-kind contributions from that projects section to avoid confusion. |
| 90 | University of North Carolina at Chapel Hill | * In the In-kind sections, we recommend further clarification on the goals and flexibility of reporting " in-kind" contributions. Many in-kind contributions reflect a variety of circumstances. For example, it's easy to provide a start-to-end date for an in-kind postdoctoral researcher but it's not as easy to include a Iist in-kind contribution of assays or cell lines, which may take place over time D96as the research project develops. | Thank you for your comment. The instructions have been modified to remove the start and end date. In lieu of these data elements the template will specify "receipt" date. |
| 91 | University of North Carolina at Chapel Hill | Consulting: The current text seems to assume an activity processed through any institution's "Outside Activity" policy means the activity falls within an individual 's appointment; this text creates confusion for researc hers and institutions a like. It is unclear if this information is being requested to illuminate some possible scientific overlap or other elements related to a possible conflict of commitment. While UNC does have an "Outside Activity" policy which delineates a cap on outside effo1t (similar to many universities), the activities approved under this "Outside Activity" process are considered, and treated, as external to the University and are not considered part of any individual's institutional duties or responsibilities. For example, any executed contracts are between the individual and the external entity, the University provides no support for these contracts, the University is not a party to them, and the conducted work is not considered within the scope of the individual's appointment. These external activities may be subject to some UNC policies, such as intellectual property or conflict of commitment, but are not a defined part of any institutional duties. It is our perception the intent of these sections is for relevant external activity information to be disclosed. However, the current text would mean nearly all of that activity by researchers would not be reportable. To avoid confusion, we recommend careful considerate ion of the intent of the two sections which currently have "consulting" in their description and whether the goal is capture reporting of profess ional knowledge-based external work where research, or research-like, activities are occurring and where there's a desire to create transparency into these activities. We would further suggest refinement to these sections could create greater congruency with the section titled "Travel supported/paid by external entity for research activities with associated time commitment" if the text is adjusted in similar vein. |
Thank you for your comment. With regard to the issue of consulting, the instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 92 | University of North Carolina at Chapel Hill | Startup Company lnforrnation: The reasoning for the reporting distinction between startup companies using organization versus non-organi zation licensed IP is unclear. The benefits of such IP distinctions are captured in the required reporting of financial relationships in conflict of interest policies. If the aim is to obtain information regarding possible conflict of commitment due the time effort related to startup companies, we recommend that these two sections be combined, and further details be provided as what information should be disclosed. | Thank you for your comment. This particular insertion to the Disclosure Table was provided by NSF and the response is as follows: NSF does not collect financial conflict of interest information from individuals who work at awardee organizations. NSF does require disclosure of current and pending support, including start-up companies based on non-organization licensed IP to assess capacity, duplication or overlap. Generally, IHE technology transfer offices track startup companies to license and commercialize technology developed at the university only. The intent of the C&P disclosure was not to impose undue burden by collecting information has already been reported elsewhere. |
| 93 | University of Wisconsin-Madison | Burden time: We believe that the burden time (hours) is underestimated. A researcher may either create the forms themselves or have an administrator help them prepare the forms. A college or school-level administrator may need to review the forms generated. Then, a central administrator may spend time reviewing the documents, verifying their accuracy to the best of their abilities, and ensuring that they adhere to sponsor requirements. If any questions arise or any items need to be corrected, back and forth can occur, and multiple iterations of documents may be generated before the final versions are ready to submit. In total, the preparation of disclosure forms may take far more than four to eight hours per proposal. | Thank you for your comment. The list of products should be organized by senior key personnel in a way that best demonstrates their ability to carry out the research proposed. Each Federal Research Funding Agency will determine the apprlicable length of the products required under agency specific requirements. |
| 94 | University of Wisconsin-Madis | Number of proposals: The number of proposals (estimated) is much too low at 47,900. The NIH alone processed over 58,000 applications in FY 20211. We understand that the intent is for these forms to be used by multiple Federal research funding agencies, which we hope includes all the members of the NSTC’s Research Security Subcommittee. If that is the case, then the number of proposals will be significantly greater than the number listed in the Federal Register Notice. If the number of proposals is exponentially larger than the number indicated, then the need for harmonization is paramount. Complex proposals may require even more time to prepare disclosure forms. Large grant applications, such as NIH P or U mechanisms, may involve many investigators. The number of investigators can result in 200 or more pages of Other Support documents for one application. Staff need to spend time preparing the documents, reviewing them, and then making any corrections. Additionally, sub-award relationships can pose challenges, because the sub-recipient and pass-through-entity may have differing interpretations of how to complete disclosure forms. These complicated situations, which are not rare, require more time than the Federal Register Notice estimates. | Thank you for your comment. There appears to be a mis-understanding of how the OMB/OIRA process will work for implementation of these Common Forms. One agency is responsible for completion of the PRA process, and, as such, the numbers reflected apply only to the sponsoring agency. Each Federal Research Funding Agency will be responsible for separately clearing use of the Common Forms through the OMB/OIRA Paperwork Reduction Act process. |
| 95 | University of Wisconsin-Madis | Number of submissions of Current and Pending (Other) Support: When capturing estimated Burden on the Public, the Notice only reflects burden related to proposal submissions. Submissions of Current and Pending (Other) Support documents are not only during time of proposal. Agencies may require submission of Current and Pending (Other) Support at proposal submission, during a Just-In-Time process or prior to making an award, and with annual progress reports. The Department of Energy (DOE) has a particularly burdensome requirement of PIs and other senior/key personnel, for which they must “after selection and during the life of the award” submit updates “if there are changes to the previously submitted current and pending support disclosures…within 30 days of the change.” For active researchers, the DOE rule creates unreasonable burden. The Federal Register Notice only mentions one point in time (proposal submission) and does not consider the varying points at which agencies require Current and Pending (Other) Support submissions, so there is significant unacknowledged burden. | Thank you for your comments. The comment has been shared with the Department of Energy for their consideration. |
| 96 | University of Wisconsin-Madison | Consulting: Consulting guidance in the NSPM-33 Implementation Guidance, Pre- and Post-award Disclosures Relating to the Biographical Sketch and Current and Pending Support (“Pre- and Post-Award Disclosures table”) could be clarified. The table draws a distinction between (1) consulting that is “part of an individual’s appointment/agreement” while being “consistent with the proposing organization’s ‘Outside Activities’ policies and procedures” and (2) consulting that “falls outside of an individual’s appointment/agreement.” These two categories of consulting do not align with our institutional policies and procedures on consulting. Our institutional guidance is that outside consulting arrangements are undertaken between the individual and the outside organization, and that our institution is not a party to the agreements. Our guidance explicitly states, “When negotiating, entering into and performing services under an agreement for outside consulting, a researcher is acting outside of the scope of his or her employment with the University. For this reason, such agreements are not negotiated, reviewed or signed by the University on a researcher's behalf.” (emphasis added) Yet, like many institutions, our policies allow researchers a certain amount of time to engage in outside consulting (two eight-hour days per calendar month). If an individual researcher wishes to engage in outside consulting more than two eight-hour days per month, the dean of the school or college must grant permission to do so. In our institution’s case, outside consulting may be “consistent with the organization’s ‘Outside Activities’ policies and procedures” if within the two eight-hour days per month or expressly permitted by the dean. Outside consulting is also explicitly “outside of an individual’s appointment/agreement.” This means that outside consulting at our institution could be considered part of both rows related to consulting in the Pre- and Post-Award Disclosures table. That being the case, how should our institution determine when consulting must be disclosed or not to the agency? An option would be to consider “consulting permitted under institutional policies and procedures” to be under the Disclosure Not Required column, whereas “consulting outside of that permitted by institutional policies and procedures” to potentially be disclosed. Further, earlier NIH guidance clarified that consulting related to an individual’s research endeavors should be disclosed. That potentially helpful qualifier is not included in the Common Disclosure Forms or Guidance. Our institution would appreciate knowing if the absence of that phrase means that any consulting, even if unrelated to one’s research endeavors, is meant to be included. |
Thank you for your comment. The instructions provided on the Current and Pending (Other) Support have been revised to improve clarity on what types of consulting must be reported as follows: "Consulting activities must be disclosed under the Proposals and Active Projects Section of the form when any of the following scenarios apply: •The consulting activity will require the senior/key person to perform research as part of the consulting activity; •The consulting activity does not involve performing research, but is related to the senior/key person’s research portfolio and may have the ability to impact funding, alter time or effort commitments, or otherwise impact scientific integrity; and •The consulting entity has provided a contract that requires the senior/key person to conceal or withhold confidential financial or other ties between the senior/key person and the entity, irrespective of the duration of the engagement." |
| 97 | University of Wisconsin-Madison | Types of Activity and data fields: For Current and Pending (Other) Support, our institution would appreciate examples of how to include information. There are certain types of activities, such as startup funding from an outside organization or a post-doctoral scholar supported by an external entity, whose research activities are not for the project being proposed and who have an associated time commitment, where the information does not fit neatly into the template typically used to disclose active or pending project information. The template for a project/proposal or in-kind contribution is quite rigid, with all fields marked as required. Previous agency guidance for an in-kind contribution, for example, allowed institutions to report either a dollar value or time commitment, but the proposed guidance appears to require both fields no matter what the circumstances. We would encourage flexibility in what fields are required and would encourage the NSTC Research Security Subcommittee to provide detailed examples that show explicitly where each Type of Activity in the Pre- and Post-Award Disclosures table belongs in the Biographical Sketch and Current and Pending (Other) Support form, including how the information should be entered. Given that our institution is still learning about what agencies’ expectations are, we would find such examples helpful in ensuring that our disclosures are addressing agency requirements. | Thank you for your comment. The instructions for completion of the in-kind contribution section of the Current and Pending (Other) Support have been modified to address this issue. |
| 98 | University of Wisconsin-Madison | Harmonization/variation: As mentioned earlier, our institution believes that an effective way to reduce burden would be to standardize forms to the greatest extent possible and eliminate variation across agency requirements whenever possible. We would like to strongly encourage that agencies must be rigorously defend requested deviations from the Common Disclosure Forms and that clear standards be created to determine the rare circumstances to make deviation requests. One example of a less common disclosure requirement is to include previous support from the past five years in a Current and Pending Support form. This is required by Department of Defense, and Department of Energy may also request this information. Also, NASA currently only requires co-investigators who spend 10% or more of their time on a project to provide current and pending support. Though we cannot commentson the utility of this information, we would recommend that unusual requests such as these be discontinued. We strongly recommend that disclosure requirements be standardized across all Federal research funding agencies. The greater the harmonization across agencies in the disclosure forms, the easier it will be for institutions and investigators to submit the necessary information. | Thank you for your comment. As the implementation Guidance states, any proposed changes or modifications to forms will be required to be submitted to the NSTC subcommittee for review and vetting. At points, agencies are bound by statute to include additional information. If the reasoning for a modification is not statutory, it will be scrutinized in greater depth. The goal is to ensure the highest level of standardization possible across federal agencies, and to encourage federal agencies to employ technological tools that lower burden for researchers. |
| 99 | University of Wisconsin-Madison | Defined terms: We ask that all terms be clearly defined and that a specific section of the instruction documents be devoted to definitions, not simply in footnotes. For instance, for the Biographical Sketch instructions, we ask for clarification on the difference between “position” and “appointment.” We also ask that the terms “academic”, “professional”, or “institutional” be defined. The terms are being used with a presumption that there is a collective understanding of each, but these terms can be interpreted and used in diverse ways. Clear examples of what is meant by “position”, appointment”, “academic”, “professional”, or “institutional” would also be helpful. | Thank you for your comment. The terminology "academic, professional, and institutional appointments" was specified in NSPM-33. In response to feedback received, definitions for each of the appointment/position categories have been added to a revised version of the Definitions Appendix in the NSPM-33 Implementation Guidance. |
| 100 | University of Wisconsin-Madison | Start-up company based on organization-licensed Intellectual Property (IP): The Pre- and Post-Award Disclosures table indicates that for a “startup company based on organization-licensed Intellectual Property (IP),” disclosure is not required. Given that Bayh-Dole enables an organization to designate an assignee subject to the same provisions as the contractor, we would suggest that this Type of Activity be amended to read, “Startup company based on Intellectual Property (IP) licensed from organization or organization's designee allowed under 37 CFR 401.14(k)(1).” | Thank you for your comment. We have modified the Disclosure table to address this issue as well as to incorporate other changes necessitated by revisions made to the Common Forms. |
| File Type | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |