Data Sharing Policy ESC

Enhancing the Diversity of the NIH-funded Workforce Program

Diversity Program Consortium Data Sharing Policy

Effective: March 16, 2016

The NIH has long recognized the need for a biomedical workforce that reflects the most talented scientific minds across all populations. With that in mind, in 2012, it convened the NIH Advisory Committee to the Director (ACD) Working Group on Diversity in the Biomedical Research Workforce to explore strategies to attract, prepare and sustain the interest of individuals in the scientific workforce, including those from underrepresented groups (NOT-OD-15-053). In response to the Working Group’s recommendations, which were endorsed by the ACD, the Enhancing the Diversity of the NIH-funded Workforce Program was established. The program through three integrated initiatives, (1) Building Infrastructure Leading to Diversity (BUILD), (2) the National Research Mentoring Network (NRMN) and (3) the Coordination & Evaluation Center (CEC), envisaged a national collaborative through which program awardees would work collectively as a Diversity Program Consortium, and collaboratively with the NIH, to develop novel and effective diversity-driven approaches to infrastructure and faculty development, student engagement, research training and mentorship for each career stage with dissemination of lessons learned to the broader research training/mentorship communities.

The National Institutes of Health and its Enhancing the Diversity of the NIH-funded Workforce Program awardees, hereafter referred to as the “Diversity Program Consortium,” will develop, implement and evaluate approaches to strengthen institutional capacity to engage and prepare individuals, including those from underrepresented populations, for successful careers in biomedical research. This Data Sharing Policy (“Policy”), which has been developed in conjunction with Diversity Program Consortium’s awardees and the Executive Steering Committee (ESC), describes how data within the Diversity Program Consortium is collected, shared, stored, and utilized for purposes of the consortium-wide evaluation. The Policy will be incorporated by reference into the terms and conditions of each U54 award.

To facilitate a national cross-site evaluation as well as provide a mechanism for reporting programmatic progress on intended goals, each site within the Diversity Program Consortium will capture, at intervals defined by the ESC, a consistent or core set of individual/student-, institutional/site- and faculty/mentor- metrics. The scope of the core, consortium-wide data elements (Hallmarks of Success) are defined in Appendix A, and have been developed by the Diversity Program Consortium, and approved by the ESC. The consortium-wide data elements may be altered over time as determined by the ESC. Each site will also capture site-level data elements, in support of a local site evaluation, as determined by each individual site. The program’s cross-site evaluation will include collection of both qualitative and quantitative data elements.

1. PURPOSE

The purpose of this Data Sharing Policy is to establish data collection, tracking and storage coordination requirements, to delineate specific administrative, technical and physical safeguards to assure data security and confidentiality, to describe access to and transfer of data to the Coordination and Evaluation Center (CEC) for use in the Diversity Program Consortium’s evaluation, and to provide a framework for use of Diversity Program Consortium data. This policy also delineates data ownership, rights, and grantee responsibilities. This policy is incorporated by reference into the terms and conditions of each U54 award, and compliance with the terms and conditions are required of all grantees.

2. PERIOD OF POLICY

The Data Sharing Policy outlining the Diversity Program Consortium data collection, sharing and dissemination, and analysis will remain effective five years after the end of the NIH Diversity Program Consortium awarded funding period (or period of no-cost extension) for each awardee.

2A. Policy Modification

This Policy is subject to change. The Diversity Program Consortium, through the ESC, which includes representation from each awardee and NIH, may propose modifications or extension of this policy for NIH’s review and approval. The original policy shall remain in effect until a revised Policy is approved by NIH.

3. GOVERNANCE, AUTHORITIES, DATA RIGHTS & COMPLIANCE

The Coordination and Evaluation Center (CEC) will be responsible for overseeing implementation of the activities described herein, in collaboration with the ESC and NIH. NIH will be responsible for oversight and adherence of the Diversity Program Consortium to this data sharing policy.

The NIH is committed to protecting the rights and privacy of those whose information is collected during the conduct of its funded research and awardees will be responsible for compliance with this Policy, as outlined in the terms and conditions of each U54 award. This Policy is made under the NIH’s authority to conduct and fund research; to provide training/training assistance; to collect information as to the practical application of such research and training activities; to assemble accurate data to evaluate research priorities and scientific opportunities; and to maintain records in connection with these or other agency functions (42 U.S.C. §§ 241 and 282, and 44 U.S.C. § 3101).  This Policy incorporates by reference the NIH’s data sharing policies for research and the Family Educational Rights and Privacy Act (FERPA) guidelines on use or disclosure of student educational records in the conduct of research, as amended [See: http://sharing.nih.gov; http://www.grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm#goals; and http://www2.ed.gov/policy/gen/guid/fpco/ferpa/index.html].

As delineated in the CEC Funding Opportunity Announcement (RFA-RM-13-015), each “awardee will retain custody of and have primary rights to the data and software developed under these awards, subject to Government rights of access consistent with DHHS, PHS, and NIH policies. The CEC and the consortium will develop plans for data sharing among awardees. All de-identified evaluation-related data will be shared with the NIH at the conclusion of the award.” During the funding period, CEC will have responsibility for management and oversight of the aggregated dataset of consortium-wide data, and the individual sites will retain ownership over the use of site-level data.

All awardees and non-consortium parties (Section 7B) granted access to Diversity Program Consortium data will adhere to responsible data use, security and disclosure provisions as outlined in section 6, Data Security & Use. Failure to abide by the terms and conditions of this Policy including data security/disclosure provisions may result in (i) denial of further access to the Diversity Program Consortium data, (ii) denial of access to NIH-funded resources, and (ii) federal or state penalties. These terms apply to each awardee (CEC, BUILD and NRMN participants). Liability will be aligned with data ownership and rights (See also Section 8 for details regarding how disagreements/disputes will be resolved).

4. DEFINITIONS

The Diversity Program Consortium is composed of awardees funded under one of the Enhancing the Diversity of the NIH-funded Workforce Program’s initiatives [BUILD, NRMN, CEC; Appendix B: Participating Institutions].

4A. Data Categories:

1. Consortium-wide Data: Data elements collected from each awardee to provide information required to complete the Consortium Wide Evaluation Plan (CWEP), and reflecting the goals articulated in the cooperative agreement funding opportunity announcement¹ to enable evaluation of intervention effects on outcomes defined by the Hallmarks of Success. Consortium-wide data will also include secondary data, including but not limited to institution records, demographic data, or other existing resources that are collected from all awardees as outlined in the ESC approved CWEP. Consortium-wide data elements will be submitted by all Member Institutions to the CEC, who will conduct quality review and risk assessment, de-identify data, and provide data for consortium use. Consortium-wide data, when submitted to the CEC, aggregated, and de-identified, is under management and oversight of the CEC on behalf of the ESC (hereafter referred to as DPC Data)². DPC data is accessible to all members of the consortium and is subject to the terms of this Data Sharing Policy. The Publications and Presentations (P&P) Policy, developed by the P&P subcommittee and approved by the ESC, outlines the procedures for consortium-wide data use. ³ See Appendix C for a detailed listing of Consortium-wide data elements.

2. Site-Level Data: Data elements collected by individual sites to evaluate the impact of site-level variables on outcomes of interest to the site. Site-level data includes both consortium-wide data elements (defined by the Hallmarks of Success and the consortium-wide evaluation plan as the data elements collected across all consortium sites) and non-consortium-wide data elements (defined as data collected only at individual sites). Member Institutions retain ownership of the use of site-level data and the publication of site-level analyses. Analyses and publications of site-level data will follow the process for tracking and review outlined in the Publications and Presentations Policy. Once the site-level data is aggregated with data from all sites and de-identified, it becomes classified as consortium-wide data for consortium use (see above), and subject to the terms of this Data Sharing Policy.

3. Third Party Data: Data collected from BUILD site partner institutions or NRMN subawardees, which can include both consortium-wide data elements and non-consortium-wide data elements. Third party institutions retain ownership of the use of their data unless and until the data is de-identified and aggregated as consortium-wide (see above). Third party data are subject to the terms of this Data Sharing Policy for all consortium-wide data elements, unless otherwise agreed upon in writing between a Member Institution and the Third Party that predates this Data Sharing Policy. In the event that such Third Party agreement does not allow for the sharing of data as described in this Data Sharing Policy, the Member Institution shall attempt to secure permission for the sharing of Third Party Data consistent with the objectives of the Diversity Program Consortium.

4B. Consortium-Wide Data Description

Consortium-wide data, under the following broad categories, will be collected during the funding period by the Diversity Program Consortium: (a) student/mentee, (b) institutional/site, and (c) faculty/mentor [see Appendix C for details regarding data elements to be collected].

1. Student/mentee: data elements collected by sites/awardees to enable evaluation of intervention effects on student/mentee-level hallmarks and outcomes

2. Institutional/site: data elements collected by sites/awardee to enable evaluation of impact of interventions on institutional-level hallmarks and outcomes.

3. Faculty/mentor: data elements collected by sites/awardees to enable evaluation of impact of activities on faculty/mentor-level hallmarks and outcomes.

Consortium-wide data may include, for example, student-participant characteristics (e.g. information from education records), faculty/mentor characteristics (e.g. time elapsed since degree completion, authorship/publication record, history of NIH vs. other sources of grant funding) and institution characteristics (e.g., geographic location, diversity of faculty/student population, number of grants submitted vs. funded, summary data on trainees enrolled in STEM majors vs. completed degrees in STEM fields) as well as interview and survey-derived data (e.g. demographics). It may also include tracking of student/participant and faculty/mentor participation in online and face-to-face services/resources (e.g., faculty e-mentorship training modules, student e-mentoring sessions).

4C. OMB and IRB Approvals/Clearance

Consortium-wide data collection instruments (e.g. questionnaires, surveys, scripts, interviewer instructions, etc.), processes, and schedules for data collection developed on behalf of the Diversity Program Consortium will adhere to those approved by the Office of Management and Budget (OMB) as well as those of the UCLA-CEC Institutional Review Board (IRB), or other relevant IRB governing the consortium-wide data collection. All site-specific data collection instruments and procedures must adhere to the governance of the local site IRB.

5. DATA SUBMISSION to Coordination & Evaluation Center (CEC)

The CEC will provide Diversity Program Consortium awardees with a process for submission of the consortium-wide data, which will include a description of data to be submitted, submission timeline, and access to the secure consortium data repository for the transfer of data. This process will include quality assurance activities to be completed by awardees and quality review to be completed by the CEC. Once consortium-wide data is submitted to the CEC, the CEC will perform quality review, risk assessment and de-identification. Quality review will include confirmation of identifiers for linking with other consortium-wide data, assessment of valid values, explanation for missing data, and completion of logic or skip patterns. Initial quality review will be completed within 10 days unless otherwise agreed upon. Disclosure risk assessment will include review for sensitive and infrequent (rare) data points that could be used to identify individuals. For both quality reviews and disclosure risk assessments, the CEC will work with each institution to resolve any outstanding issues with data quality. Diversity Program Consortium awardees will work collaboratively with CEC to meet Consortium-defined standards of data completeness and quality.

Diversity Program Consortium awardees may choose to share additional site-specific data with other awardees and/or the consortium for various analyses and collaborative research opportunities. The CEC will provide a portal for secure storage of site-level data that will not be shared with Diversity Program Consortium and also a procedure for sharing site-specific data elements, should a site choose to do so.

6. DATA SECURITY & USE

6A. Data Security

To protect the rights and privacy of individuals whose information is collected within this multisite and collaborative research project, and to ensure the confidentiality of the data to be shared, all parties under this Policy agree to adhere to the following:

1. The CEC Tracker⁴ will assign each individual identified as a participant (both students and faculty) a unique nine-digit numeric identification number, hereafter called the cross-site ID, at the time the Diversity Program Consortium awardee submits the roster of participants to the CEC. This will allow the CEC to maintain longitudinal data as each individual progresses through their career. The CEC Tracker will also allow authorized BUILD site administrators to add identifying elements to the Tracker to assist with longitudinal tracking (e.g., site-level identification numbers). Restricted information that identifies participants (such as name, address, student/faculty institutional ID number) will be maintained by the CEC Tracker and managed by the UCLA Computer Technology Research Lab (CTRL). In order to maintain confidentiality of individuals, identifiable information on BUILD participants will only be provided to authorized educational officials at individual awardee institutions; identifiable information will not be shared across BUILD and NRMN sites.

2. The physical security of the CEC’s Tracker will be maintained at the CTRL with multiple levels of security compliant with HIPAA’s data security standards. The Tracker will operate behind a firewall on a private IP space that is inaccessible to regular Internet traffic. Access to the CEC Tracker will require authentication with a virtual private network (VPN) appliance in addition to CEC Tracker web application account verification.

3. All consortium-wide data will be de-identified prior to use in analyses or publications. Individual sites may maintain identifiable information on their own participants and site-specific data, which is subject to the governance of their local IRB.

4. All Diversity Program Consortium awardees will implement, maintain, and use appropriate administrative, technical, and physical security measures to preserve the confidentiality and integrity of physical data to include storage in a secure and locked location; all electronic data collected by the awardee will be stored and maintained in a password-protected directory maintained behind the institutional firewall of the awardee with password access granted only to approved staff or officials or, for shared data, in the secured consortium data online repository.

6B. Data Use

Use of the data collected under this Policy is to support evaluation of DPC funded research, its activities and services. Accordingly, all data collected reviewed, and aggregated at the consortium level, will be de-identified and free of individually identifiable information that would allow linkage to individual participants prior to distribution to consortium members, non-consortium parties and the general public. As an added measure to ensure the integrity and security of the consortium data, the CEC will oversee data requests, release/transfer, and use as approved by the Publications and Presentations Policy from the ESC.

Unauthorized use or disclosure of consortium data should be reported to the CEC and the NIH within one day of discovery and should include both corrective actions planned to prevent future unauthorized use/disclosure as well as efforts to mitigate any adverse effect of the unauthorized use/disclosure. Reports to NIH of unauthorized use or disclosure of consortium data should be reported to the Program Official, who will implement appropriate review procedures within NIH. Diversity Program Consortium awardees agree not to use or disclose the data except as permitted or required under this Policy and/or required by applicable law.

7. METHOD OF DATA SHARING

The NIH and the DPC recognize that data sharing, in multisite and collaborative research, requires compliance with organizational policy, IRB guidelines, and local, State and federal laws and regulations to safeguard participant privacy and ensure data protection. This Policy describes access to various categories of data covered under the Policy. To share DPC data, awardees will use a secure file transfer service over an encrypted connection only. Additionally, physical and/or electronic research record data will be maintained securely, retained for up to 5 years following the end of the program funding period.

7A Data Access by Consortium Members

Data access for use in consortium-wide publications and/or presentations by consortium members will be managed through the Publications & Presentations (P&P) subcommittee within the Executive Steering Committee and governed by the P&P Policy. Requests for data access must be made using the process outlined by the P&P subcommittee and detailed in the P&P Policy. The P&P subcommittee will review and approve all requests for the use of consortium-wide data in research studies, publications and presentations, and approve the publication and presentation of results from research using consortium-wide data. Once the P&P subcommittee approves proposed data use, the CEC Data Coordination Core will be notified and will provide secure access to requested data. The de-identified data will be released through licensed access via password protected sections of the CEC public website.

Each site will have a username and password allowing full access to their own site-specific data, which is critical for their internal operations and local presentations. Scholarly activity using site-specific data must be shared with the P&P subcommittee for purposes of tracking and archiving all Diversity Program Consortium publications and presentations.

7B Data Access for non-Consortium Parties

Access to consortium-wide or site-specific data for parties outside of the Consortium will require the outside party to submit a Data Request Form describing the proposed use of the requested data and identifying a sponsor for the request that is a member of the Consortium. Access to site-specific data from an Outside Party requires sponsorship by the PI of that site. Documentation of Human Subjects Ethics training and Institutional Review Board approval or exemption should be provided with the data request. Outside Parties shall agree to use requested Data only for approved use. All such requests will be reviewed by the P&P subcommittee according to the P&P Policy. In addition, within 24 months of the termination of funding the de-identified data will be made available for public use, consistent with the NIH data sharing agreement for NIH funded research, through an open access portal on the CEC public website.

7C Data Dissemination

The processes for disseminating research findings stemming from data generated by the Diversity Program Consortium will be developed and implemented by the CEC in collaboration with the P&P subcommittee. The overall dissemination plan will include the following elements: timeline (within and external to consortium, during grant funding, post grant-funding period, etc.); dissemination methods (e.g., presentation at national scientific meetings; publications in peer-review journals or other publications; webinars sponsored by the Consortium; short articles in Consortium -sponsored newsletters/websites, etc.; Annual or Bi-annual grantee conferences; presentations at community organizations, etc.). This dissemination plan will be established by the ESC.

8. DISPUTES

As per the Cooperative Agreement Terms and Conditions, any dispute that arises between awardees and the NIH on scientific/programmatic matters, within the scope of the awards, will be brought to a Dispute Resolution Panel. The panel will be composed of an Executive Steering Committee Awardee designee chosen by the Diversity Program Consortium, one ESC NIH designee, and a third designee, to be chosen by the other two panel members, with expertise in the topic area relevant to the dispute. The panel will convene a meeting and work with the parties to achieve a resolution. To the extent permitted under the terms and conditions of an awardees award, an awardee retains the right to appeal an adverse action, beyond this special dispute procedure, in accordance with PHS regulation 42 CFR Part 50, Subpart D and DHHS regulation 45 CFR Part 16.

APPENDICES

Appendix A: List of Consortium-wide data elements (Hallmarks of Success)

	Individual-Level (Student) Hallmarks
IND-B1	Psychosocial Variables (Including: Academic and Scientific Self-Efficacy, Science/Researcher Identity, Participation in Academic/Professional Student Organizations, Satisfaction with Faculty Mentorship, Social Integration/Perceived “fit” with University Setting, Intent to Pursue Career in Biomedical Research)
IND-B2	Pursuit of Biomedical Science Undergraduate Degree
IND-B3	Retention and Persistence in Biomedical Science Discipline
IND-B4	Participation in Undergraduate/Summer Biomedical Research Training in Labs or Similar Research Environment
IND-B5	Poster or Presentation at Scientific Conferences
IND-B6	Submitted Applications and Receipt of Awards, including Research Fellowships and Scholarships
IND-B7	Evidence of Biomedical Research Career Preparedness (grades, GRE; std. exams)
IND-B8	Authorship/Co-Authorship of Peer-Reviewed Publication(s)
IND-B9	Completion of Undergraduate Degree in Biomedical Science Discipline
IND-B10	Application & Acceptance to Attend Graduate Program in Biomedical Science Discipline
IND-B11	Entrance to Graduate Program in Biomedical Science Discipline
	Institutional-Level Hallmarks
INST-B14	Increase, enhance, and/or develop Inter-Institutional collaborations to achieve BUILD outcomes related to research, mentorship, and faculty development (e.g., linkages with Community Colleges or other partner institutions, collaborations and postdocs at Research-Intensive partner institutions, engagement with NRMN)
INST-B1	Improved Undergraduate Retention Rates of Students in Programs Relevant to BUILD (biomedical/ behavioral sciences)
INST-B2	Increased Participation in Mentoring Activities (Students and Faculty) in Programs Relevant to BUILD
INST-B12	Increase in Number of Student Research Training Opportunities for students and faculty in Programs Relevant to BUILD
INST-B13	Increase in Number of underrepresented Students Enrolled in BUILD Biomedical Research-Related Programs
INST-B3	Increase in Number of Underrepresented Students Retained in BUILD Biomedical Research-Related Programs
INST-B17	Institutional commitment to BUILD sustainability evidenced by site maintenance of key elements of program interventions after grant period**
INST-B18	Increased institutional commitment to sustaining activities of BUILD (i.e. research infrastructure, FTE, scholarships, space), changing the academic culture, culture of faculty promotion, tenure, research development (release time), stronger emphasis on student mentoring and advising to increase institutional outcomes, curriculum improvements**
INST-B19	Increase enrollment and participation of underrepresented Students in biomedical research fields
INST-B20	Increase in participation of faculty in mentorship activities - defined as: Increase in number of faculty seeking and participating in mentor training
	Faculty/Mentor-Level Hallmarks
FAC-B15	Increase in the number of trainees mentored in Programs Relevant to BUILD
FAC-B4	Increase in Participation in Professional Development Activities for faculty in Programs Relevant to BUILD
FAC-B12	Increase in Faculty Participation in Mentorship Activities in Programs Relevant to BUILD (may include structured activities to train the next gen of biomedical scientists) **
FAC-B8	Increased Research Productivity in Publications, Grant Submissions and Awards as PI, multi-PI and/or collaborator for faculty in Programs Relevant to BUILD
FAC-B16	Increased quality of mentoring (Student and mentor perceptions)
FAC-B1	Change/Increase in self-efficacy as instructor, mentor and/or researcher
	Demographic/Background Variables (Student, Faculty, Institution)
Student	School/Institution
Student	Geographic Location
Student	Gender
Student	Ethnicity
Student	Disability Status
Student	Socioeconomic Status
Student	High School GPA
Student	Standardized Test Scores
Faculty/Mentor	Institution
Faculty/Mentor	Gender
Faculty/Mentor	Race/Ethnicity
Faculty/Mentor	Disability Status
Faculty/Mentor	Socioeconomic Status
Faculty/Mentor	Field of Study
Faculty/Mentor	Years Since Degree
Faculty/Mentor	Prior NIH Support
Faculty/Mentor	Prior Research Experience
Faculty/Mentor	Prior Mentoring Experience
Institution	Institution Type
Institution	MSI Status
Institution	Geographic Location
Institution	Public/Private Sponsorship
Institution	Faculty/Staff Diversity
Institution	Student/Client Diversity
Institution	Collaborations with Institutions
Institution	Research Intensiveness
Institution	Mission

Appendix B: Participating Institutions (as of Effective Date)

PI Name	Primary Institution Name	Project Title
BUILDING INFRASTRUCTURE LEADING TO DIVERSITY (BUILD)
Crespo, Carlos J.	Portland State University	Enhancing Cross Disciplinary Infrastructure and Training at Oregon (EXITO)
D'amour, Gene Foroozesh, Maryam Giguette, Marguerite Morgan, Kathleen M.	Xavier University of Louisiana	Building Integrated Pathways to Independence for Diverse Biomedical Researchers
Echegoyen, Lourdes (Contact) Cox, Marc Aley, Stephen B Morera, Osvaldo F Boland, Thomas Nazeran, Homayoun Grineski, Sarah Collins, Timothy Williams Nazeran, Homayoun	The University of Texas at El Paso	BUILDING SCHOLARS
Kamangar, Farin (Contact) Sheikhattari, Payam	Morgan State University	ASCEND Training Model to Increase Diversity in Biomedical Research
Khachikian, Crist Simon (Contact) Chavira, Gabriela Saetermoe, Carrie L Shiffrar, Margaret M.	California State University Northridge	BUILD@CSUN
Kingsford, Laura (Contact) Urizar, Guido G.	California State University Long Beach	CSULB Building Biomedical Research Program
Kuleck, Gary A (Contact) Mathur, Ambika Welch, Sally	University of Detroit Mercy	REBUILD Detroit
Marquez-Magana, Leticia M (Contact) Ben-Zeev, Avi Bibbins-Domingo, Kirsten Estrada, Mica	San Francisco State University	SF BUILD: Enabling Students to Represent in Science
Rous, Philip J (Contact) Lacourse, William Richard	University of Maryland Baltimore County	STEM BUILD@UMBC
Taylor, Barbara Elan (Contact) Hueffer, Karsten Reynolds, Arleigh	University of Alaska Fairbanks	Biomedical Learning and Student Training (BLaST) Program
NATIONAL RESEARCH MENTORING NETWORK (NRMN)
Chiles, Thomas (Contact) Ofili, Elizabeth Okuyemi, Kolawole S Pfund, Christine E Vishwanatha, Jamboor K	Boston College	National Research Mentoring Network for a Diverse Biomedical Workforce
COORDINATION AND EVALUATION CENTER (CEC)_
Norris, Keith C (Contact) Seeman, Teresa Davidson, Pamela	University of California Los Angeles	NIH Diversity Program Consortium Coordination and Evaluation Center at UCLA

Appendix C: Definitions of data elements included in the Consortium-wide Evaluation Plan

This section is intended to provide an overview of the data elements included in the Consortium-wide evaluation plan.

1. Data elements include all items listed in the following protocols (actual protocols can be found on the Intranet here]:

Intranet Path

CEC tab  Consortium-Wide Evaluation Plan (CWEP)  OMB January 2016 Submission  Proposal

• BUILD Student Follow-up Survey (OMB Attachment 12)

• BUILD Faculty Follow-up Survey (OMB Attachment 14)

• BUILD Site Visit & Case Studies Protocol (OMB Attachment 23)

• BUILD Institutional Records & Program Data Requests (OMB Attachment 25)

• BUILD Implementation Reports (OMB Attachment 26)

• NRMN Data Warehouse Baseline Data (OMB Attachment 16)

• NRMN Faculty/Mentor Follow-up Survey (OMB Attachment 17)

• NRMN Student/Mentee Follow-up Survey (OMB Attachment 18)

• NRMN Institutional Context (OMB Attachment 22)

• NRMN Site Visit & Case Studies Protocol (OMB Attachment 24)

• NRMN Modules

  • Mentee Mentor Assessment (OMB Attachment 15),

  • Mentor Skills (OMB Attachment 19)

  • Grant-writing (OMB Attachment 20)

  • Coaching (OMB Attachment 21)

Table 1: Demographic /Background Variables listed in here come from various sources including for HERI surveys and IR data for BUILD and NRMN Data Warehouse for NRMN participants.

Table 1: Demographic/Background Variables (Student, Faculty, and Institution)
Student	School/Institution
Student	Geographic Location
Student	Gender
Student	Ethnicity
Student	Disability Status
Student	Socioeconomic Status
Student	High School GPA
Student	Standardized Test Scores
Faculty/Mentor	Institution
Faculty/Mentor	Gender
Faculty/Mentor	Race/Ethnicity
Faculty/Mentor	Disability Status
Faculty/Mentor	Socioeconomic Status
Faculty/Mentor	Field of Study
Faculty/Mentor	Years Since Degree
Faculty/Mentor	Prior NIH Support
Faculty/Mentor	Prior Research Experience
Faculty/Mentor	Prior Mentoring Experience
Institution	Institution Type
Institution	MSI Status
Institution	Geographic Location
Institution	Public/Private Sponsorship
Institution	Faculty/Staff Diversity
Institution	Student/Client Diversity
Institution	Collaborations with Institutions
Institution	Research Intensiveness
Institution	Mission

2. Items from various HERI surveys that map to Consortium Hallmarks and are CWEP Data elements are outlined in the following table.

KEY:

• TFS= HERI- The Freshman Survey (corresponding item number from 2015 survey noted in red)

• YFCY = HERI- Your First College Year (corresponding item number from 2016 survey noted in red)

• CSS = HERI- College Senior Survey (corresponding item number from 2016 survey noted in red)

• FAC = HERI- Faculty Survey

• B-SAFS = BUILD Student Annual Follow-up Survey

• B-FAFS = BUILD Faculty Annual Follow-up Survey

• Mentee AFS = NRMN Annual Student/Mentee CORE Annual Follow-up Survey

• Mentor AFS = NRMN Annual Mentor CORE Annual Follow-up Survey

Table 2. Items from various HERI surveys that map to Consortium Hallmarks and are CWEP Data elements are outlined in the following table.

Domain	Elements	Hallmarks	Data Source / Measure	Operational Definition
Student Student Student Student Student Student Student Student Student Student Student Student Student Student Student Student	Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations) Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations) Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations) Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations) Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations) Psychosocial Variables (Including Perceptions of Culture and Environment, Identity, Attitude, Aspirations)	IND-B1 IND-B1 (Cont.) IND-B1 (Cont.) IND-B1 (Cont.) IND-B1 (Cont.) IND-B1 (Cont.)	TFS YFCY CSS B-SAFS Mentee AFS	TFS Academic self-efficacy (see Appendix C1) #41 Scientific self-efficacy [see Appendix C2] (will be added to 2016 survey) Science Identity (see Appendix C3) #50 (Estrada Science Identity Items will be added to 2016 survey) Intent to pursue biomedical career Do you consider yourself: pre-med #24 Please indicate your intended major #25 Intended/actual/probable career #26 What is the highest academic degree that you intend to obtain? #33 Bachelor’s Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. In deciding to go to college, how important to you was each of the following reasons? To prepare myself for graduate or professional school. #40 Below are some reasons that might have influenced your decision to attend this particular college. How important was each reason in your decision to come here? #43 This college's graduates gain admission to top graduate/professional schools Engagement in research and satisfaction with faculty mentorship Since entering this college, how often have you: Asked a professor/teacher for advice after class #34 Below are some reasons that might have influenced your decision to attend this particular college. How important was each reason in your decision to come here? High school counselor advised me #43 What is your best guess as to the changes that you will: #51 Communicate regularly with your professors Work on a professor’s research project YFCY Academic self-efficacy (see Appendix C1) #14 Scientific self-efficacy [see Appendix C2] #37 Science Identity (see Appendix C3) #24, #36 Participation in academic and professional student organizations Since entering this college have you joined a pre-professional or departmental club? #26 This institution has contributed to my: Ability to work as part of a team #21 Since entering this college, how often have you felt: #12 Isolated from campus life Unsafe on this campus Since entering this college, how has it been to: Develop close friendships with other students #15 Please indicate the extent to which you agree or disagree with the following statements: #18 My college experiences have exposed me to diverse opinions, cultures and values I see myself as part of the campus community I feel a sense of belonging to this campus I feel that I am a member of this college I feel valued at this institution Please rate your satisfaction with this college in each area: Overall sense of community among students #20 Since entering this college, indicate how often you: Discussed course content with students outside of class #28 Engagement in research and satisfaction with faculty mentorship Since entering this college, how often have you: Asked a professor/teacher for advice after class #17 Since entering this college, how often have you interacted with the following people (eg, by phone, e-mail, text, or in person): #9 Faculty during office hours Faculty outside of class or office hours Academic advisors/counselors Graduate students/ teaching assistants Since entering this college, how often have you felt: #12 That faculty provided me with feedback that helped me assess my progress in class That faculty encouraged me to ask questions and participate in discussions Please indicate the extent to which you agree or disagree with the following: #18 Faculty showed concern about my progress Faculty empower me to learn here Faculty encouraged me to meet with them outside of class At least one faculty member has taken an interest in my development Since entering this college, indicate how often you: #28 Received from your professor advice or guidance about your educational program Worked on a professor's research project CSS Academic self-efficacy (see Appendix C1) #34 Scientific self-efficacy [see Appendix C2] #32 Science Identity (see Appendix C3) #25, #31 Intent to pursue biomedical research career Probable career/occupation #15 Please indicate the highest degree you (A) will have earned as of June 2016 and (B) plan to complete eventually at any institution. #7 Bachelors Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. For the activities listed below, please indicate how often you engaged in each during the past year: Met with an advisor/ counselor about your career plans #11 Participation in academic and professional student organizations Since entering college, have you joined a pre-professional or departmental club? #16 Social integration and perceived fit with university setting Please indicate your agreement with each of the following statements. This institution has contributed to my: Ability to work as part of a team #19 Please indicate your agreement with the following statements: #24 I see myself as part of the campus community I feel a sense of belonging to this campus I feel that I am a member of this college I feel valued at this institution Please rate your satisfaction with this college in each area: Overall sense of community among students #21 Since entering this college, indicate how often you: Discussed course content with students outside of class #11 Engagement in research and satisfaction with faculty mentorship Please indicate your agreement with the following statements: #24 Faculty showed concern about my progress Faculty empower me to learn here Faculty encouraged me to meet with them outside of class At least one faculty member has taken an interest in my development Please rate your satisfaction with this college in each area: Ability to find a faculty or staff mentor #21 Since entering this college, indicate how often you: Communicated regularly with your professors #11 In the past year, how often gave you: Felt that faculty provided me with feedback that helped me assess my progress in class #23 How often have professors at your college provided you with: #27 Encouragement to pursue graduate/professional study An opportunity to work on a research project Advice and guidance about your educational program Emotional support and encouragement A letter of recommendation Help to improve your study skills Feedback on your academic work (outside of grades) An opportunity to discuss coursework outside of class Help in achieving your professional goals An opportunity to apply classroom learning to "real-life" issues How many months since entering college (including summer) did you work on a professor’s research project? (Response scale: 0, 1-3, 4-6, 7-12, 13-24, 25+) #12 B –SAFS Academic self-efficacy: [see Appendix C1] Scientific Sell-Efficacy [see Appendix C2] Science identity/ Research Identity (see Appendix C3) Participation in academic and professional student organizations In the past year, have you participated in a pre-professional or departmental club? Have you held office in a pre-professional or professional organization in the past year? Satisfaction with faculty mentorship How would you rate the overall quality of the mentoring you received? To what extent do you feel this mentor is meeting your expectations? Intent to pursue career in Biomedical Research Degree/Certificate Application Items [see Appendix C6] Will you pursue a science-related research career? definitely yes, possibly yes, uncertain, possibly no, definitely no Mentee AFS Academic self-efficacy [see Appendix C1] Scientific self-efficacy [see Appendix C2] Science identity/ Research Identity [see Appendix C3] Participation in academic and professional student organizations Have you held office in any pre-professional or professional organization in the past year? Intent to pursue career in Biomedical Research Degree/Certificate Application Items [see Appendix C6]
	Pursuit of Biomedical Science Undergraduate degree Pursuit of Biomedical Science Undergraduate degree	IND-B2 IND – B2 (Cont.)	TFS CSS B-SATS Mentee AFS	TFS Do you consider yourself pre-med #24 Please indicate your intended major. #2 What is the highest academic degree that you intend to obtain? #33 Bachelor’s Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. In deciding to go to college, how important to you was each of the following reasons? To prepare myself for graduate or professional school. #40 Below are some reasons that might have influenced your decision to attend this particular college. How important was each reason in your decision to come here? This college's graduates gain admission to top graduate/professional schools #43 CSS Please indicate the highest degree you (A) will have earned as of June 2016 and (B) plan to complete eventually at any institution. #7 Bachelor’s Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. Since entering college, how often have you: Met with an advisor/ counselor about your career plans . #11 Do you plan to do the following in fall 2016 #36 Attending graduate/professional school full-time Attending graduate/professional school part-time B –SAFS Degree/Certification Application Items [see Appendix C6] Mentee AFS Degree/Certification Application Items [see Appendix C6]
	Retention and Persistence in Biomedical Science Discipline	IND-B3	B-SAFS Mentee AFS	B- SAFS Current Position Items [see Appendix D] Degree/Certification Completion Items [see Appendix C6] Degree/Certification Application Items [see Appendix C6] Mentee AFS Current Position Items [see Appendix C4] Degree/Certification Completion Items [see Appendix C5] Degree/Certification Application Items [see Appendix C6]
	Participation in Undergraduate/Summer Biomedical Research Training in Labs or Similar research Environment Participation in Undergraduate/Summer Biomedical Research Training in Labs or Similar research Environment	IND-B4 IND-B4 (Cont.)	YFCY CSS B-SAFS Mentee AFS	YFCY Since entering this college, indicate how often you: Worked on a professor's research project #28 CSS Since entering this college, have you: Participated in: An undergraduate research program #16 How often have professors at your college provided you with: #27 An opportunity to work on a research project Emotional support and encouragement How many months since entering college (including summer) did you work on a professor's research project? Response scale: 0, 1-3, 4-6, 7-12, 13-24, 25+ #12 B –SAFS Please tell us which of the following activities you participated in since [indicate date of last contact?] <Site-specific list> Other tuition and stipend program Other summer program (i.e. Summer Research Experience) Academic Advising and Support (i.e. tutoring, STEM advising, math courses, etc.) Mentoring Research training (i.e. workshops, training, field experience, conferences, etc.) Other career advancement programs (i.e. networking, Professional Exposure, GRE Prep, field trips, career panels, applications) Have you had the opportunity to conduct your own scientific research or to participate in scientific research directed by others in the past year? Grants and Submissions Items [see Appendix C12] Research Not Covered by Grant [See Appendix C13] Mentee AFS In the past five years, have you been the Principal Investigator of an NIH-funded research grant? In the past five years, have you been the Principal Investigator of any other extramural (other government, foundation, corporate, or other) research grants? Grants and Submission Items [See Appendix C12] Research Not Covered by Grant [See Appendix C13]
	Poster or Paper Presentation at Scientific Conference	IND-B5	B-SAFS Mentee AFS	B-SAFS Have you presented at a scientific conference in the last year (including posters)? Mentee AFS Have you ever presented at a scientific conference in the last year (including posters)?
	Submitted Application & Receipt of Awards including Research Fellowship and Scholarships	IND-B6	B- SAFS Mentee AFS	B- SAFS Scholarships/Grant Items [see Appendix C7] Grants and Submissions Items [see Appendix C12] Have you received any honors or awards in the past year? Mentee AFS Scholarships/Grant Items [see Appendix C7] Grants and Submissions Items [see Appendix C12]
	Evidence of Biomedical Research Career Preparedness (grades GRE; std. exams)	IND-B7	TFS YFCY CSS	TFS What was your average grade in high school? #7 What were your scores on the SAT I and/or ACT? #8 How many Advanced Placement/International Baccalaureate courses or exams did you take in high school? Response categories: Not offered at my high school, None, 1-4, 5-9, 10-14, 15+ #17 AP Courses AP Exams IB Courses IB Exam YFCY What is your overall grade average (as of your most recently completed academic term)? #19 CSS What is the average grade you received during your college career , both overall and in your major? #22
	Authorship/ Co-Authorship of Peer-Reviewed Publication(s) Authorship/ Co-Authorship of Peer-Reviewed Publication(s)	IND-B8 IND-B8 (Cont.)	B-SAFS Mentee AFS	B-SAFS Peer Review Publication Items [see Appendix C14] Other Scientific Publication Items [see Appendix C15] Mentee AFS In the past five years, how many peer-reviewed journal articles have you published? Do you have any peer-reviewed publications accepted, in-press, or published in the past year? Peer Reviewed Publication Items [see Appendix C14] Other Scientific Publication Items [see Appendix C15]
	Completion of Undergraduate Degree in Biomedical Science Discipline	IND-B9	CSS B-SAFS Mentee AFS	CSS Please indicate the highest degree you (A) will have earned as of June 2016 and (B) plan to complete eventually at any institution. #7 Bachelor’s Degree (B.A., B.S., etc.) Master's degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. B- SAFS Current Position Items [see Appendix C4] Degree/Certification Completion [see Appendix C5] Mentee AFS Current Position Items [see Appendix C4] Degree/Certification Completion [see Appendix C5]
	Application & Acceptance to Attend Graduate Program in Biomedical Science Discipline	IND-B10	CSS B-SAFS Mentee AFS	CSS Do you plan to do the following in fall 2015? #36 Attending graduate/professional school full-time Attending graduate/professional school part-time B-SAFS Degree/Certificate Application Items [see Appendix C6] Mentee AFS Degree/Certificate Application Items [see Appendix C7]
	Entrance to Graduate Program in Biomedical Science Discipline Entrance to Graduate Program in Biomedical Science Discipline	IND-B11 IND-B11 (Cont.)	B-SAFS Mentee AFS	B-SAFS Current Position Items [see Appendix C4] Degree/Certificate Completion Items [see Appendix C5] Degree/Completion Application Items [see Appendix C6] Mentee AFS Current Position Items [see Appendix C4] Degree/Certificate Completion Items [see Appendix C5] Degree/Completion Application Items [see Appendix C6]
	Academic and scientific self –efficacy	PSY-16	TFS YFCY CSS B-SAFS Mentee AFS	TFS Academic self-efficacy (see Appendix C1) #41 Scientific self-efficacy [see Appendix C2] (will be added to 2016 survey) YFCY Academic self-efficacy (see Appendix C1) #14 Scientific self-efficacy [see Appendix C2] #37 CSS Academic self-efficacy (see Appendix C1) #34 Scientific self-efficacy [see Appendix C2] #32 BUILD-SAFS Academic self-efficacy (see Appendix C1) Scientific self-efficacy [see Appendix C2] Mentee AFS Scientific self-efficacy [see Appendix C2]
	Science Identity/ Research Identity Science Identity/ Research Identity	PSY-11 PSY-11 (Cont.)	TFS YFCY CSS B- SAFS Mentee AFS	HERI TFS Science Identity (see Appendix C3) #50 (Estrada Science Identity Items will be added to 2016 survey) YFCY Science Identity (see Appendix C3) #24, #36 CSS Science Identity (see Appendix C3) #25, #31 B-SAFS Science Identity (see Appendix C3) Mentee AFS Science Identity (see Appendix C3)
	Participation in academic and professional student organizations	PSY-SN1	TFS YFCY CSS B–SAFS Mentee AFS	YFCY Since entering this college have you joined a pre-professional or departmental club? #26 CSS Since entering college, have you joined a pre-professional or departmental club? #16 B-SAFS In the past year, have you participated in a pre-professional or departmental club? Have you held office in any pre-professional or professional organization in the past year? Mentee AFS Have you held office in any pre-professional or professional organization in the past year?
	Satisfaction with faculty mentorship Satisfaction with faculty mentorship Satisfaction with faculty mentorship	PSY-SN4 PSY-SN4 (Cont.) PSY-SN4 (Cont.)	YFCY CSS B- SAFS	YFCY Since entering this college, how often have you: Asked a professor/teacher for advice after class #17 Since entering this college, how often have you interacted with the following people (eg, by phone, e-mail, text, or in person) #9 Faculty during office hours Faculty outside of class or office hours Academic advisors/counselors Graduate students/ teaching assistants Since entering this college, how often have you felt: #12 That faculty provided me with feedback that helped me assess my progress in class That faculty encouraged me to ask questions and participate in discussions Please indicate the extent to which you agree or disagree with the following: #18 Faculty showed concern about my progress Faculty empower me to learn here Faculty encourage me to meet with them outside of class At least one faculty member has taken an interest in my development Since entering this college, indicate how often you: #28 Received from your professor advice or guidance about your educational program Worked on a professor's research project CSS Please indicate the extent to which you agree or disagree with the following: #24 Faculty showed concern about my progress Faculty empower me to learn here Faculty encourage me to meet with them outside of class At least one faculty member has taken an interest in my development Please rate your satisfaction with this college in each area: Ability to find a faculty or staff mentor #21 Since entering this college, indicate how often you: Communicated regularly with your professors #11 For the activities listed below, please indicate how often you engage in each during the past year #10 Felt that faculty provided me with feedback How often have professors at your college provided you with: #27 Encouragement to pursue graduate/professional study An opportunity to work on a research project Advice and guidance about your educational program Emotional support and encouragement A letter of recommendation Help to improve your study skills Feedback on your academic work (outside of grades) An opportunity to discuss coursework outside of class Help in achieving your professional goals An opportunity to apply classroom learning to "real-life" issues How many months since entering college (including summer) did you work on a professor’s research project? (Response scale: 0, 1-3, 4-6, 7-12, 13-24, 25+) #12 BUILD-SAFS How would you rate the overall quality of the mentoring you received? To what extent do you feel this mentor is meeting your expectations?
	Social integration/ Perceived fit with university setting Social integration/ Perceived fit with university setting	PSY-SN6 PSY-SN6	YFCY CSS	YFCY In deciding to go to college, how important to you was each of the following reasons? Popularity Please indicate your agreement with each of the following statements. This institution has contributed to my: Ability to work as part of a team #21 Since entering this college, how often have you felt: #12 Isolated from campus life Unsafe on this campus Since entering this college, how has it been to: Develop close friendships with other students #15 Please indicate the extent to which you agree or disagree with the following statements: #18 My college experiences have exposed me to diverse opinions, cultures and values I see myself as part of the campus community I feel a sense of belonging to this campus I feel that I am a member of this college I feel valued at this institution Please rate your satisfaction with this college in each area: Overall sense of community among students #20 Since entering this college, indicate how often you: Discussed course content with students outside of class #28 CSS Please indicate your agreement with each of the following statements. This institution has contributed to my: Ability to work as part of a team #19 Please indicate the extent to which you agree or disagree with the following statements: #24 I see myself as part of the campus community I feel a sense of belonging to this campus I feel that I am a member of this college I feel valued at this institution Please rate your satisfaction with this college in each area: Overall sense of community among students #21 Since entering this college, indicate how often you: Discussed course content with students outside of class #11
	Intent to pursue career in Biomedical Research Intent to pursue career in Biomedical Research	PSY-I4 PSY-I4 (Cont.)	TFS CSS B-SAFS Mentee AFS	HERI TFS Do you consider yourself: pre-med #24 Please indicate your intended major. #25 What is the highest academic degree that you intend to obtain? #33 Bachelors Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. In deciding to go to college, how important to you was each of the following reasons?: To prepare myself for graduate or professional school. #40 Below are some reasons that might have influenced your decision to attend this particular college. How important was each reason in your decision to come here? This college's graduates gain admission to top graduate/professional schools #43 CSS Please indicate the highest degree you (A) will have earned as of June 2016 and (B) plan to complete eventually at any institution. #7 Bachelors Degree (B.A., B.S., etc.) Masters Degree (M.A., M.S., etc.) Ph.D. or Ed.D. M.D., D.O., D.D.S., or D.V.M. For the activities listed below, please indicate how often you engaged in each during the past year: Met with an advisor/ counselor about your career plans #11 B-SAFS Degree/Completion Application Items [see Appendix C5] Will you pursue a science-related research career? Definitely yes, Possibly yes, Uncertain, Possibly no, Definitely no Mentee AFS Degree/Completion Application Items [see Appendix C5]
Domain	Elements	Hallmarks	Data Source / Measure	Operational Definition
Faculty Faculty Faculty Faculty Faculty Faculty	Increase in the number of trainees mentored in Programs Relevant to BUILD Increase in the number of trainees mentored in Programs Relevant to BUILD	FAC-B15 FAC-B15 (Cont.)	B-FAFS Mentor AFS	B-FAFS [FIRST TIME ONLY] Have you regularly mentored students, junior faculty, junior researchers or peers on biomedical career or research issues? About how many people have you mentored in the past five years? Do you have anyone you currently mentor who is junior to you (students, junior faculty, junior researchers)? If yes, how many different people do you mentor currently? Undergraduate students Graduate students Post-docs Junior faculty or junior researchers Others Do you have anyone you currently mentor who is at a similar career level to yours (a peer)? If yes, how many different peers do you mentor currently? Do you have a faculty member of someone else who is more senior you currently consider a mentor? If yes, how many different mentors do you regularly work with? Do you have someone at a similar educational level as you (i.e. a peer) who you currently consider a mentor? If yes, how many different peer mentors do you have? Mentor AFS [FIRST TIME ONLY] Have you every regularly mentored students, junior faculty, junior researchers, or peers on biomedical career or research issues? a. About how many people have you mentored in the past five years? Do you have anyone you currently mentor who is junior to you (students, junior faculty, junior researchers)? If yes, how many different people do you regularly mentor currently? a. Students b. Post-docs c. Junior faculty or junior researchers d. Others Do you have anyone you currently mentor who is at a similar career level to yours (a peer)? If yes, how many different peers do you mentor currently? Do you have a faculty member of someone else who is more senior you currently consider a mentor? If yes, how many different mentors do you regularly work with? Do you have someone at a similar educational level as you (i.e. a peer) who you currently consider a mentor? If yes, ho w many different peer mentors do you have?
	Increase in Participation in Professional Development Activities for faculty in Programs Relevant to BUILD Increase in Participation in Professional Development Activities for faculty in Programs Relevant to BUILD Increase in Participation in Professional Development Activities for faculty in Programs Relevant to BUILD	FAC-B4 FAC-B4 (Cont.) FAC-B4 (Cont.)	B-FAFS Mentor AFS	B-FAFS During the past year, have you taken advantage of any of the following professional development opportunities at this institution? (CHECK ALL THAT APPLY) -Paid workshops focused on teaching -Paid workshops focused on conduction research -Paid sabbatical leave -Travel funds paid by the institution -Travel funds paid by a grant (e.g. BUILD) -Incentives to conduct research -Internal grants for research -Training for administrative leadership -Incentives to develop new courses -Incentives to integrate new technology into your classroom -Incentives to integrate culturally competent practices into your classroom During the past year, have you engaged in any of the following activities? (Check all that apply) - Participated in the development of science curriculum (enhancing an existing science course or creating a new science course -Taught a newly developed science course -Participated in the development of supplementary cultural curriculum for students or faculty in the sciences (e.g. stereotype threat, cultural assets, cultural competency, etc.) -Taught a newly developed supplemental cultural course for students or faculty in the sciences -Been evaluated for tenure or promotion -Participated in self-evaluation for tenure or promotion Mentor AFS [FIRST TIME ONLY] In the past year, have you participated in any training not through NRMN to improve your skills as a mentor? [FOR RETURNING RESPONDENTS] Since we last contacted you on <date>, have you participated in any training not through NRMN to improve your skills as a mentor? a. Type of training (select all that apply) - In-person - On-line that occurs in scheduled, real-time method (synchronous) - On-line or other virtual that you can take at any time that provides feedback or interaction with others (asynchronous) - Self-study (by any means, but not interaction or feedback) - Other (specify) b. Approximately how many hours was your training over the past year? [FIRST-TIME ONLY] In the past year, have you participated in any training not through NRMN to improve your skills as a mentee? [FOR RETURNING RESPONDENTS] Since we last contact you on <date>, have you participated in any training not through NRMN to improve your skills as a mentee? c. Type of training (select all that apply) - In-person - On-line that occurs in scheduled, real-time method (synchronous) - On-line or other virtual that you can take at any time that provides feedback or interaction with others (asynchronous) - Self-study (by any means, but not interaction or feedback) - Other (specify) d. Approximately how many hours was your training over the past year? Have you competed formal training/workshops related to your career or research (minimum of at least 4 hours cumulative duration) in addition to those offered by NRMN? If yes, please indicate the focus of the training: On NIH funding On other funding On biomedical career development (e.g., training in leadership, teaching, mentoring, publishing, gaining tenure, etc.) On other biomedical research related topics?
	Increase in Faculty Participation in Mentorship Activities in Programs Relevant to BUILD (may include structured activities to train the next gen of biomedical scientists) Increase in Faculty Participation in Mentorship Activities in Programs Relevant to BUILD (may include structured activities to train the next gen of biomedical scientists)	FAC-B12 FAC-B12 (Cont.)	FAC BUILD-FAFS	FAC During the past two years, have you engaged in any of the following activities? Engaged undergraduates on your research project Participated in organized activities around enhancing pedagogy and student learning During the past two years, to what extent have you: Presented with undergraduate students at conferences Published with undergraduates During the present term, how many hours per week on average do you actually spend on each of the following activities? Advising and counseling of students Please indicate the extent to which you: Mentor undergraduate students B–FAFS During the past year, have you engaged in any of the following activities? (CHECK ALL THAT APPLY) -Participated in the development of science curriculum (enhancing an existing science course or creating a new science course) -Taught a newly developed science course -Participated in the development of supplementary cultural curriculum for students or faculty in the sciences (e.g. stereotype threat, cultural assets, cultural competency, etc.) -Taught a newly developed supplemental cultural course for students or faculty in the sciences -Been evaluated for tenure or promotion -Participated in self-evaluation for tenure or promotion In the past year, to what extent have you: [3 item scale: 1= To A Great Extent, 2=To Some Extent, 3= Not at All; N/A] a. Engaged in a research project directed by others b. Worked with undergraduates on a research project c. Worked with graduate student(s) on a research project d. Engaged in your own research project e. Engaged undergraduates on your own research project f. Engaged graduate students on your research project g. Engaged in public discourse about your research or field of study (e.g. blog, media, interviews, op-eds, etc.)
	Increased Research Productivity in Publications, Grant Submissions and Awards as PI, multi-PI and/or collaborator for faculty in Programs Relevant to BUILD Increased Research Productivity in Publications, Grant Submissions and Awards as PI, multi-PI and/or collaborator for faculty in Programs Relevant to BUILD	FAC-B8 FAC-B8 (Cont.)	B- FAFS Mentor AFS	B-FAFS [FIRST TIME ONLY] In the past five years, have you been the Principal Investigator of an NIH-funded research grant? [FIRST TIME ONLY] In the past five years, have you been the Principal Investigator of any other extramural (other government, foundation, corporate, or other) research grants? [FIRST TIME ONLY] Has your research ever been the subject of technology transfer activities (including issued patents, completed licenses, and drug, device or diagnostic approvals)? 46. In the past year, to what extent have you: a. Engaged in a research project directed by others b. Worked with undergraduates on a research project c. Worked with graduate student(s) on a research project d. Engaged in your own research project e. Engaged undergraduates on your own research project f. Engaged graduate students on your research project g. Engaged in public discourse about your research or field of study (e.g. blog, media, interviews, op-eds, etc.) Grants and Submissions Items [see Appendix C12] Research Not Covered by Grants [see Appendix C13] In the past year, have you been formally recognized, honored, or awarded in recognition of your teaching, research, or other professional activity? Mentor AFS [FIRST TIME ONLY] In the past five years, have you been the Principal Investigator of an NIH-funded research grant? [FIRST TIME ONLY] In the past five years, have you been the Principal Investigator of any other extramural (other government, foundation, corporate, or other) research grant? To allow us to follow your professional accomplishments, please provide any of the following identifiers that you may have: a. ORCID - b. NIH eRA Commons ID - c. LinkedIn ID Grants and Submissions Items [see Appendix C12] Research Not Covered by Grants [see Appendix C13] In the past year, have you been formally recognized, honored, or awarded in recognition of your teaching, research, or other professional activity?
	Increased quality of mentoring (student and mentor perceptions) Increased quality of mentoring (student and mentor perceptions)	FAC-B16 FAC-B16 (Cont.)	B –SAFS B-FAFS	B-SAFS Mentor Skill Assessment Items [see Appendix C8] Mentor Assessment Items [see Appendix C9] In your mentoring relationship, how important is it to you that your primary research mentor does the following:[3-item scale, 1=not important, 2=somewhat important, 3=very important] a. Is willing to discuss diversity b. Considers their own cultural background, as well as yours c. values and respects cultural differences In your mentoring relationship, how skilled is your primary research mentor in the following: :[3-item scale, 1=not important, 2=somewhat important, 3=very important] a. Discussing diversity b. Considers their own cultural background, as well as yours c. Values and respects cultural differences How would you rate the overall quality of the mentoring you received? [7-item scale, 1=very low, 4=average, 7=very high] To what extent do you feel this mentor is meeting your expectations? [7-item scale, 1=very low, 4=average, 7=very high] B- FAFS Mentor Skill Self-Assessment Items [see Appendix C10] In your mentoring relationship, how important is it to you that YOU do the following: a. Discussing diversity issues b. Considering my own cultural background as well as my mentee’s cultural background c. Valuing and respecting cultural differences Mentoring Self-Assessment Items [see Appendix C11]
Domain	Elements	Hallmarks	Data Source / Measure	Operational Definition
Institution Institution Institution	Increased, enhance, and/or develop Inter-Institutional Collaborations to achieve BUILD outcomes related to research, mentorship, and faculty development (e.g., linkages with Community Colleges or other partner institutions, collaborations and postdocs at Research-Intensive partner institutions, engagement with NRMN)	INST-B14	BUILD Case Study	Case Study Guiding Evaluation Questions Overarching Question: How are BUILD programs building capacity and infrastructure for Primary and Partner Institutions to advance URG bio-medical research training?  Sub-Questions: How are the strategies that are being implemented to enhance faculty development and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance student participation and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance participation in, and improve the quality of mentoring activities advancing research training for URGs? How do partner institutions enhance the primary institutions’ capacity to provide bio-medical research training?
	Improved Undergraduate Retention Rates of Students in Programs Relevant to BUILD (biomedical sciences)	INST-B1	BUILD Case Study	Case Study Guiding Evaluation Questions Sub-Question: b) How are the strategies that are being implemented to enhance student participation and engagement advancing bio-medical research training for URGs?
	Increased Participation in Mentoring Activities (Students and Faculty) in Programs Relevant to BUILD	INST-B2	BUILD Case Study	Case Study Guiding Evaluation Questions Sub-Question: How are the strategies that are being implemented to enhance participation in, and improve the quality of mentoring activities advancing research training for URGs?
	Increase in Number of Student Research Training Opportunities for students and faculty in Programs Relevant to BUILD	INST-B12	BUILD Case Study	Case Study Guiding Evaluation Questions Sub-Question: b) How are the strategies that are being implemented to enhance student participation and engagement advancing bio-medical research training for URGs?
	Increase in Number of underrepresented Students Enrolled in BUILD Biomedical Research-Related Programs	INST-B13	IR Record request
	Increase in Number of underrepresented Students Retained in BUILD Biomedical Research-Related Programs	INST-B3	IR Record request
	Institutional commitment to BUILD sustainability evidenced by site maintenance of key elements of program interventions after grant period Institutional commitment to BUILD sustainability evidenced by site maintenance of key elements of program interventions after grant period	INST-B17 INST-B17 (Cont.)	B-FAFS BUILD Case Study	B-FAFS Institutional Priority: Commitment to Diversity Indicate how important you believe each priority listed below is at your college or university: [4 item scale, 1=Low priority, 2=Medium priority, 3= High priority, 4=Highest priority] To recruit more minority students To create a diverse multi-cultural campus environment To increase the representation of minorities in the faculty and administration To increase the representation of women in faculty and administration To develop an appreciation for multiculturalism Case Study Guiding Evaluation Questions Overarching Question: How are BUILD programs building capacity and infrastructure for Primary and Partner Institutions to advance URG bio-medical research training?  Sub-Questions: How are the strategies that are being implemented to enhance faculty development and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance student participation and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance participation in, and improve the quality of mentoring activities advancing research training for URGs? How do partner institutions enhance the primary institutions’ capacity to provide bio-medical research training?
	Increased institutional commitment to sustaining activities of BUILD (i.e. research infrastructure, FTE, scholarships, space), changing the academic culture, culture of faculty promotion, tenure, research development (release time), stronger emphasis on student mentoring and advising to increase institutional outcomes, curriculum improvements	INST-B18	B-FAFS BUILD Case Study	B-FAFS Campus Assessment Items [see Appendix C18] Case Study Guiding Evaluation Questions Overarching Question: How are BUILD programs building capacity and infrastructure for Primary and Partner Institutions to advance URG bio-medical research training?  Sub-Questions: How are the strategies that are being implemented to enhance faculty development and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance student participation and engagement advancing bio-medical research training for URGs? How are the strategies that are being implemented to enhance participation in, and improve the quality of mentoring activities advancing research training for URGs? How do partner institutions enhance the primary institutions’ capacity to provide bio-medical research training?
	Increase enrollment and participation of underrepresented students in biomedical research fields	INST-B19	IR Record request BUILD activity/program rosters
	Increase in participation of faculty in mentorship activities – defined : Increase in number of faculty seeking and participating in mentor training	INST-B20	IR Record request BUILD activity/program rosters

Appendix C1

Academic Self-Efficacy (source: HERI)

Rate yourself on each of the following traits as compared with the average person your age. We want the most accurate estimate of how you see yourself. (Mark one in each row)

[5 item-scale: 1=Highest 10%, 2=Above Average, 3=Average, 4=Below Average, 5=Lowest 10%]

1. Academic ability

2. Drive to achieve

3. Mathematical ability

4. Self-confidence (intellectual)

Appendix C2

Scientific Self-Efficacy (source: Estrada)

Indicate to what extent you are confident that you can complete the following tasks:

[5 item-scale: 1-not at all confident, 2= a little confident,. 3=somewhat confident, 4=very confident, 5-extremely confident]

1. Use technical science skills (use of tools, instruments, and/or techniques)

2. Generate a research question to answer

3. Figure out what data/observations to collect and how to collect them

4. Create explanations for the result of the study

5. Use scientific literature and/or reports to guide research

6. Develop theories (integrate and coordinate results from multiple studies

Appendix C3

Science Identity

Estrada Items:

To what extent are the following statements true of you:

1. I have a strong sense of belonging to the community of scientists

2. I derive great personal satisfaction from working on a team that is doing important research

3. I have come to think of myself as a ‘scientist’

4. I feel like I belong in the field of science

HERI Items:

Indicate the importance to your personally of each of the following:

[4 item-scale: 1=not important, 2=somewhat important, 3=important, 4=very important]

1. Obtaining recognition from colleagues for contributions to my special field

2. Making a theoretical contribution to science

3. Becoming an authority in my field

Appendix C4

Current Position Items

[SCHOOL]

Last year, you told us you were ________[level] at _________ [institution]. Has that changed?

-No (skip to Employment item below)

-Yes and I am still in school (go to a-d)

-Yes but I am not in school (go to e)

a. Level

- Undergraduate

- Graduate or other post-baccalaureate

- Expected completion date (year): ______

b. Status

-Full Time

-Part Time

c. Institution: _________________________

d. Major: _________________________

e. Please tell us your current status

- I graduated from my previous institution

- I did not graduate but do have plans to attend school in the next 2 years

- I did not graduate and do NOT have plans to attend school in the next 2 years

[EMPLOYMENT]

Now we would like to ask you about any employment or internships. Last year, you told us you were _____ [at ________]. Has that changed?

If yes, go to a-f:

1. Please review the list below and check all that apply

-Working (including internships and significant volunteer positions)

-Retired

-Unemployed, looking for work

-Otherwise not in the labor force

-Other (specify): _________________

2. Paid Employment (check all that apply):

-Full Time

-Part Time

-Internships or student placement

-No paid employment

3. Is this paid position related to research?

-No

-Yes

4. Is this new job/position considered a promotion or advancement?

-No

-Yes

5. Volunteer

-Position

-Company

-No volunteer position

6. Is this volunteer position related to research

-No

-Yes

Appendix C5

Degree/Certificate Completion Items

During the past year, did you complete any degree or certificate program? No/Yes

If yes, please indicate the following:

a. Degree/certificate

b. Major/area of study

c. Awarding institution

d. Date awarded: MM/YR

Appendix C6

Degree/Certificate Application Items

During the past year, did you apply to any degree or certificate program? No/Yes

If yes, please indicate the following:

a. Degree/certificate the program awards:

b. Major area of study:

c. Institution:

d. Date applied: MM/YR

e. Status of Application

-Accepted and will attend

-Accepted and will not attend

-Waitlisted

-Pending

-Not accepted

Appendix C7

Scholarship/Grant Items

During the past year, did you receive any scholarships or grants for education expenses that you do not need to repay? No/Yes

If yes, please indicate the following:

a. Name of scholarship/grant: ____________________________

b. Amount (total value including value of an fee/tuition waivers):

-Less than $1,000

-$1,000 - $4,999

-$5,000 -$ 9,999

-$10,000 or more

c. Period of award: MM/YYYY to MM/YYYY

d. Was this award based on:

-Need

-Merit

-Need and Merit

-Other (specify)

Appendix C8

Mentor Skill Assessment

Please rate how skilled you feel your mentor was in the following areas: [7 item scale, 1=not at all and 7=extremely]

a. Active listening

b. Providing you constructive feedback

c. Establishing a relationship based on trust with you

d. Identifying and accommodating different communication styles

e. Employing strategies to improve communication with you

f. Coordinating effectively with other mentors with whom you work with

g. Working with you to set clear expectation of the mentoring relationship

h. Aligning his/her expectations with your own

i. Considering how personal and professional differences may impact expectations

j. Working with you to set research goals

k. Helping you develop strategies to meet goals

l. Accurately estimating your level of scientific knowledge

m. Accurately estimating your ability to conduct research

n. Employing strategies to enhance your understanding of that research

o. Motivating you

p. Building your confidence

q. Stimulating your creativity

r. Acknowledging your professional contributions

s. Negotiating a path to professional independence with you

t.Taking into account the biases and prejudices s/he brings to your mentor/mentee relationship

u. Working effectively with mentees whose personal background is different from his/her own (age, race, gender, class, region, culture, religion, family composition, etc.)

v. Helping you network effectively

w. Helping you set career goals

x. Helping you balance work with your personal life

y. Understanding his/her impact as a role model for you

z. Helping you acquire resources (e.g. grants, etc.)

Appendix C9

Mentor Assessment

Please respond to the following statements about your primary mentor that you have worked with in the past year: [4-item scale, 1-my mentor did not do this, 2=my mentor tried to do this, 3=my mentor did this sometimes and was effective, 4=my mentor did this frequently and was effective]

a. My mentor gave me an overview of how my research fit into an overall research project

b. My mentor helped me develop my research skills

c. My mentor showed interest in my research project

d. My mentor was available to me when I had problems or questions about my research

e. My mentor offered constructive feedback when necessary

f. My mentor and I developed a relationship based on trust

g. My mentor understood how I learn best

h. My mentor created an environment that allowed me to achieve my goals

i. My mentor seemed so busy that I was afraid to interrupt her/him

j. My mentor had an effective mentoring style

k. My mentor acted as a positive role model

l. My mentor showed interest in me as a person

m. My mentor fostered my independence

n. My mentor fostered confidence in my skills

o. My mentor appreciated my contributions

p. My mentor encouraged me to be creative

q. My mentor made me enthusiastic about my project

r. My mentor helped me feel curious about my project

s. My mentor treated me as a colleague

t. My mentor helped me decide on a career path

u. My mentor communicated his/her expectations of me

v. My mentor respected my goals

w. My mentor allowed me to take ownership in my research

x. My mentor created an environment where I felt safe to make mistakes

y. My mentor made me feel included in the lab

z. My mentor regularly assessed skills and knowledge that I gained in the lab

Appendix C10

Mentor Skill Self-Assessment

Please rate how skilled you feel you are in each of the following areas [7 item scale, 1=not at all and 7=extremely]

a. Active listening

b. Providing constructive feedback

c. Establishing a relationship based on trust

d. Identifying and accommodating different communication styles

e. Employing strategies to improve communication with mentees

f. Coordinating effectively with your mentees’ other mentor

g. Working with mentees to set clear expectations of the mentoring relationship

h. Aligning you expectations with your mentees’

i. Considering how personal and professional differences may impact expectations

j. Working with mentees to set research goals

k. Helping mentees develop strategies to meet goals

l. Accurately estimating your mentees’ level of scientific knowledge

m. Accurately estimating your mentees’ ability to conduct research

n. Employing strategies to enhance your mentees’ knowledge and abilities

o. Motivating your mentees

p. Building mentees’ confidence

q. Stimulating your mentees’ creativity

r. Acknowledging your mentees’ professional contributions

s. Negotiating a path to professional independence with your mentees

t. Taking into account the biases and prejudices you bring to the mentor/mentee relationship

u. Working effectively with mentees whose personal background is different from your own (age, race, gender, class, region, culture, religion, family composition, etc.)

v. Helping your mentees network effectively

w. Helping your mentees set career goals

x. Helping your mentees balance work with their personal life

y. Understanding your impact as a role model

z. Helping your mentees acquire resources (e.g. grants, etc.)

Appendix C11

Mentoring Self-Assessment

Please respond to the following statements regarding your primary mentee, you have worked with in the past year. [4 item scale, 1=This is not one of my mentoring objectives, 2= I have considered how to include this in my mentoring, 3= I have tried to do this in my mentoring, 4= I have evidence I have done this effectively in my mentoring]

a. My mentor gave me an overview of how my research fit into an overall research project.

b. My mentor helped me develop my research skills.

c. My mentor showed interest in my research project.

d. My mentor was available to me when I had problems or questions about my research.

e. My mentor offered constructive feedback when necessary.

g. My mentor and I developed a relationship based on trust.

h. My mentor understood how I learn best.

i. My mentor created an environment that allowed me to achieve my goals.

j. My mentor and I discussed diversity issues.

k. My mentor seemed so busy that I was afraid to interrupt her/him.

l. My mentor had an effective mentoring style.

m. My mentor acted as a positive role model.

n. My mentor showed interest in me as a person.

o. My mentor expressed consideration for her/his cultural background as well as my own.

p. My mentor fostered my independence.

q. My mentor fostered confidence in my skills.

r. My mentor appreciated my contributions.

s. My mentor encouraged me to be creative.

t. My mentor made me enthusiastic about my project.

u. My mentor helped me feel curious about my project.

v. My mentor treated me as a colleague.

w. My mentor helped me decide on a career path.

x. My mentor communicated his/her expectations of me.

y. My mentor respected my goals.

z. My mentor allowed me to take ownership in my research.

aa. My mentor created an environment where I feel safe to make mistakes.

bb. My mentor made me feel included in the lab.

cc. My mentor valued and respected cultural differences.

dd. My mentor regularly assessed skills and knowledge that I gained in the lab.

Appendix C12

Grants and Submission

[For undergrad and grad students FIRST TIME ADMINISTRATION] Since you started college, have you applied for or received funding to support your own research? Please do not include fellowships or scholarships that primarily pay for tuition, fees, or living expenses. Also, please do not include service projects unless they include a research component. No/Yes

[ONLY RETRUNING RESPONDENTS] Below is a list of all research funding support you have provided to us in the past. Have you applied for any additional funding to support your own research that is not on the list (either as the lead investigator, with a faculty advisor as the lead, or for a other paid position) OR has the status of previous submissions changed? No/Yes

[If yes]

Please complete or update information for each grant or proposal:

1. Funding Agency Type

-Your university

-Federal agency (e.g. NIH, NSF)

-Nonfederal government

-Nonprofit (e.g. foundations)

-For-profit

-Other (specify)

2. [If NIH or Other Federal] Full grant number if available

3. [If Nonfederal, Nonprofit, For-Profit, Other]

Agency/Foundation/Company Name

4. Role

-Principal Investigator

-Co-PI

-Investigator

-Other (specify)

5. Submission status:

-Submitted (first time for this proposal) (skip to j)

-Re-submitted with revisions (skip to j)

-Not funded (skip to j)

NIH Impact score (if applicable)

-Funded (skip to j)

NIH Impact score (if applicable)

6. Project Title

7. Amount (total across all years):

-Less than $50,000

-$50,000 - $99,999

-$100,000 - $249,999

-$250,000 - $499,999

-$500,000 or more

8. Start Date (e.g. 07/08/2015)

9. End Date (e.g. 07/08/2015)

10. What was the main purpose of the funding?

-Research

-Training of other (e.g. students, peers, workforce)

-Your career development

-Other (specify)

Appendix C13

Research Not Covered by Grants Items

Have you conducted research that is not covered by the grants listed above? No/Yes

If yes, please answer the following questions:

a. What type of research was this?

-Literature review, synthesis of existing knowledge, and/or conceptual

-Analysis of existing (secondary) data

-Analysis of data you collect (primary analysis)

b. What was the research for?

-A class or degree requirement

-Part of a paid job (e.g. research assistant), internship, or training program

-Related to your role as an independent researcher

c. What was your role?

-PI/Co-PI or Project Lead

-Investigator

-Research Assistant

-Other (Specify): _______________________________

Appendix C14

Peer-Reviewed Publications

[FIRST TIME ONLY] Do you have any peer-reviewed publications accepted, in-press, or published in the past year? No/Yes

[RETURNING RESPONDENTS] Below is a list of publications you have provided to us in the past. Do you have any peer-reviewed publications that you have had accepted to add to this list? No/Yes

If yes, please answer the following questions:

1. PMID#: ___________________________ (if provided, skip to question #29)

2. List of all Authors (Last FM, comma separated): _____, ______, ______, ______

3. Title: ________________________________

4. Journal Name: __________________________________

5. Year Published (or indicate “in press” or epub ahead”): ________

6. Volume: ________

7. Issue: _______

8. Page Numbers: _______

9. DOI or URL or epubs: _______________________

Appendix C15

Other Scientific Publications

Are there other scientific publications (e.g. book chapters, books, reports, non-peer reviewed journal articles, working papers, other) to add to this list? No/Yes

a. Publication Type:

-Chapter

-Book

-Other, specify: _____________

b. List all Authors (Last FM, comma separated): _____, ______, ______, ______,

Complete the applicable information below depending on publication type:

c. [If Book or Chapter] Book/ Anthology Title: _____________________________

d. [If Chapter] Chapter Title: __________________________________

e. [If Chapter] Editors: ________________________________

f. Year Published: __________

g. [If Book or Chapter] Edition: _________

h. [If Book or Chapter] Publisher: _________

i. City: ________________________

j. State: ____________________________

k. [If Book or Chapter] Page Numbers: _______

l. DOI or URL for epubs: ______________________

Appendix C16

Teaching Self-Efficacy Assessment Items

17. How confident are you that you can do the following: [5 item scale, 1=Extremely Confident, 2=Very Confident, 3=Somewhat Confident, 4=Slightly Confident, 5=Not at all Confident]

a. Setting learning goals

b. Selecting reading materials

c. Designing assignments

d. Planning class activities

e. Using various teaching strategies

f. Engaging students in learning

g. Providing students opportunities to practice skills

h. Promoting student collaboration

i. Encouraging students to ask questions

j. Encouraging students to express ideas

k. Encouraging participation from women and minorities

l. Accurately assessing students’ knowledge

m. Grading assignments using criteria

n. Providing students constructive suggestions

o. Providing students prompt feedback

p. Fostering students’ independent thinking

q. Addressing sensitive issues in ways that help students to deal with them maturely

r. Fostering students’ confidence in ability to learn

s. Providing students an overview of discipline

t. Demonstrating passion for teaching

u. Staying current in subject knowledge

v. Helping students understand the relevance of learning

w. Enriching teaching with research

Appendix C17

Faculty Research and Grant Writing Self-Assessment Items

We would like to know how confident you are today that you can successfully perform the tasks listed below. Using a 0-10 scale, indicate your level between No Confidence (0) and Total Confidence (10) in your current abilities in these general areas of research and grant writing. Use NA when a task statement does not seem appropriate for your training.

a. Select a suitable topic area for study.

b. Refine a problem so that it can be investigated.

c. Develop a logical rationale for a particular research idea.

d. Organize your proposed research ideas in writing.

e. Articulate a clear purpose for the research.

f. Place your study in the context of existing research and justify how it contributes to important questions in the area.

g. Relate specific questions of interest to underlying theory.

h. Convince grant reviewers your proposed study is worth funding.

i. Choose an appropriate research design that will answer a set of research questions and/or test a set of hypothesis.

j. State the purpose, strengths, and limitations of each study design.

k. Determine the universe, population, and appropriate sample for a given study.

l. Determine an adequate number of subjects for your research project.

m. elect methods of data collection appropriate to the study population and variable(s) of interest.

n. Determine how each variable will be measured

o. Design the best data analysis strategy for your study.

p. Identify appropriate funding sources (local, state, national) to support a study.

q. Speak with a person at the funding agency regarding your project or project ideas.

r. Describe a major funding agency’s (e.g. NIH, foundation) proposal review and award process.

s. Write a competitive grant application.

t. Write up research findings for publication in a peer-reviewed journal.

u. Conduct the appropriate statistical analyses to answer your research questions.

v. Summarize research findings in a traditional research report.

w. Summarize and highlight research findings for publication in a peer-reviewed journal.

x. Communicate key research findings to a wide audience of stakeholders.

y. Submit paper and/or poster presentations to conferences related to your topic area.

z. Present research findings at conferences related to your topic area.

Appendix C18

Campus Assessment Items

Below are some statements about your college or university. Indicate the extent to which you agree or disagree with each of the following: [4 item scale, 1=Disagree Strongly, 2=Disagree Somewhat, 3=Agree Somewhat, 4=Agree Strongly]

1. Faculty are interested in students’ personal problems.

2. Racial and ethnic diversity is reflected in the curriculum.

3. Most students are well-prepared academically.

4. This institution has effective hiring practices and policies that increase faculty diversity.

5. This institution takes diversity of faculty into consideration as part of the promotion process.

6. Student Affairs staff have the support and respect of faculty.

7. Faculty are committed to the welfare of this institution.

8. Faculty are strongly interested in the academic problems of undergraduates

9. There is a lot of campus racial conflict here.

10. My research is valued by faculty in the department.

11. My teaching is valued by faculty in the department.

12. My service is values by faculty in the department.

13. This institution takes into consideration quality mentoring efforts by faculty as part of the promotion process.

14. Faculty are sufficiently involved in campus decision making.

15. This institution takes responsibility for educating underprepared students.

16. The criteria for advancement and promotion decisions are clear.

17. Most of the students I teach lack the basic skills for college level work.

18. There is adequate support for faculty development.

19. This institution successfully educates students in remedial/developmental education.

20. Faculty are not prepared to deal with conflict over diversity issues in the classroom.

1 BUILD (RFA-RM-013-16), NRMN (RFA-RM-013-017), CEC (RFA-RM-013-15)

2 DPC data refers to the comprehensive data set comprised of consortium-wide data across all awardee institutions

3 The Publications and Presentations Policy will articulate a separate and unique process for tracking and review of each type of data (which can range from no review or simple notification, to comprehensive review). Further discussion is needed among the ESC, to define the process for tracking and review for each category of data (site-level, and consortium-level data), and therefore is still under development.

4 The CEC tracker is a tool developed by the CEC and utilized by the consortium to upload, collect, store, and manage consortium data.

Page 14 of 37