Form 7 BUILD Site Visit & Case Studies Protocol

OMB #0925-0747
EXP. 11/2019

Attachment 18:
BUILD Site Visit & Case Studies Protocol

Public reporting burden for this collection of information is estimated to vary from 90
minutes to 40 hours per response, including the time for reviewing instructions,
searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. An agency may not conduct or
sponsor, and a person is not required to respond to, a collection of information unless
it displays a currently valid OMB control number. Send comments regarding this
burden estimate or any other aspect of this collection of information, including
suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705
Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0747). Do
not return the completed form to this address.

1

OMB #0925-0747
EXP. 11/2019

2

OMB #0925-0747
EXP. 11/2019

NIH Diversity Program Consortium – CEC – BUILD Site Visit Plan
Site Visit Guiding Evaluation Questions
The following evaluation questions will be answered through collection of qualitative site visit data.

1. How are BUILD programs implementing their site-specific vision for advancing underrepresented
groups (URG) in biomedical research training?
Sub- Questions:
a) What activities are being implemented to promote and support student involvement in the BUILD
program?
b) What activities are being implemented to promote and support faculty involvement in the BUILD
program?
c) What institutional focused activities are being implemented at the BUILD institution and partner
institutions that support program development?
a. How do partner institutions enhance the primary institutions’ capacity to provide biomedical
research training?
d) How are the activities that are being implemented to promote and support participation in the
BUILD program enhancing short-term program outcomes?

BUILD Site Visit Process and Participants
The CEC PI and/or site leads will submit a request for site visits by email to the local to the BUILD PIs.
BUILD PIs will be the primary point of contact for email recruitment of participants. Site visit participants will
include 12-15 people involved with the local BUILD site. Suggested involvement includes: 1 BUILD PI, 3
BUILD Core Directors/Co-PIs, 1 Institutional Research Staff Member, 2 BUILD Evaluators (internal/external),
2 BUILD Faculty, 2 BUILD Students, and 3 other BUILD Administrators and/or Partner Representatives.
The two-day site visit will be comprised of presentations from and discussions with BUILD site
representatives about student, faculty, institutional, and evaluation activities. A tour of BUILD facilities will
also be included. The CEC will conduct one group interview on the development of partnerships and
collaborations at the BUILD site and will meet with BUILD leadership and evaluation to discuss site-level and
consortium-wide evaluation progress. A working lunch is expected to maximize the time spent with sites.
Note: BUILD Site Visit Agenda, BUILD Site Visit Observation Protocol, BUILD Site Visit Group Interview
Protocol and Consent Waivers are attached to this document

3

OMB #0925-0747
EXP. 11/2019

Sample BUILD Site Visit Agenda
Purpose: The purpose of this site visit is to develop an understanding of how your BUILD site is advancing the
goals of the NIH Diversity Program Consortium. We are interested in learning more about your BUILD
student, faculty, and institutional interventions, as well as the development of partnerships and implementation
of site-level and consortium-wide evaluation plans. In this way, site visits serve as an opportunity to document
the degree to which each BUILD site is fulfilling its project objectives and goals.
BUILD site participants (12-15): Suggested Attendance: PI (1), Co-PIs and Core Directors (3), Institutional
Research Staff (1), Evaluators (2), Faculty (2), Students (2), and other BUILD Administration or Partner
Representatives (3)
Day 1
8:00-8:30

Welcome and Introductions Presenting: BUILD PI(s)
PI, Local BUILD, CEC

8:30-10:30

Institutional Level Presentation and Discussion Presenting: BUILD Leadership
and/or BUILD Administration and/or Partner Representatives
Focus: implementation of activities and progress towards meeting stated goals (particular
emphasis on progress generating institutional support and the development of
partnerships and collaborations with pipeline and research-intensive partners), challenges
and barriers, and goal setting for next year(s)

10:30-12:00

Partnerships and Collaborations Group Interview Facilitated by CEC
Focus: gathering of qualitative data on the development of institutional partnerships and
collaborations through a semi-structured group interview
Suggested Attendance: BUILD Leadership, BUILD Administration and Partner
Representative(s)

12:00-1:00

Working Lunch – NRMN Partnership

1:00-2:30

Faculty Level Presentation and Discussion Presenting: Core Director(s)
Focus: implementation of activities and progress towards meeting stated goals, challenges
and barriers, and goal setting for next year(s)

2:30-4:00

Student Level Presentation and Discussion Presenting: Core Director(s)
Focus: implementation of activities and progress towards meeting stated goals, challenges
and barriers, and goal setting for next year(s)

4:00-5:00

Student & Faculty Presentations Presenting: BUILD Students and Faculty

Day 2
8:30-10:00

Tour of BUILD Facilities
4

OMB #0925-0747
EXP. 11/2019

10:00-12:00

Site-Level Evaluation Presentation and Discussion Presenting: BUILD Evaluators
Focus: Review progress on site-level evaluation plans, data collection, and findings

12:00-1:00

Working Lunch – Consortium Activities and Experiences

1:00-3:00

Consortium-Wide Evaluation Plan (CWEP) Implementation Discussion
Focus: Review progress on CWEP implementation, challenges and barriers, questions,
and opportunities for further alignment with site-level evaluation plans
Suggested Attendance: BUILD Leadership, BUILD Evaluators, Institutional Research
Staff

3:00-3:30

Remaining Questions and Next Steps

5

OMB #0925-0747
EXP. 11/2019

6

OMB #0925-0747
EXP. 11/2019

7

OMB #0925-0747
EXP. 11/2019

8

OMB #0925-0747
EXP. 11/2019

9

OMB #0925-0747
EXP. 11/2019

10

OMB #0925-0747
EXP. 11/2019

11

OMB #0925-0747
EXP. 11/2019

12

OMB #0925-0747
EXP. 11/2019

BUILD Site Visit Semi-Structured Group Interview Protocol
BUILD Partnerships and Collaborations
This semi-structured interview protocol contains a list of questions to be drawn from a group interview with
individuals participating in the BUILD Site Visit – Day 1, 10:30am-12:00pm (BUILD PI (1), BUILD Co-PIs
and Core Directors (3), BUILD Administration and/or Partner Representatives (3)). As a flexible framework,
probes may be added or omitted from the interview in response to participant feedback. Interview questions,
however, are expected to stay within the content areas detailed below.
1) It looks like your BUILD site has [insert number here] partnerships. Could you please tell us a little bit
more about each of those partnerships. Specifically, how each partnership was established, the nature of the
partnership or collaboration, and how the BUILD activities “fit” into the larger vision of the [partner
name’s] work in biomedical research.
Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is
currently implementing--the processes of implementation, some of the challenges you may have encountered, as
well as areas of success.
2) Building off of what was shared during the Institutional Core Presentation (8:30am-10:30am), could you
please tell us more about the BUILD activities that are specifically related to developing partnerships and
collaborations. Additionally, please share a little bit about the challenges, how those challenges have been
overcome, and any related successes.
a) Please share the BUILD activities that generate institutional support.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
b) Please share the BUILD activities that focus on intra-departmental collaborations.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
c) Please share the BUILD activities that focus on development of partnerships with pipeline institutions.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
d) Please share the BUILD activities that focus on the development of partnerships with research intensive
institutions.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
3) How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) Please share the BUILD activities that focus on the development of a partnership with NRMN.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
13

iii) What might be some of the success experienced in this area?

OMB #0925-0747
EXP. 11/2019

4) In what ways do you see your work under partnerships and collaborations contributing to the advancement
of URG in biomedical research training?
a) In what ways, if any, do you believe your institution has benefited from BUILD partnerships and
collaborations?
b) In what ways, if any, could engagement with partners and collaborators be improved to better support
the advancement of URG in biomedical research training?
5) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully develop successful partnerships and collaborations? If so, what?
6) How might you summarize lessons learned from developing partnerships and collaborations in the BUILD
program?
7) Are there any questions I haven't asked, or are there additional points you would like to raise that we have
not yet discussed?

14

OMB #0925-0747
EXP. 11/2019

NIH Diversity Program Consortium – CEC – BUILD Case Study Plan
Case Study Guiding Evaluation Questions
The following evaluation questions will be answered through collection of qualitative data at case study visits.
1. How are BUILD programs building capacity and infrastructure for Primary and Partner Institutions to
advance URG in biomedical research training?
Sub- Questions:
a) How are the strategies that are being implemented to enhance faculty development and engagement
advancing biomedical research training for URGs?
b) How are the strategies that are being implemented to enhance student participation and engagement
advancing biomedical research training for URGs?
c) How are the strategies that are being implemented to enhance participation in, and improve the quality
of mentoring activities advancing research training for URGs?
d) How do partner institutions enhance the primary institutions’ capacity to provide biomedical research
training?

Case Study Design
We propose a descriptive, embedded multiple-case design with comparison groups (Yin, 2006). In order
to address the overall research question at the holistic level (i.e., How BUILD programs support building
capacity and infrastructure for Primary and Partner Institutions to advance URG in biomedical research
training), we will examine four embedded sub-cases, specifically, BUILD students, BUILD faculty,
Partnerships with institutions and Partnerships with NRMN. Within each case we will identify the conditions,
features and characteristics that lead to and impede the success of BUILD Programs. These data will be
examined across multiple BUILD sites to describe the extent to which they are replicated across sites and
potentially replicable in other conditions.
In addition to studying BUILD sites, the case study design also includes two matched comparison
institutions without BUILD programs. These institutions will be selected based on similar characteristics to the
BUILD sites. Namely, these institutions would have been recipients of BUILD planning year grants (but not
recipients of full BUILD funding) and they will have other funded institutional undergraduate training grants
(i.e. NIGMS programs: Bridges, IMSD, MARC U-STAR, PREP, and RISE). Ideally these institutions will also
have Higher Education Research Institute (HERI) survey (freshman, senior, and faculty) data available.
Including like institutions with like programs in the case study design allows for an even deeper understanding
of the unique processes and effects of the BUILD program.
We will use culturally responsive evaluation principles/strategies
(http://www.nsf.gov/pubs/2002/nsf02057/nsf02057_5.pdf; American Evaluation Association, 2011) to guide
our case study work. The case studies will allow us to capture the unique cultural experiences, values, norms,
15

OMB #0925-0747
EXP. 11/2019

beliefs and practices of each BUILD program that may influence processes and outcomes (Stake, 1995).

Case Study Methods and Participants
Data Collected. The case studies use multiple data sources. Primary data collection will include direct
observation, group interviews and individual interviews. We will also collect documents related to the BUILD
site development as data sources, including local level BUILD evaluation reports. Consortium level evaluation
data, i.e., survey data and site visit data will also be used in our analyses. Use of these multiple data sources will
ensure a robust and rigorous approach.
The primary data to be will be collected by the case study research team will be through direct
observation and interviews (both group and individual). Direct observation can be one of the most valuable
sources of primary data collection for case study research. We will conduct open-ended observations at the
BUILD Prime and Partner Institution sites and comparison sites. For BUILD programs, if there are more than
four partner sites, we will visit up to four sites, which will be selected based on responses from PI and Program
Manager interviews. We will use the attached observation protocol as a guide during observations.
At BUILD sites, we will also conduct semi-structured individual interviews with BUILD PI’s, program
managers, partner institution directors, as well as semi-structured group interviews with BUILD faculty and
students (graduate and undergraduate). At non-BULD sites, a semi-structured interview with an institutional
director or program manager, as well as semi-structured interviews with faculty and students will be conducted.
The individual and group interview protocols and consent forms are included at the end of this document.
Participants and Incentives. In total, there will be 62 PIs/Directors/Managers, 100 Faculty, and 240
students interviewed from BUILD and non-BUILD programs combined. 37 participants will be interviewed
from each of the BUILD programs and 16 participants will be interviewed from each of the non-BUILD
programs. All site participants will receive a meal as an incentive for participation and student participants will
also receive $20. BUILD (10 sites) and Non-BUILD (2 sites) case study participants per site include:
BUILD Individual Interview (II) Participants:
a) BUILD PI – (1) – 1 hour
b) BUILD Program Managers – (1) – 1 hour
c) BUILD Partner Institution Directors – (4) – 1 hour
BUILD Group Interview (GI) Participants:
d) BUILD Faculty Leads/Co-Leads (coordinating faculty – approximately 3 participants) – 1.5 hour
e) BUILD Faculty Participants (including mentors) (approximately 6 participants who are not also
coordinating faculty) – 1.5 hour
f) BUILD Undergraduate Participants (including mentees and BUILD program dropouts) (approximately
16 participants – 2 groups of 6 (active participants) and 1 group of 4 (dropouts)) – 1.5 hours for each
group
g) BUILD Graduate/Post-Doc Participants (including mentees) (approximately 6 participants) – 1.5 hours
Non-BUILD Individual Interview (II) Participants:
h) Institutional Director/Program Manager – (1) – 1 hour
Non-BUILD Group Interview (GI) Participants:
i) BUILD Comparison Faculty Participants (approximately 5 participants) – 1.5 hours
16

OMB #0925-0747
EXP. 11/2019

j) BUILD Comparison Undergraduate Participants (undergraduate, graduate, and post-doctorate)
(approximately 5 participants) – 1.5 hours
k) BUILD Comparison Graduate/Post-Doc Participants (graduate, and post-doctorate) (approximately 5
participants) – 1.5 hours

Timeline of Data Collection. A case study research team (4 researchers at BUILD sites and 2 researchers at
non-BUILD sites) will visit each site once in year 4 (2018). Each team will spend approximately one week at
BUILD sites and 2 days at Non-BUILD sites. The BUILD and non-BUILD sites will help facilitate visits and
assist in scheduling specific observations and interviews (sites will not be responsible for organizing formal
presentations to the site team). The NIH will help encourage case study cooperation from comparison sites. The
following tables provide an overview of data collection at BUILD and Non-BUILD programs/sites.

TABLE 1. Case Study Team Data Collection Timeline within BUILD Programs
Research
Day 1
Day 2
Day 3
Day 4
Team
Member

Day 5

Lead
Researcher

Observations
(Prime Institution)

PI (II)
Program Manager
(II)
Observations (Prime
Institution)

Faculty Leads (GI)
Observations
(Prime Institution)

Faculty BUILD
Participants (GI)
Observations
(Prime Institution)

Observations
(Prime Institution)

Researcher 2

Observations
(Prime Institution)

Group of
Undergraduate (GI)
Observations (Prime
Institution)

Group of
Undergraduate
(GI)
Observations
(Prime Institution)

Group of
Undergraduate (GI)
Observations
(Prime Institution)

Group of
Graduate/Post-Doc
(GI)
Observations
(Prime Institution)

Researcher 3

Observations
(Prime Institution)

Researcher 4

Observations
(Prime Institution)

Partner Institution
Director (II) and
Observation
Observations (Partner
Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner
Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner Institution)

TABLE 2. Case Study Team Data Collection Timeline within Non-BUILD Sites
Research
Day 1
Day2
Team
Member
Lead
Researcher

BUILD Comparison
Institutional Director/
Program Manager (II)
Observations

BUILD Comparison
Faculty (GI)
Observations

Researcher 2

BUILD Comparison
Graduate Students (GI)
Observations

BUILD Comparison
Graduate Students (GI)
Observations

17

OMB #0925-0747
EXP. 11/2019

Analysis. Observation and interview data will be analyzed in four phases cycles (Saldaña, 2013). First,
data will be assigned preliminary codes through attribute, in vivo, narrative and magnitude coding. During the
second coding cycle, initial codes will be sorted by categorical, thematic, conceptual organization of the data.
Through pattern coding we will be synthesized findings into more meaningful units of analysis (Miles &
Huberman, 1994). By grouping similarly coded passages together and assessing the groupings for thematic
similarity, difference, frequency, sequence and correspondence the final coding scheme is established. Finally,
through elaborative coding, we will examine the data with an eye toward the consortium level logic model (our
conceptual framework). Coding will be done using the qualitative data analysis software Dedoose.

One of the drawbacks of a conceptual framework is that it may limit the inductive approach when
exploring a phenomenon. To safeguard against becoming deductive, researchers will journal their thoughts and
decisions and discuss them to determine if their thinking has become too driven by the framework. We will also
be sure that our multiple data sources converge to understand the overall case.

Site Selection Criteria (If needed).
Ideally, we would conduct a case study at each of the ten BUILD sites. Just like with any other research
design, the more cases included in the case study, the greater confidence we will have in our findings. If we are
asked to conduct the case study on a sample of BUILD sites, we have developed criteria for how we might
select a sample of BUILD sites. Selections have been made based on BUILD site program and activity foci, as
presented at Orientation Site Visits. Table 3 details the key program and activity elements that were taken into
consideration for each site. Table 4 details the mapping of BUILD sites to the potential selection criteria,
inclusion decisions to be determined. While NRMN is not included as a case study site, BUILD site
involvement with NRMN is a selection consideration for BUILD sites. BUILD involvement with NRMN will
be captured in the case studies; special care will be taken to ensure that variability in NRMN involvement is
present in the sample (in the case of low variability, an additional BUILD site will be selected for case study
participation).

18

Case Study Selection Criteria Elements and Elemental Considerations
Element Considered

OMB #0925-0747
EXP. 11/2019

Elemental Considerations

Program Focus
Partnerships

Who is doing intensive (in-depth) and/or extensive (in-breath) partnership
engagement?

Single URM Intensive

Who has a unique focus on a particular Underrepresented Group (URM)?

NIH/NIGMS History

Who has a long history of involvement with NGMIS/NIH programs?

Activity/Goal Focus
Student

Who has an intensive and unique student focus?

Faculty

Who has an intensive and unique faculty focus?

Institutional

Who has intensive and unique institutional capacity building and
infrastructure development?

Cross-Sectional (Systematic
Partnership Alliance)

Who is intensively and uniquely focused on creating a seamless integration of
cross-institutional partnership alliances?

Cross-Sectional (Curriculum
Enhancement)

Who has intensive and unique curriculum enhancement and/or supplemental
instruction?

NRMN Involvement

Who has high and low levels of integration with NRMN mentoring activities?

19

OMB #0925-0747
EXP. 11/2019

Case Study Selection Criteria Mapped with BUILD Program and Activity Focus
Program Focus
Partnerships
Intensive

Expansive

Single
URM
Intensive

Activity/Goal Focus
NIH/
NIGMS
History

Student

CSULB

CSULB

CSULB

CSUN

CSUN

CSUN
Univ.
Detroit
Mercy

Faculty

Univ.
Detroit
Mercy
Morgan
State

Morgan
State

Portland
State

Portland
State

SFSU

SFSU

Morgan
State

CrossSectional
(Systemic
Partnership
Alliance)

CrossSectional
(Curriculum
Enhancement)

CSULB

CSULB

CSULB

CSUN

CSUN

CSUN

Univ. Detroit
Mercy

Univ. Detroit
Mercy

Morgan State

Morgan
State

Institutional

UTEP

SFSU

SFSU

SFSU

UMBC

UMBC

UMBC

UTEP
Xavier

High

TBD

UAF
UMBC

Low

Portland State

UAF
UMBC

NRMN

Xavier

Xavier

Note. All BUILD sites are doing research training and support - embedded in student, faculty, and institutional activities.

20

UMBC

OMB #0925-0747
EXP. 11/2019

References
American Evaluation Association. (2011). American Evaluation Association Public Statement on Cultural
Competence in Evaluation. Fairhaven, MA: Author. Retrieved from www.eval.org.)
Frierson, H. T., Hood, S., & Hughes, G. B. (2010). A guide to conducting culturally-responsive evaluations. In
Frechtling, J., The 2010 user-friendly handbook for project evaluation (pp. 75-96). National Science
Foundation.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. San Francisco,
CA: Sage.
Saldaña, J. (2014). The coding manual for qualitative researchers (2nd ed.). Thousand Oaks, CA: Sage.
Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Yin, R. K. (2006). Case study research: Design and methods (3rd ed.). Thousand Oaks,
CA: Sage.

21

CASE STUDY INTERVIEW AND FOCUS GROUP PROTOCOLS

OMB #0925-0747
EXP. 11/2019

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
PIs (a) & Program Managers (b)
This semi-structured interview protocol contains a list of possible questions to be drawn from in interviews with
individuals participating in the BUILD Case Study. As a flexible framework, probes may be added or omitted
from the interview in response to participant feedback. Interview questions, however, are expected to stay
within the content areas detailed below.
1) Please provide an overall summary of the BUILD activities currently being implemented at your site.
2) How would you describe the URG sub-population you are targeting through BUILD and their unique needs
as a URG sub-population?
3) What, if any, strategies have been implemented at [site name] prior to BUILD to expand URG participation
in biomedical research?
4) How would you say BUILD activities "fit" into the larger vision of [site name's] work in biomedical
research?
Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is
currently implementing--the processes of implementation, some of the challenges you may have encountered, as
well as areas you continue to strengthen.
First, I'd like to start with faculty development and engagement in BUILD programming, then student
participation and engagement, then participation in mentoring activities, as well as the quality of mentoring
activities. I'll be sure to leave time at the end of our conversation for issues or topics you would like to raise
that I might not have asked about. Does that sound all right?
5) Please describe what BUILD activities are currently being implemented to enhance faculty development and
engagement in biomedical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and
participation in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, faculty
development needs are the highest priority currently?
e) Please describe pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
6) Please describe what BUILD activities are currently being implemented to enhance student participant and
engagement in biomedical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
22

OMB #0925-0747
EXP. 11/2019

b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and
participation in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, student needs
are the highest priority currently?
e) Please describe pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?
7) Please describe what BUILD activities are currently being implemented to enhance participation in
mentoring activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement
and participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?

8) Please describe what BUILD activities are currently being implemented to improve the quality of mentoring
activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring
activities?
9) How would you characterize the nature of [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to [site name's] capacity to advance
URG biomedical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG biomedical research training?
10) Please describe the ways in which [site name] has been working with other partner organizations under
BUILD and/or NRMN.
a) In what ways do you see your work under these partnerships contributing to the advancement of URG
biomedical research training?
b) What challenges have you faced in building these partnerships, if any?
23

OMB #0925-0747
EXP. 11/2019

c) What challenges, if any, might these partnerships bring to the implementation of BUILD as you had
envisioned?

11) Overall, how would you characterize [site name's] involvement in BUILD?
a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully implement this programming? If so, what?
b) Are there any questions I haven't asked, or are there additional points you would like to raise that we
have not yet discussed?

24

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
BUILD Partnership Institutions – Director/Lead (c)

OMB #0925-0747
EXP. 11/2019

This semi-structured interview protocol contains a list of questions to be drawn from in interviews with
individuals participating in the BUILD Case Study. As a flexible framework, probes may be added or omitted
from the interview in response to participant feedback. Interview questions, however, are expected to stay
within the content areas detailed below.
8) Please provide an overall summary of the BUILD activities your institution is currently involved in
implementing.
9) Please describe why your institution became partners with [BUILD site] and how this partnership was
established.
10) How would you describe the URG sub-population you are targeting through BUILD and their unique needs
as a URG sub-population?
11) How would you say BUILD activities "fit" into the larger vision of [site name's] work in biomedical
research?
Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is
currently implementing--the processes of implementation, some of the challenges you may have encountered, as
well as areas you continue to strengthen.
First, I'd like to start with faculty development and engagement in BUILD programming, then student
participation and engagement, then participation in mentoring activities, as well as the quality of mentoring
activities. I'll be sure to leave time at the end of our conversation for issues or topics you would like to raise
that I might not have asked about. Does that sound all right?
1) Please describe what, if any, BUILD activities you are currently involved in implementing to enhance
faculty development and engagement in biomedical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and
participation in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, faculty
development needs are the highest priority currently?
e) Please describe any pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
2) Please describe what, if any, BUILD activities you are currently involved in implementing to enhance
student participant and engagement in biomedical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student participation and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
25

OMB #0925-0747
EXP. 11/2019

i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and
participation in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, student needs
are the highest priority currently?
e) Please describe any pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?

3) Please describe what BUILD activities you are currently involved in implementing to enhance participation
in mentoring activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve participation in mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement
and participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
4) Please describe what BUILD activities you are currently involved in implementing to improve the quality of
mentoring activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve the quality of mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring
activities?
5) How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to BUILD’s capacity to advance
URG biomedical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG biomedical research training?
6) In what ways do you see your work under this BUILD partnership contributing to the advancement of URG
biomedical research training?
a) In what ways, if any, do you believe your institution has benefited from this BUILD partnership?
b) What challenges have you faced in establishing this partnership, if any?
c) What challenges, if any, might this partnership bring to your institution?
7) Overall, how would you describe [site name's] involvement in BUILD?
a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully implement this programming? If so, what?
26

OMB #0925-0747
EXP. 11/2019

b) Are there any questions I haven't asked, or are there additional points you would like to raise that we
have not yet discussed?

27

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
BUILD Faculty – Leads/Co-Leads (d)

OMB #0925-0747
EXP. 11/2019

This semi-structured interview protocol contains a list of questions to be drawn from in interviews with
individuals participating in the BUILD Case Study. As a flexible framework, probes may be added or omitted
from the interview in response to participant feedback. Interview questions, however, are expected to stay
within the content areas detailed below.
1. Please provide an overall summary of the BUILD activities your institution is currently involved in
implementing.
2. Please describe why your institution became partners with [BUILD site] and how this partnership was
established.
3. How would you describe the URM sub-population you are targeting through BUILD and their unique needs
as a URM sub-population?
4. How would you say BUILD activities "fit" into the larger vision of [site name's] work in biomedical
research?
Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is
currently implementing--the processes of implementation, some of the challenges you may have encountered, as
well as areas you continue to strengthen.
First, I'd like to start with faculty development and engagement in BUILD programming, then student
participation and engagement, then participation in mentoring activities, as well as the quality of mentoring
activities. I'll be sure to leave time at the end of our conversation for issues or topics you would like to raise
that I might not have asked about. Does that sound all right?
5. Please describe what, if any, BUILD activities you are currently involved in implementing to enhance
faculty development and engagement in biomedical research training for URMs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and
participation in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing biomedical research training for URMs what, in your perspective, faculty
development needs are the highest priority currently?
e) Please describe any pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
6. Please describe what, if any, BUILD activities you are currently involved in implementing to enhance
student participant and engagement in biomedical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student participation and engagement in
BUILD programming?
b) What challenges have you faced in implementation, if any?
28

OMB #0925-0747
EXP. 11/2019

i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and
participation in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing biomedical research training for URMs what, in your perspective, student needs
are the highest priority currently?
e) Please describe any pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?

7. Please describe what BUILD activities you are currently involved in implementing to enhance participation
in mentoring activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve participation in mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement
and participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
8. Please describe what BUILD activities you are currently involved in implementing to improve the quality of
mentoring activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve the quality of mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring
activities?
9. How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to BUILD’s capacity to advance
URM biomedical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URM biomedical research training?
10. In what ways do you see your work under this BUILD partnership contributing to the advancement of URM
biomedical research training?
a) In what ways, if any, do you believe your institution has benefited from this BUILD partnership?
b) What challenges have you faced in establishing this partnership, if any?
c) What challenges, if any, might this partnership bring to your institution?
11. Overall, how would you describe [site name's] involvement in BUILD?
a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully implement this programming? If so, what?
29

OMB #0925-0747
EXP. 11/2019

b) Are there any questions I haven't asked, or are there additional points you would like to raise that we
have not yet discussed?

30

GROUP INTERVIEW PROTOCOL
BUILD Faculty Participants (e)

OMB #0925-0747
EXP. 11/2019

This semi-structured focus group protocol contains a list of possible questions to be drawn from in interviews
with individuals participating in the BUILD Case Study. As a flexible framework, probes may be added or
omitted from the interview in response to participant feedback. Interview questions, however, are expected to
stay within the content areas detailed below.
Understanding current BUILD implementation
1) Please introduce yourself by name, your position within [site name], your role in BUILD programming, and
what you most hope to achieve as a result of BUILD participation.
2) In view of the goals and objectives just raised, what kind of progress do you think [site name] has made
towards realizing those objectives?
a) In your opinion, what are the structures, systems, or processes that have supported this progress?
b) In your opinion, what are some of the challenges or hurdles you have come across in program
implementation and facilitation?
c) In your opinion, what are some of the needed supports, systems, or processes that would greater assist
BUILD implementation and facilitation?
Understanding URG participation and engagement
3) How would you characterize the URG student population at [site name]?
a) To what extent do you feel URG’s are already engaged in biomedical research training?
b) In your view, what are the greatest needs of this sub-population in terms of their enhanced participation
in biomedical research training?
c) In your opinion, to what extent do you feel BUILD programming is addressing these needs?
d) What additional structures, systems and supports need to be in place to further engage URG’s in
biomedical research training?
Understanding faculty participation and engagement
4) In terms of supporting biomedical research training for URGs, what are the greatest areas of need for faculty
development?
a) To what extent have faculty participated in professional development targeted towards the engagement
of URG’s in biomedical research training prior to BUILD?
b) To what extent has participation in BUILD supported faculty development and engagement in URG
biomedical research training?
i) In your opinion, what are the most essential links between faculty development and then, the
translation of this training into strengthened engagement of URG’s in biomedical research training?
c) What additional structures, systems and supports need to be in place to further engage faculty in the
training of URG’s in biomedical research through BUILD?
Understanding BUILD mentoring
5) To what extent do you feel faculty and URG’s are engaged in BUILD mentoring activities?
a) What might some of the structures, systems, and supports in place that incentivize participation?
b) What might be some of the challenges that exist in getting greater numbers of URG’s and faculty
involved in BUILD mentoring activities?
6) If you have participated in mentoring training, how would you describe the quality of BUILD mentoring
training you have received?
a) What could strengthen your preparation as a mentor?
b) What are some of the challenges you have come across as a mentor?
31

7) How would you describe the quality of mentoring URG’s currently receive?
a) What factors either support, or present challenges to improved mentoring quality?

OMB #0925-0747
EXP. 11/2019

Understanding BUILD Partner Sites
8) To what extent have you been involved in any activities with institutional partner sites [provide a site name
as an example]? If applicable, please describe your involvement.
a) In your opinion, has participation in partner site activities been beneficial to you? Please provide detail
as to why, or why not.
b) What, if any, aspects of partner site activities have been particularly valuable to you?
c) What, if any, aspects of partner site activities could be further improved to support you?
9) To what extent have BUILD with partner institutions contributed to the development of the BUILD program
at this site?
a) How have partner institution activities been beneficial to the overall BUILD program development?
b) What, if any, aspects of partner activities have been particularly valuable to URG faculty and students?
c) What, if any, aspects of partner activities could be further improved to support URG faculty and
students?
Understanding NRMN
10) To what extent have you been involved in any NRMN activities? If applicable, please describe your
involvement.
a) In your opinion, has participation in NRMN activities been beneficial to you as a mentor to URG’s?
Please provide detail as to why, or why not.
b) What, if any, aspects of NRMN activities have been particularly valuable to you as a faculty mentor?
c) What, if any, aspects of NRMN activities could be further improved to support you as a faculty mentor?
11) To what extent have you been involved in BUILD activities in association with partner institutions? If
applicable, please describe your involvement.
a) In your opinion, has participation in partner activities been beneficial to you as a mentor to URG’s?
Please provide detail as to why, or why not.
b) What, if any, aspects of partner activities have been particularly valuable to you as a faculty mentor?
c) What, if any, aspects of partner activities could be further improved to support you as a faculty mentor?

32

GROUP INTERVIEW PROTOCOL
BUILD Undergrad (f) and Grad/Post-Doc (g) Participants

OMB #0925-0747
EXP. 11/2019

This semi-structured focus group protocol contains a list of possible questions to be drawn from in interviews
with individuals participating in the BUILD Case Study. As a flexible framework, probes may be added or
omitted from the interview in response to participant feedback. Interview questions, however, are expected to
stay within the content areas detailed below.
Understanding BUILD participating and engagement
1) Please introduce yourself by name, your current area of study, and what inspired your interest in pursuing
biomedical research?
2) How did you find out about BUILD programming?
3) Why did you decide to participate in BUILD programming?
4) To what extent are in you involved in BUILD programming? (What activities do you participate in, and how
involved in those activities are you?)
a) To what extent are in you specifically involved in biomedical research? (What activities do you
participate in, and how involved in those activities are you?)
Participant feedback on overall BUILD programming
5) In your opinion, has participation in these activities been beneficial to you as [title]? If so, how? If not, why
not?
a) What (if any) opportunities or experiences has BUILD participation exposed you to that you think you
might not otherwise have experienced?
b) What might improve your experiences as a BUILD participant?
i) Are there systems, structures, or processes that could further improve your participation in BUILD
activities?
ii) Are there particular skill areas you feel could be further addressed by BUILD programming?
Participant feedback on BUILD URG focus
6) As a URG what, in your opinion, are some of the greatest challenges you face in your success as a
biomedical researcher?
7) In what ways do feel that your personal needs as an URG are sufficiently addressed through BUILD
programming, or in what ways do you feel that they could be more effectively addressed?
8) In what ways do you feel BUILD programming has, or has not acknowledged and respected your cultural,
racial and/or gender background?
9) In what ways do you feel that BUILD faculty are, or are not well prepared to support you as an URG [title]
in biomedical research?
a) What, if any, additional structures, systems and supports need to be in place to further engage students in
the training of URG’s in biomedical research through BUILD?
i) What has been the most valuable about this experience?
ii) How can this experience be strengthened or improved?
Participant feedback on mentoring
10) Do you have a faculty mentor and how long you have been working with them?
a) How did you initially connect with them?
11) How often would you say you meet with your mentor, and for how long?
12) What do you discuss in mentoring sessions with your faculty mentor?
13) In what ways do you feel that having a faculty mentor has been beneficial to you as a [title] in biomedical
research?
33

OMB #0925-0747
EXP. 11/2019

14) In what ways, if any, do you feel that your mentoring experience could be improved to better support you as
a biomedical researcher?
15) In what ways, if any, do you feel that your faculty mentor could be better prepared to support you as a URG
[title] in biomedical research?
Reflection
16) Do you plan to pursue a career in biomedical research?
a) What are the motivating factors behind this decision?
b) What are the greatest challenges URGs face in pursuing a career in biomedical research?
c) If not a career in biomedical research, then what do you aspire to do? How has the BUILD program
supported this decision and success in this area?
Understanding NRMN and Partnerships
17) To what extent have you been involved in any NRMN activities? If applicable, please describe your
involvement.
a) In your opinion, has participation in NRMN activities been beneficial to you as a [title]? Please provide
detail as to why, or why not.
b) What, if any, aspects of NRMN activities have been particularly valuable to you as a [title]?
c) What, if any, aspects of NRMN activities could be further improved to support you as a [title]?
Understanding Partnerships
18) To what extent have you been involved in BUILD activities in association with partner institutions? If
applicable, please describe your involvement.
a) In your opinion, has participation in partner activities been beneficial to you as a [title]? Please provide
detail as to why, or why not.
b) What, if any, aspects of partner activities have been particularly valuable to you as a [title]?
c) What, if any, aspects of partner activities could be further improved to support you as a [title]?
Reflection
19) Would you recommend BUILD participation to other URG [title]? Why or why not?
20) What, in your opinion, would further incentivize additional [title] to participate in BUILD programming?
21) What, if any, barriers to participation exist amongst URG [title]?

34

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
Comparison Group – Institutional Director/Program Manager (h)

OMB #0925-0747
EXP. 11/2019

This semi-structured interview protocol contains a list of possible questions to be drawn from in interviews with
individuals participating in the BUILD Case Study Comparison Group. As a flexible framework, probes may be
added or omitted from the interview in response to participant feedback. Interview questions, however, are
expected to stay within the content areas detailed below.
Understanding current engagement with biomedical students and junior faculty
1) Please introduce yourself by name and your position within [site name]?
2) How would you describe your institutions level of involvement with undergraduate/graduate/post-doctoral
students in biomedical research?
a) Please provide an overall summary of the activities training program initiatives currently being
implemented at your site.
b) How would you describe the URG sub-population you are targeting through BUILD and their unique
needs as a URG sub-population?
c) You might think about the number and types of classes you taught in the field of biomedical research,
advisory roles with students in biomedical research, mentoring roles with student in biomedical
research, and/or specialized programs, training, or support roles with students in biomedical research.
[Probe for: specific examples, numbers of students involved, time/intensity of activities]
3) How would you describe your institutions level of involvement with junior faculty in biomedical research?
a) What, if any, type of advisory or mentoring support do you offer junior faculty, either formally or
informally? [Probe for specific examples, time/intensity of activities]
4) What, if any, strategies have been implemented at [site name] to expand URG participation in biomedical
research?
5) How would you say these activities "fit" into the larger vision of [site name's] work in biomedical
research?
Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is
currently implementing--the processes of implementation, some of the challenges you may have encountered, as
well as areas you continue to strengthen.
First, I'd like to start with faculty development and engagement in BUILD programming, then student
participation and engagement, then participation in mentoring activities, as well as the quality of mentoring
activities. I'll be sure to leave time at the end of our conversation for issues or topics you would like to raise
that I might not have asked about. Does that sound all right?
6) Please describe what activities are currently being implemented to enhance faculty development and
engagement in biomedical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and
participation in programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, faculty
development needs are the highest priority currently?
35

e) Please describe pathways of communication with faculty participants.
i) What feedback, if any, have you received from participating faculty?

OMB #0925-0747
EXP. 11/2019

7) Please describe what BUILD activities are currently being implemented to enhance student participant and
engagement in biomedical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and
participation in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing biomedical research training for URGs what, in your perspective, student needs
are the highest priority currently?
e) Please describe pathways of communication with students’ participants.
i) What feedback, if any, have you received from participating students?
8) Please describe what activities are currently being implemented to enhance participation in mentoring
activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement
and participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
9) Please describe what activities are currently being implemented to improve the quality of mentoring
activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring
activities?
10) How would you characterize the nature of [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to [site name's] capacity to advance
URG biomedical research training?
36

OMB #0925-0747
EXP. 11/2019

c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG biomedical research training?
11) Please describe the ways in which [site name] has been working with other partner organizations.
a) In what ways do you see your work under these partnerships contributing to the advancement of URG
biomedical research training?
b) What challenges have you faced in building these partnerships, if any?
c) What challenges, if any, might these partnerships bring to the implementation as you had envisioned?
12) Overall, how would you characterize [site name's] involvement in undergraduate training programs?
a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully implement this programming? If so, what?
b) Are there any questions I haven't asked, or are there additional points you would like to raise that we
have not yet discussed?

37

GROUP INTERVIEW PROTOCOL
Comparison Group – Faculty (i)

OMB #0925-0747
EXP. 11/2019

This semi-structured focus group protocol contains a list of questions to be drawn from in interviews with
individuals participating in the BUILD Case Study comparison group. As a flexible framework, probes may be
added or omitted from the interview in response to participant feedback. Interview questions, however, are
expected to stay within the content areas detailed below.
Understanding current engagement with biomedical students and junior faculty
12) Please introduce yourself by name and your position within [site name]?
13) How would you describe your level of involvement with undergraduate/graduate/post-doctoral students in
biomedical research?
a) You might think about the number and types of classes you teach in the field of biomedical research,
advisory roles you may have with students in biomedical research, mentoring roles you may have with
student in biomedical research, and/or specialized programs, training, or support roles you may have
with students in biomedical research. [Probe for: specific examples, numbers of students involved,
time/intensity of activities]
14) How would you describe your level of involvement with junior faculty in biomedical research?
a) What, if any, type of advisory or mentoring support do you offer junior faculty, either formally or
informally? [Probe for specific examples, time/intensity of activities]
Understanding URG participation and engagement in biomedical research
15) How would you characterize the URG student population at [site name]?
a) To what extent do you feel URG’s are engaged in biomedical research training?
b) In your view, what are the greatest needs of these sub-populations in terms of their enhanced
participation in biomedical research training?
c) In your opinion, to what extent do you feel current programming at [site name] is addressing these
needs?
d) What additional structures, systems and supports need to be in place to further engage URG’s in
biomedical research training?
Understanding faculty development needs in URG biomedical research training
16) In terms of supporting biomedical research training for URGs, what might be some areas for faculty
development?
a) To what extent have faculty participated in professional development targeted towards the engagement
of URG’s in biomedical research training?
b) Would you say there is an expressed need by faculty for this type of training?
c) What additional structures, systems and supports would need to be in place to further engage faculty in
the training of URG’s in biomedical research?
Understanding faculty mentoring activities
17) To what extent do you feel faculty are engaged in mentoring activities?
a) What might some of the structures, systems, and supports in place that incentivize participation?
b) What might be some of the challenges that exist in getting greater numbers of faculty involved in
mentoring activities?
c) To what extent do you feel faculty are engaged in mentoring activities with biomedical students/postdocs/junior faculty from URG?
18) How would you describe the quality of mentoring biomedical students/post-docs/junior faculty currently
receive?
38

a) What factors either support, or present challenges to improved mentoring quality?

OMB #0925-0747
EXP. 11/2019

Reflection
22) In general, do you see your URG biomedical students pursuing careers in biomedical research? [Also: Do
you have access to this information?]
a) What do you think are the motivating factors behind their decisions?
b) What do you think are the greatest challenges URGs face in pursuing a career in biomedical research?
23) In what ways do you feel students are prepared to pursue your career interests as a result of their research
and training experiences at [site name]?
a) Are there particular research, or specific skill areas in which you feel your program could provide
additional training/guidance to bolster student success in pursuing careers in biomedical research?
24) How would you describe what it means to be successful as an undergraduate/graduate/post-doc student in
the biomedical sciences at [site name]?
25) How would you describe what it means to be successful as a junior faculty member in the biomedical
sciences at [site name]?

39

GROUP INTERVIEW PROTOCOL
Comparison Group – Undergrad (j) Grad/Post-Doc (k) Students

OMB #0925-0747
EXP. 11/2019

This semi-structured focus group protocol contains a list of questions to be drawn from in interviews with
individuals participating in the BUILD Case Study comparison group. As a flexible framework, probes may be
added or omitted from the interview in response to participant feedback. Interview questions, however, are
expected to stay within the content areas detailed below.
Understanding participation and engagement in biomedical research experiences
1) Please introduce yourself by name, your current area of study, and what inspired your interest in pursuing
this academic area?
2) How would you describe your opportunities to engage in research as a student at [site name]?
a) What activities come to mind as being part of your research experience here? [Probe for: specific
examples, time/intensity of research activities]
i) Outside of your standard coursework and labs, have there been additional opportunities to engage in
biomedical research, for example, through supplementary programs, workshops, or internships?
(1) If so, please describe your involvement in these activities. [Probe for: specific examples,
time/intensity of research activities]
(2) How would you describe you interest in engaging in these types of activities? (i.e., Have you
pursued involvement in these activities – why or why not?)
(3) What barriers, if any, might have challenged your participation in these activities?
b) How would you describe the value of your research experience at [site name] to your academic career,
or other career interests you may have?
c) What could be improved about your research experience at [site name]?
i) Are there systems, structures, or processes that could be put in place to further improve your
research experience as a student?
d) In your experience, have you come across any barriers to participating in biomedical research during
your time at [site name]?
i) In answering, please describe how you have come to this conclusion.
Understanding participation and engagement in biomedical mentoring experiences
3) How would you describe your opportunities to be mentored by faculty as a student at [site name]?
a) Would you say you have a mentor at [site name]?
i) If so, how did you connect with your mentor?
(1) In your experience, what were some of the important factors that facilitated this connection?
(2) How would you describe your own personal interest in finding a mentor or in being mentored?
ii) If not, have opportunities arisen for you to connect with a mentor?
(1) How would you describe your own personal interest in finding a mentor or in being mentored?
(2) What challenges might you have experienced in connecting with a mentor?
b) Whether you consider yourself as having a designated mentor or not, what activities come to mind as
being part of your mentoring experience as a student at [site name]? [Probe for: specific examples,
time/intensity of mentoring, who is doing the mentoring, what activities mentoring encompasses]
c) How would you describe the value of your current mentoring experience at [site name] to your
academic career, or other career interests you may have?
d) What could be improved about your current mentoring experience at [site name]?
i) Are there systems, structures, or processes that could be put in place to further improve your
mentoring experience as a student?
Experiences as a URG biomedical student
40

OMB #0925-0747
EXP. 11/2019

4) As a biomedical student from an URG, what, in your opinion, are some of the greatest challenges you face
in your success as a biomedical researcher?
5) In what ways do feel that your personal needs as a student from an URG are sufficiently addressed through
your current academic program, or in what ways do you feel that they could be more effectively addressed?
6) In what ways do you feel your current academic program has, or has not acknowledged and respected your
cultural, racial and/or gender background?
7) In what ways do you feel that faculty are, or are not well prepared to support you as an URG [title] in
biomedical research?
a) What, if any, additional structures, systems and supports need to be in place to further engage URG
students in biomedical research?
8) In what ways, if any, do you feel that faculty you interact with could be better prepared to support you as a
URG [title] in biomedical research?
Reflection
9) Do you plan to pursue a career in biomedical research?
a) What are the motivating factors behind your decision?
b) What are the greatest challenges URGs face in pursuing a career in biomedical research?
c) If not a career in biomedical research, then what do you aspire to do?
10) In what ways do you feel prepared, or feel that you will be prepared, to pursue your career interests as a
result of your research and training experiences at [site name]?
a) Are there particular research, or specific skill areas in which you feel you could use additional
training/guidance outside of what your current academic program provides?
11) How would you describe what it means to be successful in your current academic program?

41

OMB #0925-0747
EXP. 11/2019

BUILD CASE STUDY GENERAL OPEN-ENDED OBSERVATION PROTOCOL

Semi-structured observation will be conducted during case study data collection visits. Participating BUILD
institutions will be asked to prepare in advance for the visiting research team to observe BUILD-specific
activities in progress, such as BUILD program development meetings, mentoring sessions, or lessons in which
new curriculum is currently in use. Because BUILD activities vary extensively by site, the case study research
team will work in close coordination with the BUILD site visit monitoring team and each BUILD institution to
ensure appropriate, practical activities are selected for observation. Participants will include faculty and staff
charged with implementing BUILD at the primary and partner sites as well as students involved in BUILD
activities.
The purpose of observation in this context is to gain a contextualized understanding of the ways in which
different groups of BUILD stakeholders (i.e., program leadership, program implementers, and partners) interact
and collaboratively strategize to advance BUILD’s overarching objective of increasing URG participation and
engagement in biomedical research. While this can often be a dynamic, fluid process wherein observable details
may frequently change, or which may be spontaneously revealed, there are several key areas of interest that are
considered most relevant to this research inquiry. These are as follows:
Context
• Persons/roles present
• Stated purpose/objectives of the meeting/activity being observed
• Type of activity being observed (planned and as it is actually implemented)
• Questions, answers, comments raised (and by who)
Discussion and dialogue amongst those present:
• Capacity of the BUILD Prime institution to successfully implement BUILD activities
o Successes and challenges in implementation
o Benefits and drawbacks of partner collaboration in implementation
o Negotiation of human, financial, time resource allocations
• Goal-setting and progress monitoring
o Planning implementation timelines
o Achievement of, and challenges in achieving scheduled milestones
o Strategic planning to achieve BUILD program objectives
o Perceived stakeholder accountability to implementation partners
o Perceived stakeholder accountability to evaluation theory of success (e.g., output, intermediate
outcomes, and long-term outcomes)
• Development of a shared vision among key stakeholders (admin, faculty at and among BUILD
partner sites)
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
• Development, dissemination and implementation of curriculum and pedagogy
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
• Enacting policy
42

OMB #0925-0747
EXP. 11/2019

•

•
•

o Resource and support needs identified by various BUILD stakeholders
o Response to, and consideration of expressed needs by implementing partners
o Perspective and opinions that take precedence in determining implications for action
Identifying and resolving barriers to effective collaboration across program activities
o Identification of systemic, structural, procedural weaknesses in links among BUILD stakeholders
o Actionable next steps developed to resolve identified issues
o Evidence of follow-up and implementation of action items
Systems, structures, and processes required to promote BUILD partnership sustainability
o Identification of factors promoting long-term BUILD partnership sustainability
o Resource allocation (human, financial, time) in support of sustainability needs
Data-informed decision-making
o Data sources presented
o Perceived reliability/quality of data presented
o How discussion and dialogue is facilitated (and by who) around data interpretation
o What and how data are used to inform next steps
o Perspectives and opinions that take precedence in determining implications for action
o Expressed data needs

43

BUILD CASE STUDY – COMPARISON GROUP –
GENERAL OPEN-ENDED OBSERVATION PROTOCOL

OMB #0925-0747
EXP. 11/2019

Semi-structured observation will be conducted during case study data collection visits. Participating nonBUILD institutions will be asked to prepare in advance for the visiting research team to observe specific
activities in progress, such as program development meetings, mentoring sessions, or lessons in which new
curriculum is currently in use. Because activities vary extensively by site, the case study research team will
work in close coordination with the site visit monitoring team and each non-BUILD institution to ensure
appropriate, practical activities are selected for observation. Participants will include faculty and staff charged
with implementing undergraduate/graduate training programs.
The purpose of observation in this context is to gain a contextualized understanding of the ways in which
different groups of stakeholders (i.e., program leadership, program implementers, and partners) interact and
collaboratively strategize to advance URG participation and engagement in biomedical research. While this can
often be a dynamic, fluid process wherein observable details may frequently change, or which may be
spontaneously revealed, there are several key areas of interest that are considered most relevant to this research
inquiry. These are as follows:
Context
• Persons/roles present
• Stated purpose/objectives of the meeting/activity being observed
• Type of activity being observed (planned and as it is actually implemented)
• Questions, answers, comments raised (and by who)
Discussion and dialogue amongst those present:
• Capacity of the institution to successfully implement activities
o Successes and challenges in implementation
o Benefits and drawbacks of partner collaboration in implementation
o Negotiation of human, financial, time resource allocations
• Goal-setting and progress monitoring
o Planning implementation timelines
o Achievement of, and challenges in achieving scheduled milestones
o Strategic planning to achieve program objectives
o Perceived stakeholder accountability to implementation partners
o Perceived stakeholder accountability to evaluation theory of success (e.g., output, intermediate
outcomes, and long-term outcomes)
• Development of a shared vision among key stakeholders (admin, faculty, students)
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
• Development, dissemination and implementation of curriculum and pedagogy
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
• Enacting policy
o Resource and support needs identified by various stakeholders
44

OMB #0925-0747
EXP. 11/2019

•

•
•

o Response to, and consideration of expressed needs by implementing partners
o Perspective and opinions that take precedence in determining implications for action
Identifying and resolving barriers to effective collaboration across program activities
o Identification of systemic, structural, procedural weaknesses in links among stakeholders
o Actionable next steps developed to resolve identified issues
o Evidence of follow-up and implementation of action items
Systems, structures, and processes required to promote partnership sustainability
o Identification of factors promoting long-term partnership sustainability
o Resource allocation (human, financial, time) in support of sustainability needs
Data-informed decision-making
o Data sources presented
o Perceived reliability/quality of data presented
o How discussion and dialogue is facilitated (and by who) around data interpretation
o What and how data are used to inform next steps
o Perspectives and opinions that take precedence in determining implications for action
o Expressed data needs

45