Form 11 BUILD Site Visit and Case Studies protocol

Attachment 23:  
BUILD Site Visit & Case Studies Protocol 

1

Note: The first pages of this document describe the protocols for the proposed activity.  The required burden statement is included 
in the interview protocols to be found with each of the specific data collection activities described subsequently (pages 3, 12, 21, 22, 
25, 28, 31, 33, 35, 38, and 40). 

NIH Diversity Program Consortium – CEC – BUILD Site Visit Plan  
As part of the mixed methods consortium level evaluation design, we propose to conduct three one-to-twoday site visits of the 10 BUILD sites during the grant period. The purpose of the site visits is to develop an
understanding of each BUILD site in their effort to advance programs for URG bio-medical research training. In
particular, we are interested in better understanding how each BUILD site is implementing student, faculty, and
institutional interventions, as well as the development of partnerships and implementation of site-level and
consortium-wide evaluation plans. In this way, site visits will serve as an opportunity to document the degree to
which each BUILD site is fulfilling its project objectives and goals.

Site Visit Focus

Site Visits will largely focus on describing the activities BUILD sites implement to promote and support
URG bio-medical research training at each site. Using the site level BUILD program logic model as a guiding
framework, site visits will offer the CEC the opportunity to provide a narrative description of the relationships
among each BUILD site’s inputs, activities and outputs and some, but not all, short-term program outcomes.
Qualitative data will be collected on the ways and extent to which institutional partnerships are being developed at
sites.
Site visits are an occasion for sites to showcase the defining features of their programs as well as to discuss
any challenges related to program implementation and evaluation. The site visit is a three-way exchange of
information among the BUILD site, the CEC and NIH that will allow for critical face-to-face learning to transpire.
They are an opportunity to develop trusting relationships that help to promote knowledge exchange and learning
from the evaluation.

Site Visit Guiding Evaluation Questions

The following evaluation questions will be answered through collection of qualitative site visit data.
1. How are BUILD programs implementing their site-specific vision for advancing URG bio-medical
research training?
Sub- Questions:

a) What activities are being implemented to promote and support student involvement in the BUILD
program?
b) What activities are being implemented to promote and support faculty involvement in the BUILD
program?
c) What institutional focused activities are being implemented at the BUILD institution and partner
institutions that support program development?
1

OMB #0925-XXXX
EXP. XX/XXXX

a. How do partner institutions enhance the primary institutions’ capacity to provide bio-medical
research training?

d) How are the activities that are being implemented to promote and support participation in the BUILD
program enhancing short-term program outcomes?

Timeline

We will conduct three site visits at each BUILD site during the grant period. Site visits will be conducted
over the course of one-two days by a team of CEC faculty and staff in coordination with NIH project officers and
scientists. Site visits will be conducted in 2016 (Spring/Summer), 2017 (Fall), and 2018/2019 (Fall/Winter). CEC
BUILD Teams (2 CEC EC/DC Senior Leads, 2 CEC EC/DC Associate Leads, 1 Research/Admin Staff) will
attend site visits. To the extent possible, there will be continuity in CEC team membership across the site visits.
Additional CEC staff may attend the site visit as deemed necessary.

BUILD Site Visit Process and Participants

The CEC PI and/or site leads will submit a request for site visits by email to the local to the BUILD PIs.
BUILD PIs will be the primary point of contact for email recruitment of participants. Site visit participants will
include 12-15 people involved with the local BUILD site. Suggested involvement includes: 1 BUILD PI, 3 BUILD
Core Directors/Co-PIs, 1 Institutional Research Staff Member, 2 BUILD Evaluators (internal/external), 2 BUILD
Faculty, 2 BUILD Students, and 3 other BUILD Administrators and/or Partner Representatives.
The two-day site visit will be comprised of presentations from and discussions with BUILD site
representatives about student, faculty, institutional, and evaluation activities. A tour of BUILD facilities will also be
included. The CEC will conduct one group interview on the development of partnerships and collaborations at the
BUILD site and will meet with BUILD leadership and evaluation to discuss site-level and consortium-wide
evaluation progress. A working lunch is expected so as to maximize the time spent with sites.
Note: BUILD Site Visit Agenda, BUILD Site Visit Observation Protocol, BUILD Site Visit Group Interview
Protocol and Consent Waivers are attached to this document

2

Sample BUILD Site Visit Agenda

OMB #0925-XXXX
EXP. XX/XXXX

Public reporting burden for this collection of information is estimated to average 24 hours per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining
the data needed, and completing and reviewing the collection of information. An agency may not
conduct or sponsor, and a person is not required to respond to, a collection of information unless
it displays a currently valid OMB control number. Send comments regarding this burden estimate or
any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH,
Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892‐7974, ATTN: PRA
(0925‐xxxx*). Do not return the completed form to this address.

Purpose: The purpose of this site visit is to develop an understanding of how your BUILD site is
advancing the goals of the NIH Diversity Program Consortium. We are interested in learning more about
your BUILD student, faculty, and institutional interventions, as well as the development of partnerships
and implementation of site‐level and consortium‐wide evaluation plans. In this way, site visits serve as
an opportunity to document the degree to which each BUILD site is fulfilling its project objectives and
goals.

BUILD site participants (12‐15): Suggested Attendance: PI (1), Co‐PIs and Core Directors (3),
Institutional Research Staff (1), Evaluators (2), Faculty (2), Students (2), and other BUILD Administration
or Partner Representatives (3)

Day 1

8:00‐8:30        Welcome and Introductions Presenting: BUILD PI(s) 
PI, Local BUILD, CEC  

8:30‐10:30    Institutional Level Presentation and Discussion Presenting: BUILD Leadership  
                       and/or BUILD Administration and/or Partner Representatives 
Focus: implementation of activities and progress towards meeting stated goals 
(particular emphasis on progress generating institutional support and the development 
of partnerships and collaborations with pipeline and research intensive partners), 
challenges and barriers, and goal setting for next year(s) 

10:30‐12:00     Partnerships and Collaborations Group Interview Facilitated by CEC 
Focus: gathering of qualitative data on the development of institutional partnerships 
and collaborations through a semi‐structured group interview 
Suggested Attendance: BUILD Leadership, BUILD Administration and Partner
Representative(s) 

12:00‐1:00      Working Lunch – NRMN Partnership 

3

OMB #0925-XXXX
EXP. XX/XXXX

1:00‐2:30        Faculty Level Presentation and Discussion Presenting: Core Director(s) 
Focus: implementation of activities and progress towards meeting stated goals, 
challenges and barriers, and goal setting for next year(s) 

2:30‐4:00         Student Level Presentation and Discussion Presenting: Core Director(s) 
Focus: implementation of activities and progress towards meeting stated goals, 
challenges and barriers, and goal setting for next year(s)   

4:00‐5:00         Student & Faculty Presentations Presenting: BUILD Students and Faculty 

Day 2 

8:30‐10:00      Tour of BUILD Facilities 

10:00‐12:00    Site‐Level Evaluation Presentation and Discussion Presenting: BUILD Evaluators 
Focus: Review progress on site‐level evaluation plans, data collection, and findings 

12:00‐1:00      Working Lunch – Consortium Activities and Experiences 

1:00‐3:00        Consortium‐Wide Evaluation Plan (CWEP) Implementation Discussion 
Focus: Review progress on CWEP implementation, challenges and barriers, questions, 
and opportunities for further alignment with site‐level evaluation plans 
Suggested Attendance: BUILD Leadership, BUILD Evaluators, Institutional Research Staff 

3:00‐3:30        Remaining Questions and Next Steps 

4

OMB #0925-XX
XXX
EXP. XX/XX
XXX

5

OMB #0925-XX
XXX
EXP. XX/XX
XXX

6

OMB #0925-XX
XXX
EXP. XX/XX
XXX

7

OMB #0925-XX
XXX
EXP. XX/XX
XXX

8

OMB #0925-XX
XXX
EXP. XX/XX
XXX

9

OMB #0925-XX
XXX
EXP. XX/XX
XXX

10

OMB #0925-XX
XXX
EXP. XX/XX
XXX

11

OMB #0925-XXXX
EXP. XX/XXXX

BUILD Site Visit Semi‐Structured Group Interview Protocol 

BUILD Partnerships and Collaborations
This semi-structured interview protocol contains a list of questions to be drawn from a group interview with individuals participating in
the BUILD Site Visit – Day 1, 10:30am-12:00pm (BUILD PI (1), BUILD Co-PIs and Core Directors (3), BUILD
Administration and/or Partner Representatives (3)). As a flexible framework, probes may be added or omitted from the interview in
response to participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
1) It looks like your BUILD site has [insert number here] partnerships. Could you please tell us a little bit more about
each of those partnerships. Specifically, how each partnership was established, the nature of the partnership or
collaboration, and how the BUILD activities “fit” into the larger vision of the [partner name’s] work in biomedical research.

Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is currently implementing-the processes of implementation, some of the challenges you may have encountered, as well as areas of success.
2) Building off of what was shared during the Institutional Core Presentation (8:30am-10:30am), could you please
tell us more about the BUILD activities that are specifically related to developing partnerships and
collaborations. Additionally, please share a little bit about the challenges, how those challenges have been
overcome, and any related successes.
a) Please share the BUILD activities that generate institutional support.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
b) Please share the BUILD activities that focus on intra-departmental collaborations.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
c) Please share the BUILD activities that focus on development of partnerships with pipeline institutions.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
d) Please share the BUILD activities that focus on the development of partnerships with research intensive
institutions.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?
12

OMB #0925-XXXX
EXP. XX/XXXX

3) How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) Please share the BUILD activities that focus on the development of a partnership with NRMN.
i) What might be some of the challenges encountered?
ii) How have those challenges been overcome?
iii) What might be some of the success experienced in this area?

4) In what ways do you see your work under partnerships and collaborations contributing to the advancement of
URG bio-medical research training?
a) In what ways, if any, do you believe your institution has benefited from BUILD partnerships and
collaborations?
b) In what ways, if any, could engagement with partners and collaborators be improved to better support the
advancement of URG bio-medical research training?
5) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to successfully
develop successful partnerships and collaborations? If so, what?
6) How might you summarize lessons learned from developing partnerships and collaborations in the BUILD
program?
7) Are there any questions I haven't asked, or are there additional points you would like to raise that we have not
yet discussed?

13

OMB #0925-XXXX
EXP. XX/XXXX

NIH Diversity Program Consortium – CEC – BUILD Case Study Plan 
In addition to the site visits, we are proposing to conduct in-depth case studies of the BUILD sites. The
purpose of the case studies is to provide a holistic, in-depth description of the BUILD program at the consortium
level. Case studies will provide a clearer understanding of each BUILD site’s successes and challenges by offering
explanations and descriptions that will allow for us to generalize at the consortium level. They will also provide us
with the opportunity to leverage the findings of the site/local level BUILD evaluations, the survey data collected for
the consortium level evaluation, and to better understand the relationship between the BUILD and NRMN
programs. The case study will promote learning across the sites and cultures, which will enhance the validity and use
of the overall consortium evaluation.

Case Study Focus
The primary focus of the BUILD case study it to describe how the BUILD program is enhancing the
capacity of campuses to attract, serve, and promote the success of URGs in biomedical research. The BUILD case
studies will focus on the processes and procedures that build capacity and infrastructure to advance bio-medical
research training. This work requires an understanding and analysis of how to build and successfully implement the
“systems” “structures” at the institutional (in some cases this includes building or enhancing facilities), faculty and
student levels— to achieve this goal. The primary theoretical/conceptual framework that will guide the analysis of
case study is the consortium level logic model, developed by the CEC.

Case Study Guiding Evaluation Questions

The following evaluation questions will be answered through collection of qualitative data at case study

visits.
1. How are BUILD programs building capacity and infrastructure for Primary and Partner Institutions to
advance URG bio-medical research training?
Sub- Questions:
a) How are the strategies that are being implemented to enhance faculty development and engagement
advancing bio-medical research training for URGs?
b) How are the strategies that are being implemented to enhance student participation and engagement
advancing bio-medical research training for URGs?
c) How are the strategies that are being implemented to enhance participation in, and improve the quality of
mentoring activities advancing research training for URGs?
d) How do partner institutions enhance the primary institutions’ capacity to provide bio-medical research
training?

Case Study Design

We propose a descriptive, embedded multiple-case design with comparison groups (Yin, 2006). In order to
address the overall research question at the holistic level (i.e., How BUILD programs support building capacity and
infrastructure for Primary and Partner Institutions to advance URG bio-medical research training), we will examine
14

OMB #0925-XXXX
EXP. XX/XXXX

four embedded sub-cases, specifically, BUILD students, BUILD faculty, Partnerships with institutions and
Partnerships with NRMN. Within each case we will identify the conditions, features and characteristics that lead to
and impede the success of BUILD Programs. These data will be examined across multiple BUILD sites to describe
the extent to which they are replicated across sites and potentially replicable in other conditions.
In addition to studying BUILD sites, the case study design also includes two matched comparison
institutions without BUILD programs. These institutions will be selected based on similar characteristics to the
BUILD sites. Namely, these institutions would have been recipients of BUILD planning year grants (but not
recipients of full BUILD funding) and they will have other funded institutional undergraduate training grants (i.e.
NIGMS programs: Bridges, IMSD, MARC U-STAR, PREP, and RISE). Ideally these institutions will also have
Higher Education Research Institute (HERI) survey (freshman, senior, and faculty) data available. Including like
institutions with like programs in the case study design allows for an even deeper understanding of the unique
processes and effects of the BUILD program.

We will use culturally responsive evaluation principles/strategies
(http://www.nsf.gov/pubs/2002/nsf02057/nsf02057_5.pdf; American Evaluation Association, 2011) to guide our
case study work. The case studies will allow us to capture the unique cultural experiences, values, norms, beliefs and
practices of each BUILD program that may influence processes and outcomes (Stake, 1995).

Case Study Methods and Participants
Data Collected. The case studies use multiple data sources. Primary data collection will include direct
observation, group interviews and individual interviews. We will also collect documents related to the BUILD site
development as data sources, including local level BUILD evaluation reports. Consortium level evaluation data, i.e.,
survey data and site visit data will also be used in our analyses. Use of these multiple data sources will ensure a
robust and rigorous approach.
The primary data to be will be collected by the case study research team will be through direct observation
and interviews (both group and individual). Direct observation can be one of the most valuable sources of primary
data collection for case study research. We will conduct open-ended observations at the BUILD Prime and Partner
Institution sites and comparison sites. For BUILD programs, if there are more than four partner sites, we will visit
up to four sites, which will be selected based on responses from PI and Program Manager interviews. We will use
the attached observation protocol as a guide during observations.
At BUILD sites, we will also conduct semi-structured individual interviews with BUILD PI’s, program
managers, partner institution directors, as well as semi-structured group interviews with BUILD faculty and students
(graduate and undergraduate). At non-BULD sites, a semi-structured interview with an institutional director or
program manager, as well as semi-structured interviews with faculty and students will be conducted. The individual
and group interview protocols and consent forms are included at the end of this document.
Participants and Incentives. In total, there will be 62 PIs/Directors/Managers, 100 Faculty, and 240
students interviewed from BUILD and non-BUILD programs combined. 37 participants will be interviewed from
each of the BUILD programs and 16 participants will be interviewed from each of the non-BUILD programs. All
site participants will receive a meal as an incentive for participation and student participants will also receive $20.
BUILD (10 sites) and Non-BUILD (2 sites) case study participants per site include:
BUILD Individual Interview (II) Participants:
a) BUILD PI – (1) – 1.5 hours
b) BUILD Program Managers – (1) – 1.5 hours
15

OMB #0925-XXXX
EXP. XX/XXXX

c) BUILD Partner Institution Directors – (4) – 1.5 hours

BUILD Group Interview (GI) Participants:
d) BUILD Faculty Leads/Co-Leads (coordinating faculty – approximately 3 participants) – 1.5 hours
e) BUILD Faculty Participants (including mentors) (approximately 6 participants who are not also
coordinating faculty) – 1.5 hours
f) BUILD Undergraduate Participants (including mentees and BUILD program dropouts) (approximately 16
participants – 2 groups of 6 (active participants) and 1 group of 4 (dropouts)) – 1.5 hours for each group
g) BUILD Graduate/Post-Doc Participants (including mentees) (approximately 6 participants) – 1.5 hours
Non-BUILD Individual Interview (II) Participants:
h) Institutional Director/Program Manager – (1) – 1.5 hours
Non-BUILD Group Interview (GI) Participants:
i) BUILD Comparison Faculty Participants (approximately 5 participants) – 1.5 hours
j) BUILD Comparison Undergraduate Participants (undergraduate, graduate, and post-doctorate)
(approximately 5 participants) – 1.5 hours
k) BUILD Comparison Graduate/Post-Doc Participants (graduate, and post-doctorate) (approximately 5
participants) – 1.5 hours
Timeline of Data Collection. A case study research team (4 researchers at BUILD sites and 2 researchers at
non-BUILD sites) will visit each site once in year 4 (2018). Each team will spend approximately one week at
BUILD sites and 2 days at Non-BUILD sites. The BUILD and non-BUILD sites will help facilitate visits and assist
in scheduling specific observations and interviews (sites will not be responsible for organizing formal presentations
to the site team). The NIH will help encourage case study cooperation from comparison sites. The following tables
provide an overview of data collection at BUILD and Non-BUILD programs/sites.

TABLE 1. Case Study Team Data Collection Timeline within BUILD Programs
Research
Day 1
Day 2
Day 3
Day 4
Team
Member

Day 5

Lead Researcher

Observations
(Prime Institution)

PI (II)
Program Manager (II)
Observations (Prime
Institution)

Faculty Leads (GI)
Observations
(Prime Institution)

Faculty BUILD
Participants (GI)
Observations
(Prime Institution)

Observations
(Prime Institution)

Researcher 2

Observations
(Prime Institution)

Group of
Undergraduate (GI)
Observations (Prime
Institution)

Group of
Undergraduate (GI)
Observations
(Prime Institution)

Group of
Undergraduate (GI)
Observations
(Prime Institution)

Group of
Graduate/Post-Doc
(GI)
Observations
(Prime Institution)

Researcher 3

Observations
(Prime Institution)

Researcher 4

Observations
(Prime Institution)

Partner Institution
Director (II) and
Observation
Observations (Partner
Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner Institution)

Partner Institution
Director (II) and
Observation
Observations
(Partner Institution)

16

TABLE 2. Case Study Team Data Collection Timeline within Non-BUILD Sites
Research
Day 1
Day2
Team
Member
Lead Researcher

BUILD Comparison
Institutional Director/
Program Manager (II)
Observations

BUILD Comparison
Faculty (GI)
Observations

Researcher 2

BUILD Comparison
Graduate Students (GI)
Observations

BUILD Comparison
Graduate Students (GI)
Observations

OMB #0925-XXXX
EXP. XX/XXXX

Analysis. Observation and interview data will be analyzed in four phases cycles (Saldaña, 2013). First, data
will be assigned preliminary codes through attribute, in vivo, narrative and magnitude coding. During the second
coding cycle, initial codes will sorted by categorical, thematic, conceptual organization of the data. Through pattern
coding we will synthesized findings into more meaningful units of analysis (Miles & Huberman, 1994). By grouping
similarly coded passages together and assessing the groupings for thematic similarity, difference, frequency,
sequence and correspondence the final coding scheme is established. Finally, through elaborative coding, we will
examine the data with an eye toward the consortium level logic model (our conceptual framework). Coding will be
done using the qualitative data analysis software Dedoose.
One of the drawbacks of a conceptual framework is that it may limit the inductive approach when exploring
a phenomenon. To safeguard against becoming deductive, researchers will journal their thoughts and decisions and
discuss them to determine if their thinking has become too driven by the framework. We will also be sure that our
multiple data sources converge in an attempt to understand the overall case.

Site Selection Criteria (If needed).

Ideally, we would conduct a case study at each of the ten BUILD sites. Just like with any other research
design, the more cases included in the case study, the greater confidence we will have in our findings. If we are
asked to conduct the case study on a sample of BUILD sites, we have developed criteria for how we might select a
sample of BUILD sites. Selections have been made based on BUILD site program and activity foci, as presented at
Orientation Site Visits. Table 3 details the key program and activity elements that were taken into consideration for
each site. Table 4 details the mapping of BUILD sites to the potential selection criteria, inclusion decisions to be
determined. While NRMN is not included as a case study site, BUILD site involvement with NRMN is a selection
consideration for BUILD sites. BUILD involvement with NRMN will be captured in the case studies; special care
will be taken to ensure that variability in NRMN involvement is present in the sample (in the case of low variability,
an additional BUILD site will be selected for case study participation).

17

TABLE 3.
Case Study Selection Criteria Elements and Elemental Considerations
Element Considered

OMB #0925-XXXX
EXP. XX/XXXX

Elemental Considerations

Program Focus
Partnerships

Who is doing intensive (in-depth) and/or extensive (in-breath) partnership
engagement?

Single URM Intensive

Who has a unique focus on a particular Underrepresented Group (URM)?

NIH/NIGMS History

Who has a long history of involvement with NGMIS/NIH programs?

Activity/Goal Focus
Student

Who has an intensive and unique student focus?

Faculty

Who has an intensive and unique faculty focus?

Institutional

Who has intensive and unique institutional capacity building and infrastructure
development?

Cross-Sectional (Systematic
Partnership Alliance)

Who is intensively and uniquely focused on creating a seamless integration of
cross-institutional partnership alliances?

Cross-Sectional (Curriculum
Enhancement)

Who has intensive and unique curriculum enhancement and/or supplemental
instruction?

NRMN Involvement

Who has high and low levels of integration with NRMN mentoring activities?

18

OMB #0925-XXXX
EXP. XX/XXXX

TABLE 4.
Case Study Selection Criteria Mapped with BUILD Program and Activity Focus

Program Focus
Partnerships

Intensive

Expansive

Single
URM
Intensive

Activity/Goal Focus
NIH/
NIGMS
History

Student

CSULB

CSULB

CSULB

CSUN

CSUN

CSUN
Univ.
Detroit
Mercy

Faculty

Univ.
Detroit
Mercy
Morgan
State

Morgan
State
Portland
State

Portland
State

SFSU

SFSU

Morgan
State

CrossSectional
(Systemic
Partnership
Alliance)

CrossSectional
(Curriculum
Enhancement)

CSULB

CSULB

CSULB

CSUN

CSUN

CSUN

Univ. Detroit
Mercy

Univ. Detroit
Mercy

Morgan State

Morgan State

Institutional

UTEP

SFSU

SFSU

UMBC

UMBC

UTEP
Xavier

Xavier

High

TBD
SFSU

UAF
UMBC

Low

Portland State

UAF
UMBC

NRMN

Xavier

Note. All BUILD sites are doing research training and support - embedded in student, faculty, and institutional activities.

19

UMBC

UMBC

OMB #0925-XXXX
EXP. XX/XXXX

References
American Evaluation Association. (2011). American Evaluation Association Public Statement on Cultural Competence in
Evaluation. Fairhaven, MA: Author. Retrieved from www.eval.org.)
Frierson, H. T., Hood, S., & Hughes, G. B. (2010). A guide to conducting culturally-responsive evaluations. In
Frechtling, J., The 2010 user-friendly handbook for project evaluation (pp. 75-96). National Science Foundation.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. San Francisco, CA: Sage.
Saldaña, J. (2014). The coding manual for qualitative researchers (2nd ed.). Thousand Oaks, CA: Sage.
Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Yin, R. K. (2006). Case study research: Design and methods (3rd ed.). Thousand Oaks,
CA: Sage.

20

CASE STUDY PREPARATION

OMB #0925-XXXX
EXP. XX/XXXX

Public reporting burden for this collection of information is estimated to average 40 hours per response, including
the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person
is not required to respond to, a collection of information unless it displays a currently valid OMB control
number. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC
7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to this address.

Interviews

Please work with the case study team to identify and schedule interviews with the following people in your program:
Individual Interviews:
 Institutional Director/Program Manager
 Principal Investigator (if a BUILD program)
 Partner Institution Directors (if a BUILD program)
Group Interviews:
 Faculty (including those with leadership positions in programs and those that are mentors)
 Undergraduate students active in your program (including those who may be mentees in your program)
 Undergraduate students who are no longer active in your program
 Graduate students or post-doctoral scholars who are part of your program, including as mentees
Observation Activities
Please work with the case study team to identify program-specific activities that can be observed by the team, such
as program development meetings, mentoring sessions, or lessons in which new curriculum is currently in use.

21

CASE STUDY INTERVIEW AND FOCUS GROUP PROTOCOLS

OMB #0925-XXXX
EXP. XX/XXXX

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
PIs (a) & Program Managers (b)
This semi-structured interview protocol contains a list of possible questions to be drawn from in interviews with individuals participating
in the BUILD Case Study. As a flexible framework, probes may be added or omitted from the interview in response to participant
feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
1) Please provide an overall summary of the BUILD activities currently being implemented at your site.
2) How would you describe the URG sub-population you are targeting through BUILD and their unique needs as
a URG sub-population?
3) What, if any, strategies have been implemented at [site name] prior to BUILD in an effort to expand URG
participation in bio-medical research?
4) How would you say BUILD activities "fit" into the larger vision of [site name's] work in bio-medical research?

Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is currently implementing-the processes of implementation, some of the challenges you may have encountered, as well as areas you continue to strengthen.
First I'd like to start with faculty development and engagement in BUILD programming, then student participation and engagement,
then participation in mentoring activities, as well as the quality of mentoring activities. I'll be sure to leave time at the end of our
conversation for issues or topics you would like to raise that I might not have asked about. Does that sound all right?
5) Please describe what BUILD activities are currently being implemented to enhance faculty development and
engagement in bio-medical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and participation
in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing bio-medical research training for URGs what, in your perspective, faculty
development needs are the highest priority at this time?
e) Please describe pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
6) Please describe what BUILD activities are currently being implemented to enhance student participant and
engagement in bio-medical research.
22

OMB #0925-XXXX
EXP. XX/XXXX

a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and participation
in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing bio-medical research training for URGs what, in your perspective, student needs are
the highest priority at this time?
e) Please describe pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?
7) Please describe what BUILD activities are currently being implemented to enhance participation in mentoring
activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement and
participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
8) Please describe what BUILD activities are currently being implemented to improve the quality of mentoring
activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring activities?

9) How would you characterize the nature of [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to [site name's] capacity to advance URG
bio-medical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG bio-medical research training?
10) Please describe the ways in which [site name] has been working with other partner organizations under BUILD
and/or NRMN.
a) In what ways do you see your work under these partnerships contributing to the advancement of URG biomedical research training?
23

OMB #0925-XXXX
EXP. XX/XXXX

b) What challenges have you faced in building these partnerships, if any?
c) What challenges, if any, might these partnership bring to the implementation of BUILD as you had
envisioned?

11) Overall, how would your characterize [site name's] involvement in BUILD?
a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to successfully
implement this programming? If so, what?
b) Are there any questions I haven't asked, or are there additional points you would like to raise that we have
not yet discussed?

24

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
BUILD Partnership Institutions – Director/Lead (c)

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured interview protocol contains a list of questions to be drawn from in interviews with individuals participating in the
BUILD Case Study. As a flexible framework, probes may be added or omitted from the interview in response to participant feedback.
Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
1) Please provide an overall summary of the BUILD activities your institution is currently involved in
implementing.
2) Please describe why your institution became partners with [BUILD site] and how this partnership was
established.
3) How would you describe the URG sub-population you are targeting through BUILD and their unique needs as
a URG sub-population?
4) How would you say BUILD activities "fit" into the larger vision of [site name's] work in bio-medical research?

Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is currently implementing--

the processes of implementation, some of the challenges you may have encountered, as well as areas you continue to strengthen.

First I'd like to start with faculty development and engagement in BUILD programming, then student participation and engagement,
then participation in mentoring activities, as well as the quality of mentoring activities. I'll be sure to leave time at the end of our
conversation for issues or topics you would like to raise that I might not have asked about. Does that sound all right?
1) Please describe what, if any, BUILD activities you are currently involved in implementing to enhance faculty
development and engagement in bio-medical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and participation
in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing bio-medical research training for URGs what, in your perspective, faculty
development needs are the highest priority at this time?
e) Please describe any pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
2) Please describe what, if any, BUILD activities you are currently involved in implementing to enhance student
participant and engagement in bio-medical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
25

OMB #0925-XXXX
EXP. XX/XXXX

b)
c)
d)
e)

i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student participation and engagement in BUILD
programming?
What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
From your perspective, what about your current programming elicits student engagement and participation
in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
In terms of enhancing bio-medical research training for URGs what, in your perspective, student needs are
the highest priority at this time?
Please describe any pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?

3) Please describe what BUILD activities you are currently involved in implementing to enhance participation in
mentoring activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve participation in mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement and
participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
4) Please describe what BUILD activities you are currently involved in implementing to improve the quality of
mentoring activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve the quality of mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring activities?
5) How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to BUILD’s capacity to advance URG
bio-medical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG bio-medical research training?
6) In what ways do you see your work under this BUILD partnership contributing to the advancement of URG
bio-medical research training?
a) In what ways, if any, do you believe your institution has benefited from this BUILD partnership?
b) What challenges have you faced in establishing this partnership, if any?
c) What challenges, if any, might this partnership bring to your institution?
7) Overall, how would your describe [site name's] involvement in BUILD?
26

OMB #0925-XXXX
EXP. XX/XXXX

a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to successfully
implement this programming? If so, what?
b) Are there any questions I haven't asked, or are there additional points you would like to raise that we have
not yet discussed?

27

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
BUILD Faculty – Leads/Co-Leads (d)

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured interview protocol contains a list of questions to be drawn from in interviews with individuals participating in the
BUILD Case Study. As a flexible framework, probes may be added or omitted from the interview in response to participant feedback.
Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
1. Please provide an overall summary of the BUILD activities your institution is currently involved in
implementing.
2. Please describe why your institution became partners with [BUILD site] and how this partnership was
established.
3. How would you describe the URM sub-population you are targeting through BUILD and their unique needs as
a URM sub-population?
4. How would you say BUILD activities "fit" into the larger vision of [site name's] work in bio-medical research?

Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is currently implementing--

the processes of implementation, some of the challenges you may have encountered, as well as areas you continue to strengthen.

First I'd like to start with faculty development and engagement in BUILD programming, then student participation and engagement,
then participation in mentoring activities, as well as the quality of mentoring activities. I'll be sure to leave time at the end of our
conversation for issues or topics you would like to raise that I might not have asked about. Does that sound all right?
5. Please describe what, if any, BUILD activities you are currently involved in implementing to enhance faculty
development and engagement in bio-medical research training for URMs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in BUILD
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits faculty engagement and participation
in BUILD programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing bio-medical research training for URMs what, in your perspective, faculty
development needs are the highest priority at this time?
e) Please describe any pathways of communication with faculty participating in BUILD.
i) What feedback, if any, have you received from participating faculty?
6. Please describe what, if any, BUILD activities you are currently involved in implementing to enhance student
participant and engagement in bio-medical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
28

OMB #0925-XXXX
EXP. XX/XXXX

b)
c)
d)
e)

i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student participation and engagement in BUILD
programming?
What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
From your perspective, what about your current programming elicits student engagement and participation
in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
In terms of enhancing bio-medical research training for URMs what, in your perspective, student needs are
the highest priority at this time?
Please describe any pathways of communication with students participating in BUILD.
i) What feedback, if any, have you received from participating students?

7. Please describe what BUILD activities you are currently involved in implementing to enhance participation in
mentoring activities, both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve participation in mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement and
participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
8. Please describe what BUILD activities you are currently involved in implementing to improve the quality of
mentoring activities, both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve the quality of mentoring activities?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring activities?
9. How would you describe [site name's] involvement with NRMN?
a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to BUILD’s capacity to advance URM
bio-medical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URM bio-medical research training?
10. In what ways do you see your work under this BUILD partnership contributing to the advancement of URM
bio-medical research training?
a) In what ways, if any, do you believe your institution has benefited from this BUILD partnership?
b) What challenges have you faced in establishing this partnership, if any?
c) What challenges, if any, might this partnership bring to your institution?
11. Overall, how would your describe [site name's] involvement in BUILD?
29

OMB #0925-XXXX
EXP. XX/XXXX

a) Do you feel that additional supports, systems, or structures could enhance [site name's] ability to successfully
implement this programming? If so, what?
b) Are there any questions I haven't asked, or are there additional points you would like to raise that we have
not yet discussed?

30

GROUP INTERVIEW PROTOCOL
BUILD Faculty Participants (e)

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured focus group protocol contains a list of possible questions to be drawn from in interviews with individuals
participating in the BUILD Case Study. As a flexible framework, probes may be added or omitted from the interview in response to
participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
Understanding current BUILD implementation
1) Please introduce yourself by name, your position within [site name], your role in BUILD programming, and what
you most hope to achieve as a result of BUILD participation.
2) In view of the goals and objectives just raised, what kind of progress do you think [site name] has made towards
realizing those objectives?
a) In your opinion, what are the structures, systems, or processes that have supported this progress?
b) In your opinion, what are some of the challenges or hurdles you have come across in program
implementation and facilitation?
c) In your opinion, what are some of the needed supports, systems, or processes that would greater assist
BUILD implementation and facilitation?
Understanding URG participation and engagement
3) How would you characterize the URG student population at [site name]?
a) To what extent do you feel URG’s are already engaged in bio-medical research training?
b) In your view, what are the greatest needs of this sub-population in terms of their enhanced participation in
bio-medical research training?
c) In your opinion, to what extent do you feel BUILD programming is addressing these needs?
d) What additional structures, systems and supports need to be in place to further engage URG’s in biomedical research training?
Understanding faculty participation and engagement
4) In terms of supporting bio-medical research training for URGs, what are the greatest areas of need for faculty
development?
a) To what extent have faculty participated in professional development targeted towards the engagement of
URG’s in bio-medical research training prior to BUILD?
b) To what extent has participation in BUILD supported faculty development and engagement in URG biomedical research training?
i) In your opinion, what are the most essential links between faculty development and then, the translation
of this training into strengthened engagement of URG’s in bio-medical research training?
c) What additional structures, systems and supports need to be in place to further engage faculty in the training
of URG’s in bio-medical research through BUILD?
Understanding BUILD mentoring
5) To what extent do you feel faculty and URG’s are engaged in BUILD mentoring activities?
31

OMB #0925-XXXX
EXP. XX/XXXX

a) What might some of the structures, systems, and supports in place that incentivize participation?
b) What might be some of the challenges that exist in getting greater numbers of URG’s and faculty involved
in BUILD mentoring activities?
6) If you have participated in mentoring training, how would you describe the quality of BUILD mentoring
training you have received?
a) What could strengthen your preparation as a mentor?
b) What are some of the challenges you have come across as a mentor?
7) How would you describe the quality of mentoring URG’s currently receive?
a) What factors either support, or present challenges to improved mentoring quality?

Understanding BUILD Partner Sites
8) To what extent have you been involved in any activities with institutional partner sites [provide a site name as an
example]? If applicable, please describe your involvement.
a) In your opinion, has participation in partner site activities been beneficial to you? Please provide detail as to
why, or why not.
b) What, if any, aspects of partner site activities have been particularly valuable to you?
c) What, if any, aspects of partner site activities could be further improved to support you?
9) To what extent have BUILD with partner institutions contributed to the development of the BUILD program
at this site?
a) How have partner institution activities been beneficial to the overall BUILD program development?
b) What, if any, aspects of partner activities have been particularly valuable to URG faculty and students?
c) What, if any, aspects of partner activities could be further improved to support URG faculty and students?
Understanding NRMN
10) To what extent have you been involved in any NRMN activities? If applicable, please describe your
involvement.
a) In your opinion, has participation in NRMN activities been beneficial to you as a mentor to URG’s? Please
provide detail as to why, or why not.
b) What, if any, aspects of NRMN activities have been particularly valuable to you as a faculty mentor?
c) What, if any, aspects of NRMN activities could be further improved to support you as a faculty mentor?
11) To what extent have you been involved in BUILD activities in association with partner institutions? If
applicable, please describe your involvement.
a) In your opinion, has participation in partner activities been beneficial to you as a mentor to URG’s? Please
provide detail as to why, or why not.
b) What, if any, aspects of partner activities have been particularly valuable to you as a faculty mentor?
c) What, if any, aspects of partner activities could be further improved to support you as a faculty mentor?

32

GROUP INTERVIEW PROTOCOL
BUILD Undergrad (f) and Grad/Post-Doc (g) Participants

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured focus group protocol contains a list of possible questions to be drawn from in interviews with individuals
participating in the BUILD Case Study. As a flexible framework, probes may be added or omitted from the interview in response to
participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
Understanding BUILD participating and engagement
1) Please introduce yourself by name, your current area of study, and what inspired your interest in pursuing biomedical research?
2) How did you find out about BUILD programming?
3) Why did you decide to participate in BUILD programming?
4) To what extent are in you involved in BUILD programming? (What activities do you participate in, and how
involved in those activities are you?)
a) To what extent are in you specifically involved in bio-medical research? (What activities do you participate
in, and how involved in those activities are you?)
Participant feedback on overall BUILD programming
5) In your opinion, has participation in these activities been beneficial to you as [title]? If so, how? If not, why not?
a) In particular, what (if any) opportunities or experiences has BUILD participation exposed you to that you
think you might not otherwise have experienced?
b) What might improve your experiences as a BUILD participant?
i) In particular, are there systems, structures, or processes that could further improve your participation in
BUILD activities?
ii) Are there particular skill areas you feel could be further addressed by BUILD programming?
Participant feedback on BUILD URG focus
6) As a URG what, in your opinion, are some of the greatest challenges you face in your success as a bio-medical
researcher?
7) In what ways do feel that your personal needs as an URG are sufficiently addressed through BUILD
programming, or in what ways do you feel that they could be more effectively addressed?
8) In what ways do you feel BUILD programming has, or has not acknowledged and respected your cultural, racial
and/or gender background?
9) In what ways do you feel that BUILD faculty are, or are not well prepared to support you as an URG [title] in
bio-medical research?
a) What, if any, additional structures, systems and supports need to be in place to further engage students in
the training of URG’s in bio-medical research through BUILD?
i) What has been the most valuable about this experience?
ii) How can this experience be strengthened or improved?
Participant feedback on mentoring
33

OMB #0925-XXXX
EXP. XX/XXXX

10) Do you have a faculty mentor and how long you have been working with them?
a) How did you initially connect with them?
11) How often would you say you meet with your mentor, and for how long?
12) What do you discuss in mentoring sessions with your faculty mentor?
13) In what ways do you feel that having a faculty mentor has been beneficial to you as a [title] in bio-medical
research?
14) In what ways, if any, do you feel that your mentoring experience could be improved to better support you as a
bio-medical researcher?
15) In what ways, if any, do you feel that your faculty mentor could be better prepared to support you as a URG
[title] in bio-medical research?
Reflection
16) Do you plan to pursue a career in bio-medical research?
a) What are the motivating factors behind this decision?
b) What are the greatest challenges URGs face in pursing a career in bio-medical research?
c) If not a career in bio-medical research, then what do you aspire to do? How has the BUILD program
supported this decision and success in this area?

Understanding NRMN and Partnerships
17) To what extent have you been involved in any NRMN activities? If applicable, please describe your
involvement.
a) In your opinion, has participation in NRMN activities been beneficial to you as a [title]? Please provide detail
as to why, or why not.
b) What, if any, aspects of NRMN activities have been particularly valuable to you as a [title]?
c) What, if any, aspects of NRMN activities could be further improved to support you as a [title]?
Understanding Partnerships
18) To what extent have you been involved in BUILD activities in association with partner institutions? If
applicable, please describe your involvement.
a) In your opinion, has participation in partner activities been beneficial to you as a [title]? Please provide detail
as to why, or why not.
b) What, if any, aspects of partner activities have been particularly valuable to you as a [title]?
c) What, if any, aspects of partner activities could be further improved to support you as a [title]?
Reflection
19) Would you recommend BUILD participation to other URG [title]? Why or why not?
20) What, in your opinion, would further incentivize additional [title] to participate in BUILD programming?
21) What, if any, barriers to participation exist amongst URG [title]?

34

SEMI-STRUCTURED INDIVIDUAL INTERVIEW PROTOCOL
Comparison Group – Institutional Director/Program Manager (h)

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured interview protocol contains a list of possible questions to be drawn from in interviews with individuals participating
in the BUILD Case Study Comparison Group. As a flexible framework, probes may be added or omitted from the interview in
response to participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
Understanding current engagement with biomedical students and junior faculty
1) Please introduce yourself by name and your position within [site name]?
2) How would you describe your institutions level of involvement with undergraduate/graduate/post-doctoral
students in biomedical research?
a) Please provide an overall summary of the activities training program initiatives currently being implemented
at your site.
b) How would you describe the URG sub-population you are targeting through BUILD and their unique needs
as a URG sub-population?
c) You might think about the number and types of classes you taught in the field of biomedical research,
advisory roles with students in biomedical research, mentoring roles with student in biomedical research,
and/or specialized programs, training, or support roles with students in biomedical research. [Probe for:
specific examples, numbers of students involved, time/intensity of activities]
3) How would you describe your institutions level of involvement with junior faculty in biomedical research?
a) What, if any, type of advisory or mentoring support do you offer junior faculty, either formally or
informally? [Probe for specific examples, time/intensity of activities]
4) What, if any, strategies have been implemented at [site name] in an effort to expand URG participation in biomedical research?
5) How would you say these activities "fit" into the larger vision of [site name's] work in bio-medical research?

Interviewer: I would like to talk to you in detail about some of the major BUILD activities [site name] is currently implementing-the processes of implementation, some of the challenges you may have encountered, as well as areas you continue to strengthen.
First I'd like to start with faculty development and engagement in BUILD programming, then student participation and engagement,
then participation in mentoring activities, as well as the quality of mentoring activities. I'll be sure to leave time at the end of our
conversation for issues or topics you would like to raise that I might not have asked about. Does that sound all right?
6) Please describe what activities are currently being implemented to enhance faculty development and engagement
in bio-medical research training for URGs.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
35

OMB #0925-XXXX
EXP. XX/XXXX

c) From your perspective, what about your current programming elicits faculty engagement and participation
in programming?
i) What impediments might exist to greater faculty engagement and participation?
d) In terms of enhancing bio-medical research training for URGs what, in your perspective, faculty
development needs are the highest priority at this time?
e) Please describe pathways of communication with faculty participants.
i) What feedback, if any, have you received from participating faculty?

7) Please describe what BUILD activities are currently being implemented to enhance student participant and
engagement in bio-medical research.
a) How might this overall structure differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve student development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student engagement and participation
in BUILD programming?
i) What impediments might exist to greater student engagement and participation?
d) In terms of enhancing bio-medical research training for URGs what, in your perspective, student needs are
the highest priority at this time?
e) Please describe pathways of communication with students participants.
i) What feedback, if any, have you received from participating students?
8) Please describe what activities are currently being implemented to enhance participation in mentoring activities,
both for students and faculty.
a) How might these strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what about your current programming elicits student and faculty engagement and
participation in mentoring?
i) What impediments might exist to greater student or faculty engagement and participation?
9) Please describe what activities are currently being implemented to improve the quality of mentoring activities,
both for students and faculty.
a) How might these overall strategies differ from your original proposal/plan, if at all?
i) If there are deviations from the original proposal/plan, why were these changes made?
ii) How are these revised strategies expected to improve faculty development and engagement in
programming?
b) What challenges have you faced in implementation, if any?
i) How have you addressed those challenges?
c) From your perspective, what should be the priority areas of focus of mentor training and development?
i) What impediments might exist to more quality faculty and student engagement and participation in
mentoring?
d) What feedback, if any, have you received from students and faculty participating in mentoring activities?
10) How would you characterize the nature of [site name's] involvement with NRMN?
36

OMB #0925-XXXX
EXP. XX/XXXX

a) Which NRMN activities do your faculty and students take part in, and to what extent?
b) How, if it all, do you feel that NRMN programming has contributed to [site name's] capacity to advance URG
bio-medical research training?
c) In what ways, if any, could engagement with NRMN be improved to better support the advancement of
URG bio-medical research training?
11) Please describe the ways in which [site name] has been working with other partner organizations.
a) In what ways do you see your work under these partnerships contributing to the advancement of URG biomedical research training?
b) What challenges have you faced in building these partnerships, if any?
c) What challenges, if any, might these partnership bring to the implementation as you had envisioned?
5) Overall, how would your characterize [site name's] involvement in undergraduate training programs?
a. Do you feel that additional supports, systems, or structures could enhance [site name's] ability to
successfully implement this programming? If so, what?
b. Are there any questions I haven't asked, or are there additional points you would like to raise that we
have not yet discussed?

37

GROUP INTERVIEW PROTOCOL
Comparison Group – Faculty (i)

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured focus group protocol contains a list of questions to be drawn from in interviews with individuals participating in the
BUILD Case Study comparison group. As a flexible framework, probes may be added or omitted from the interview in response to
participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
Understanding current engagement with biomedical students and junior faculty
1) Please introduce yourself by name and your position within [site name]?
2) How would you describe your level of involvement with undergraduate/graduate/post-doctoral students in
biomedical research?
a. You might think about the number and types of classes you teach in the field of biomedical research,
advisory roles you may have with students in biomedical research, mentoring roles you may have with
student in biomedical research, and/or specialized programs, training, or support roles you may have with
students in biomedical research. [Probe for: specific examples, numbers of students involved, time/intensity
of activities]
3) How would you describe your level of involvement with junior faculty in biomedical research?
a. What, if any, type of advisory or mentoring support do you offer junior faculty, either formally or
informally? [Probe for specific examples, time/intensity of activities]
Understanding URG participation and engagement in biomedical research
4) How would you characterize the URG student population at [site name]?
a. To what extent do you feel URG’s are engaged in bio-medical research training?
b. In your view, what are the greatest needs of these sub-populations in terms of their enhanced participation
in bio-medical research training?
c. In your opinion, to what extent do you feel current programming at [site name] is addressing these needs?
d. What additional structures, systems and supports need to be in place to further engage URG’s in biomedical research training?
Understanding faculty development needs in URG biomedical research training
5) In terms of supporting bio-medical research training for URGs, what might be some areas for faculty
development?
a. To what extent have faculty participated in professional development targeted towards the engagement of
URG’s in bio-medical research training?
b. Would you say there is an expressed need by faculty for this type of training?
c. What additional structures, systems and supports would need to be in place to further engage faculty in the
training of URG’s in bio-medical research?
Understanding faculty mentoring activities
6) To what extent do you feel faculty are engaged in mentoring activities?
a. What might some of the structures, systems, and supports in place that incentivize participation?
38

OMB #0925-XXXX
EXP. XX/XXXX

b. What might be some of the challenges that exist in getting greater numbers of faculty involved in mentoring
activities?
c. To what extent do you feel faculty are engaged in mentoring activities with biomedical students/postdocs/junior faculty from URG?
7) How would you describe the quality of mentoring biomedical students/post-docs/junior faculty currently
receive?
a. What factors either support, or present challenges to improved mentoring quality?

Reflection
8) In general, do you see your URG biomedical students pursuing careers in bio-medical research? [Also: Do you
have access to this information?]
a. What do you think are the motivating factors behind their decisions?
b. What do you think are the greatest challenges URGs face in pursing a career in bio-medical research?
9) In what ways do you feel students are prepared to pursue your career interests as a result of their research and
training experiences at [site name]?
a. Are there particular research, or specific skill areas in which you feel your program could provide additional
training/guidance to bolster student success in pursuing careers in biomedical research?
10) How would you describe what it means to be successful as an undergraduate/graduate/post-doc student in the
biomedical sciences at [site name]?
11) How would you describe what it means to be successful as a junior faculty member in the biomedical sciences at
[site name]?

39

GROUP INTERVIEW PROTOCOL
Comparison Group – Undergrad (j) Grad/Post-Doc (k) Students

OMB #0925-XXXX
EXP. XX/XXXX

This semi-structured focus group protocol contains a list of questions to be drawn from in interviews with individuals participating in the
BUILD Case Study comparison group. As a flexible framework, probes may be added or omitted from the interview in response to
participant feedback. Interview questions, however, are expected to stay within the content areas detailed below.
Public reporting burden for this collection of information is estimated to average 90 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor,
and a person is not required to respond to, a collection of information unless it displays a currently valid
OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge
Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-xxxx*). Do not return the completed form to
this address.
Understanding participation and engagement in biomedical research experiences
1) Please introduce yourself by name, your current area of study, and what inspired your interest in pursuing this
particular academic area?
2) How would you describe your opportunities to engage in research as a student at [site name]?
a) What activities come to mind as being part of your research experience here? [Probe for: specific examples,
time/intensity of research activities]
i) Outside of your standard coursework and labs, have there been additional opportunities to engage in
biomedical research, for example, through supplementary programs, workshops, or internships?
(1) If so, please describe your involvement in these activities. [Probe for: specific examples,
time/intensity of research activities]
(2) How would you describe you interest in engaging in these types of activities? (i.e., Have you pursued
involvement in these activities – why or why not?)
(3) What barriers, if any, might have challenged your participation in these activities?
b) How would you describe the value of your research experience at [site name] to your academic career, or
other career interests you may have?
c) What could be improved about your research experience at [site name]?
i) In particular, are there systems, structures, or processes that could be put in place to further improve
your research experience as a student?
d) In your experience, have you come across any barriers to participating in biomedical research during your
time at [site name]?
i) In answering, please describe how you have come to this conclusion.
Understanding participation and engagement in biomedical mentoring experiences
3) How would you describe your opportunities to be mentored by faculty as a student at [site name]?
a) In particular, would you say you have a mentor at [site name]?
i) If so, how did you connect with your mentor?
(1) In your experience, what were some of the important factors that facilitated this connection?
(2) How would you describe your own personal interest in finding a mentor or in being mentored?
ii) If not, have opportunities arisen for you to connect with a mentor?
(1) How would you describe your own personal interest in finding a mentor or in being mentored?
(2) What challenges might you have experienced in connecting with a mentor?
b) Whether you consider yourself as having a designated mentor or not, what activities come to mind as being
part of your mentoring experience as a student at [site name]? [Probe for: specific examples, time/intensity of
mentoring, who is doing the mentoring, what activities mentoring encompasses]
40

OMB #0925-XXXX
EXP. XX/XXXX

c) How would you describe the value of your current mentoring experience at [site name] to your academic
career, or other career interests you may have?
d) What could be improved about your current mentoring experience at [site name]?
i) In particular, are there systems, structures, or processes that could be put in place to further improve
your mentoring experience as a student?

Experiences as a URG biomedical student
4) As a biomedical student from an URG, what, in your opinion, are some of the greatest challenges you face in
your success as a bio-medical researcher?
5) In what ways do feel that your personal needs as a student from an URG are sufficiently addressed through
your current academic program, or in what ways do you feel that they could be more effectively addressed?
6) In what ways do you feel your current academic program has, or has not acknowledged and respected your
cultural, racial and/or gender background?
7) In what ways do you feel that faculty are, or are not well prepared to support you as an URG [title] in biomedical research?
a) What, if any, additional structures, systems and supports need to be in place to further engage URG
students in bio-medical research?
8) In what ways, if any, do you feel that faculty you interact with could be better prepared to support you as a
URG [title] in bio-medical research?
Reflection
9) Do you plan to pursue a career in bio-medical research?
a) What are the motivating factors behind your decision?
b) What are the greatest challenges URGs face in pursing a career in bio-medical research?
c) If not a career in bio-medical research, then what do you aspire to do?
10) In what ways do you feel prepared, or feel that you will be prepared, to pursue your career interests as a result of
your research and training experiences at [site name]?
a) Are there particular research, or specific skill areas in which you feel your could use additional
training/guidance outside of what your current academic program provides?
11) How would you describe what it means to be successful in your current academic program?

41

OMB #0925-XXXX
EXP. XX/XXXX

BUILD CASE STUDY GENERAL OPEN-ENDED OBSERVATION PROTOCOL

Semi-structured observation will be conducted during case study data collection visits. Participating BUILD
institutions will be asked to prepare in advance for the visiting research team to observe BUILD-specific activities in
progress, such as BUILD program development meetings, mentoring sessions, or lessons in which new curriculum
is currently in use. Because BUILD activities vary extensively by site, the case study research team will work in
close coordination with the BUILD site visit monitoring team and each BUILD institution to ensure appropriate,
practical activities are selected for observation. Participants will include faculty and staff charged with implementing
BUILD at the primary and partner sites as well as students involved in BUILD activities.
The purpose of observation in this context is to gain a contextualized understanding of the ways in which different
groups of BUILD stakeholders (i.e., program leadership, program implementers, and partners) interact and
collaboratively strategize to advance BUILD’s overarching objective of increasing URG participation and
engagement in bio-medical research. While this can often be a dynamic, fluid process wherein observable details
may frequently change, or which may be spontaneously revealed, there are several key areas of interest that are
considered most relevant to this research inquiry. These are as follows:
Context
 Persons/roles present
 Stated purpose/objectives of the meeting/activity being observed
 Type of activity being observed (planned and as it is actually implemented)
 Questions, answers, comments raised (and by who)
Discussion and dialogue amongst those present:
 Capacity of the BUILD Prime institution to successfully implement BUILD activities
o Successes and challenges in implementation
o Benefits and drawbacks of partner collaboration in implementation
o Negotiation of human, financial, time resource allocations
 Goal-setting and progress monitoring
o Planning implementation timelines
o Achievement of, and challenges in achieving scheduled milestones
o Strategic planning to achieve BUILD program objectives
o Perceived stakeholder accountability to implementation partners
o Perceived stakeholder accountability to evaluation theory of success (e.g., output, intermediate
outcomes, and long-term outcomes)
 Development of a shared vision among key stakeholders (admin, faculty at and among BUILD
partner sites)
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
 Development, dissemination and implementation of curriculum and pedagogy
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
 Enacting policy
o Resource and support needs identified by various BUILD stakeholders
o Response to, and consideration of expressed needs by implementing partners
42

OMB #0925-XXXX
EXP. XX/XXXX






o Perspective and opinions that take precedence in determining implications for action
Identifying and resolving barriers to effective collaboration across program activities
o Identification of systemic, structural, procedural weaknesses in links among BUILD stakeholders
o Actionable next steps developed to resolve identified issues
o Evidence of follow-up and implementation of action items
Systems, structures, and processes required to promote BUILD partnership sustainability
o Identification of factors promoting long-term BUILD partnership sustainability
o Resource allocation (human, financial, time) in support of sustainability needs
Data-informed decision-making
o Data sources presented
o Perceived reliability/quality of data presented
o How discussion and dialogue is facilitated (and by who) around data interpretation
o What and how data are used to inform next steps
o Perspectives and opinions that take precedence in determining implications for action
o Expressed data needs

43

BUILD CASE STUDY – COMPARISON GROUP –
GENERAL OPEN-ENDED OBSERVATION PROTOCOL

OMB #0925-XXXX
EXP. XX/XXXX

Semi-structured observation will be conducted during case study data collection visits. Participating non-BUILD
institutions will be asked to prepare in advance for the visiting research team to observe specific activities in
progress, such as program development meetings, mentoring sessions, or lessons in which new curriculum is
currently in use. Because activities vary extensively by site, the case study research team will work in close
coordination with the site visit monitoring team and each non-BUILD institution to ensure appropriate, practical
activities are selected for observation. Participants will include faculty and staff charged with implementing
undergraduate/graduate training programs.
The purpose of observation in this context is to gain a contextualized understanding of the ways in which different
groups of stakeholders (i.e., program leadership, program implementers, and partners) interact and collaboratively
strategize to advance URG participation and engagement in bio-medical research. While this can often be a
dynamic, fluid process wherein observable details may frequently change, or which may be spontaneously revealed,
there are several key areas of interest that are considered most relevant to this research inquiry. These are as follows:
Context
 Persons/roles present
 Stated purpose/objectives of the meeting/activity being observed
 Type of activity being observed (planned and as it is actually implemented)
 Questions, answers, comments raised (and by who)
Discussion and dialogue amongst those present:
 Capacity of the institution to successfully implement activities
o Successes and challenges in implementation
o Benefits and drawbacks of partner collaboration in implementation
o Negotiation of human, financial, time resource allocations
 Goal-setting and progress monitoring
o Planning implementation timelines
o Achievement of, and challenges in achieving scheduled milestones
o Strategic planning to achieve program objectives
o Perceived stakeholder accountability to implementation partners
o Perceived stakeholder accountability to evaluation theory of success (e.g., output, intermediate
outcomes, and long-term outcomes)
 Development of a shared vision among key stakeholders (admin, faculty, students)
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
 Development, dissemination and implementation of curriculum and pedagogy
o Alignment of goals and objectives
o Alignment of implementation timelines
o Alignment of program resources (human and financial)
o Investments in establishing/sustaining partnerships
 Enacting policy
o Resource and support needs identified by various stakeholders
o Response to, and consideration of expressed needs by implementing partners
o Perspective and opinions that take precedence in determining implications for action
44






OMB #0925-XXXX
EXP. XX/XXXX

Identifying and resolving barriers to effective collaboration across program activities
o Identification of systemic, structural, procedural weaknesses in links among stakeholders
o Actionable next steps developed to resolve identified issues
o Evidence of follow-up and implementation of action items
Systems, structures, and processes required to promote partnership sustainability
o Identification of factors promoting long-term partnership sustainability
o Resource allocation (human, financial, time) in support of sustainability needs
Data-informed decision-making
o Data sources presented
o Perceived reliability/quality of data presented
o How discussion and dialogue is facilitated (and by who) around data interpretation
o What and how data are used to inform next steps
o Perspectives and opinions that take precedence in determining implications for action
o Expressed data needs

45