IGERT Study Cover Memo

IGERT Study
Cover Memo

October 4, 2011

Prepared by:
Beth Gamse, Project Director
Kristen Neishi
Amanda Parsad
Radha Roy

Overview of Proposed IGERT Study: How are IGERT projects preparing
interdisciplinary researchers?
Program Purpose
Since 1998, the National Science Foundation (NSF) has supported interdisciplinary training of
doctoral students across the nation through the Integrative Graduate Education and Research
Traineeships (IGERT) Program. NSF awards five-year grants to higher education institutions to
develop innovative, interdisciplinary doctoral training programs in STEM disciplines, reflecting
NSF’s commitment to high quality research that increasingly demands interdiscipinary
approaches. The IGERT program has three broad goals: (1) to educate Ph.D.-level scientists with
the depth and breadth of knowledge and skills to become leaders in their fields; (2) to catalyze
changes in graduate education by establishing models for collaborative research across
disciplinary boundaries; and (3) to promote diversity among participating students and the
professional science and engineering workforce.
Each IGERT project is overseen by one or more Principal Investigators (PIs) who are faculty
members from various departments and/or disciplines both within and across institutions. The
faculty members from each IGERT project develop a series of education and research activities
in which students and faculty from multiple departments participate. These activities are
organized to support an interdisciplinary theme, and include a combination of multidisciplinary
research collaborations, cross-departmental lab rotations, interdisciplinary seminars, teamtaught courses, and/or off-campus internships, among others. Students from multiple
disciplines/departments related to the project’s interdisciplinary theme are recruited by faculty
members to participate in the program. Most IGERT trainees are enrolled in a single-discipline
Ph.D. program and participate in IGERT activities in addition to their department activities;
however, some projects develop a new, interdisciplinary degree program for students.
Generally, IGERT trainees complete all the requirements of their home department (or discipline
within a department) as well as the requirements of the interdisciplinary IGERT project.
Trainees receive a graduate stipend of $30,000 and a cost of education allowance of $10,500 per
year (12 months). On average, trainees participate in the program for two years. Since 1998,
the IGERT program has made 260 awards to over 100 lead universities, providing funding for
more than 5,800 graduate students.

Rationale for the study
As interdisciplinary science becomes more common, and the demand for scientists who have
the skills to conduct interdisciplinary research grows, understanding how to prepare
interdisciplinary researchers becomes increasingly salient. The IGERT program represents a large
scale investment in preparing graduate students to become interdisciplinary researchers.1 Prior
1

For the purpose of brevity, “interdisciplinary researchers” are used to represent “interdisciplinary
scientists and engineers” throughout this document.
Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 1

studies of the IGERT program have examined multiple aspects of that preparation, including
project-level activities, project implementation, early career outcomes of recent IGERT
graduates, and IGERT project Principal Investigators’ perceptions of institutional change. NSF
has examined the IGERT program using both annual monitoring data as well as targeted
evaluation studies that addressed implementation (Chase and Carney, 2001; Chase et al., 2002;
Martinez et al., 2006), initial impacts (Carney et al., 2006), and follow-up of IGERT graduates
(Carney et al., 2010). However, these studies have not examined a central element of the
program: how interdisciplinarity itself is defined and operationalized across IGERT projects.
Understanding interdisciplinarity, particularly in terms of how to prepare scientists with the
skills needed in an increasingly interdisciplinary research environment is increasingly salient. The
current study will focus on the IGERT program’s first broad goal: to prepare Ph.D. students to
conduct interdisciplinary research. The study will describe how IGERT projects design, provide,
and experience interdisciplinary graduate education.
Prior research about interdisciplinary training is also limited. Some researchers acknowledge the
challenges associated with preparing interdisciplinary researchers within institutions and
departments that are discipline-focused (Coppola, Banaszak Holl, & Karbstein, 2007; Feller,
2006); some researchers have examined context-specific interdisciplinary research models
within specific laboratory or disciplinary settings (e.g., Lattuca and Knight, 2010; Nersessian,
2009), while others point to a dearth of empirical research about learning outcomes, methods,
or benchmarks for interdisciplinary learning, especially in science and other technical fields
(Aboelela et al., 2007; Boix Mansilla, 2006; Borrego & Newswander, 2010; Jacobs & Frickel,
2009; Schilling, 2001; Van Hartesfeldt & Giordan, 2008).
Two recent studies conceptualized likely outcomes of interdisciplinary education. Lattuca and
Knight (2010) reviewed the engineering education and higher education literature, and from
that review, developed a working definition of interdisciplinary competence. Such
interdisciplinary competence is defined as one’s ability to understand and utilize knowledge and
modes of inquiry drawn from disciplines other than one’s own, and that understanding and use
of knowledge includes the following skills: a) an appreciation of various disciplinary
perspectives; b) an ability to incorporate and evaluate multiple disciplinary approaches to
problem-solving; c) an ability to recognize the strengths or weaknesses of one's own disciplinary
perspective; and d) an ability to recognize the shared assumptions, skills, or knowledge among
disciplines. Borrego and Newswander (2010) conducted an analysis of peer-reviewed literature
from interdisciplinary studies in the humanities and social sciences fields and reviewed
information from 129 funded IGERT proposals, and from that analysis, identified five similar
categories of learning outcomes for interdisciplinary education, including: a) disciplinary
grounding; b) integration; c) teamwork; d) communication; and e) critical awareness.
Building on the interdisciplinary skills and learning outcomes described in the literature above,
as well as feedback from the study’s Evaluation Advisory Committee (EAC),2 the study team

2

The EAC includes: Monica Cox – Director, Pedagogical Evaluation Laboratory and Associate Professor of
Engineering/Purdue University; Irwin Feller – Professor Emeritus of Economics/Pennsylvania State
Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 2

identified the following knowledge, skills and abilities as important in preparing students to
conduct interdisciplinary research:
1. Depth of knowledge in one discipline or field of study
2. Ability to recognize the strengths and weaknesses of multiple disciplines
3. Ability to apply the approaches and tools from multiple disciplines to address a research
problem
4. Ability to work in a team with individuals trained in different disciplines
5. Ability to communicate research based in one discipline or field of study to academic
researchers trained in different disciplines
6. Ability to communicate about interdisciplinary research to non-academic audiences
(laypersons)
The study will collect data from IGERT PIs and from currently enrolled IGERT trainees, both
those currently funded as well as those who had been funded in prior years (and who are still
enrolled in their graduate programs) to learn whether and how they perceive the knowledge,
skills, and abilities listed above as important to conducting interdisciplinary research. Further,
PIs will be asked how their projects are designed to develop trainees in these areas. The results
will provide the program staff with an understanding of program participants’ perceptions of the
importance of these skills to conducting interdisciplinary research, whether and how projects
provide training to develop trainees’ skills in these six areas, the frequencies with which various
approaches are used, and the successes and challenges reported by the study’s IGERT
participants.
Research questions
The study will address the following research questions:
1. Whether and in what ways do IGERT participants (PIs and trainees) perceive the knowledge,
skills, or abilities drawn from the literature as important to conducting interdisciplinary
research?
2. What activities do projects implement to develop trainees’ interdisciplinary research
capacity, as characterized by these knowledge, skills, or abilities? How do projects assess
trainees’ development as interdisciplinary scientists?
3. How helpful do trainees perceive their IGERT training to be in developing their capacity to
conduct interdisciplinary research as characterized by these six areas?
4. How confident are IGERT trainees of their knowledge, skills, and abilities in these six areas?
5. What challenges do trainees encounter with the IGERT traineeship?
Exhibit 1 below summarizes the study’s research questions and data collection sources.

University; Lisa Lattuca – Professor of Higher Education/University of Michigan; and Nancy Nersessian –
Regents’ Professor of Cognitive Science/Georgia Institute of Technology.
Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 3

Exhibit 1: Research Questions by Data Sources
Data Sources
Primary Data Collection
Research Question/Topics
IGERT PIs
(Interview)

IGERT Trainees
(Survey)

Secondary
(Extant) Data
Distance
Monitoring
Data

1. Whether and in what ways do IGERT participants (PIs and trainees) perceive the knowledge, skills or
abilities drawn from the literature as important to conducting interdisciplinary research?
IGERT trainees’ perception of the importance of the
knowledge, skills or abilities to conducting
interdisciplinary research



IGERT trainees’ perception of the importance of the
knowledge, skills or abilities to conducting research
in one discipline or field of study



IGERT trainees’ perception of other areas that are
important to conducting interdisciplinary research



IGERT PI's perception of the importance of the
knowledge, skills or abilities to conducting
interdisciplinary research



2. What activities do projects implement to develop trainees’ interdisciplinary research capacity, as
characterized by these knowledge, skills or abilities? How do projects assess trainees’ development as
interdisciplinary scientists?
Role of the different IGERT training activities in
developing trainees' capacity in the six areas





IGERT faculty’s assessment of the development of
trainees' interdisciplinary research capacity





3. How helpful do trainees perceive their IGERT training to be in developing their capacity to conduct
interdisciplinary research as characterized by these six areas?
Perceived helpfulness of IGERT training in
developing IGERT trainees' capacity in the six areas



Other knowledge, skills or abilities trainees report
that their IGERT training helps to develop



4. How confident are IGERT trainees of their knowledge, skills, and abilities in these six areas?
IGERT trainees’ perceptions of their confidence in
these areas



5. What challenges do trainees encounter with the IGERT traineeship?
Perceived challenges to participating in IGERT



Proposed IGERT Study Summary Prepared by Abt Associates August 2011





Page 4

Research design
The study will collect data from a purposive sample of active IGERT projects and their
participants (PIs and trainees). The study plans to focus specifically on all 40 projects initially
funded in the 2007 and 2008 cohorts. These projects meet the following criteria:





Provide representation from each directorate that supports IGERT projects
Have been operational long enough to have moved beyond initial implementation
Have trainees at different points along the graduate education trajectory, and
Are housed at a variety of higher education institutions

One potential drawback of this approach is that the study findings would not be representative
of the full set of IGERT projects; rather, the findings would only be applicable within a particular
set of projects funded over a defined three to four year time period. However, the study is not
seeking to generalize to all projects, but, as an exploratory and descriptive study, is designed to
learn whether and how a subset of projects define and operationalize interdisciplinary training
in ways consistent with an emerging literature on interdisciplinary graduate education.
Research data collection
The study will collect data in the 2011-12 academic year from two respondent groups: project
PIs and current and formerly funded IGERT trainees who are still enrolled in their doctoral
program. As there are no prior studies of interdisciplinary graduate education that apply across
diverse disciplinary boundaries and that ask about the six skill areas described above, the study
has developed new instruments. The study design and instruments have been reviewed by an
Evaluation Advisory Committee (EAC) comprised of experts in graduate STEM education,
interdisciplinary research in science and engineering, and evaluation of higher education STEM
programs.
The EAC provided input on the instruments to ensure that they are appropriate for the study’s
research questions. Specifically, the EAC reviewed the overall design as well the draft
instruments and individual items to ensure that individual items are designed to measure
underlying constructs (construct validity) and that the items (and overall instruments) have
adequate face validity for respondents who participate in IGERT projects. The EAC members
also reviewed the study’s data collection plan and strategies to ensure high response rates.
Taken together, the EAC comments have informed the study’s overall design, instruments, data
collection and analysis plan, and therefore serve to enhance the ultimate reliability of study
findings. Additionally, both the PI interview and the survey have been and will continue to be
pilot tested to ensure clarity of language and concepts, logical sequencing of questions and
items, appropriate skip patterns, and ease of navigation (for online surveys). The draft
instruments reflect feedback from the EAC and information obtained to date from pilot testing.

Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 5

Principal Investigator Interviews: The PIs will have first-hand (and current) knowledge of
how and why they have designed their training programs to prepare trainees with the
skills they believe are essential for interdisciplinary research. The 40 PIs from the 2007
and 2008 cohorts of projects will be interviewed using a semi-structured interview
protocol so that the study can collect detailed and potentially project-specific
information on project activities, decision-making processes, and assessment of
trainees’ development of interdisciplinary skills.
Survey of Currently Enrolled IGERT Trainees: The IGERT trainees will have first-hand
experience acquiring skills that their respective projects believe are those required of
interdisciplinary scientists. The study will field an online survey to all currently enrolled
IGERT trainees from the 2007 and 2008 projects. IGERT projects typically support 5 to
10 trainees each year over their five-year grant periods. It is important to note that
while projects generally support trainees for some, not all, of their graduate education,
projects typically identify all enrolled students as IGERT trainees, whether the students
are currently receiving financial support or not. Therefore, it is estimated that each
project from these two cohorts will have approximately 17-20 currently enrolled
trainees for an estimated total of 750 trainees who will be invited to participate in the
survey. The study anticipates an 80 percent response rate, based on recent experience
conducting studies of other NSF fellowship programs.
Monitoring data
The study will rely upon the Distance Monitoring System (DMS) data to learn about the sample
projects. The DMS is operated by ICF Macro, a survey research and information technology
company. Each year, participating projects submit information about numbers of participants (at
institution, department, faculty, and student levels), nature/frequency of activities, and selected
project accomplishments. Study researchers have reviewed DMS data to inform the
development of study instruments (interview protocols and trainee survey), and will continue to
use the data to prepare for interviews with individual PIs.
Methodology/instrumentation
Because monitoring system data about projects’ activities and participants already exist and will
have been reviewed prior to data collection, the study can more efficiently conduct interviews
and field surveys, as the study team will already have obtained and reviewed relevant
background information. The study team did not find existing survey items or databases from
which to select survey items, thus new instruments have been developed for this study. Given
that monitoring system data about projects’ activities and participants already exist and will
have been reviewed prior to data collection, the study can more efficiently conduct interviews
and field surveys, as the study team will already have obtained and reviewed relevant
background information.

Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 6



The PIs will be interviewed using a semi-structured interview protocol so that the study
can collect detailed and potentially project-specific information about such topics as:
PIs’ initial reasons for applying for IGERT funding; how and why project activities help
prepare trainees to work in an interdisciplinary environment; how PIs recruit and retain
students who can succeed in an interdisciplinary context; and PIs’ descriptions of
challenges and corresponding strategies in their host departments and institutions. The
interviews will take 45 minutes to complete. After developing the final interview
protocols, all interviewers will be trained on the interview protocol to standardize
questions and prompts across interviewers and interviews. Only Abt staff members
with working knowledge of the IGERT program will conduct these semi-structured
interviews. Interviews will be scheduled using contact information obtained from the
PIs. Interviewees will be assured that information they provide will not be released in
any form that identifies them as individuals and their responses will be kept
confidential. Interviews will be tape-recorded so that notes can be captured and
analyzed using a combination of Microsoft Access and NVivo software packages. The
study will use well-established qualitative data analytic techniques to analyze responses
in terms of commonalities across projects, institutions, and interdisciplinary clusters or
themes. Given that the (interview) sample size is below 50, responses will be reported
in terms of numbers of respondents rather than percentages.



The study will field an online survey to an estimated 750 IGERT trainees. The survey will
include questions about which skills trainees report they are developing to become
interdisciplinary researchers in their respective contexts, how their respective programs
develop/enhance such skills, the specific IGERT activities they perceive as important to
the development of their capacity to conduct interdisciplinary research, and the
challenges they have experienced in developing interdisciplinary skills. The survey will
take approximately 20-25 minutes to complete. Survey responses will be analyzed using
descriptive statistics, and will be presented in terms of aggregate frequencies and
proportions, and, if possible, in terms of common patterns of responses across projects,
institutions, and for trainees, in terms of stage of graduate training.

Limitations
There are three major limitations of the proposed study. First, there is no one commonlyaccepted definition of interdisciplinarity, so the dimensions/traits being explored in the study
are necessarily exploratory. A second limitation is that student outcomes are self-reported by
the trainees—they will be asked about their perceived confidence in skills across these areas to
conduct interdisciplinary research. The study is not measuring these skills directly. However,
given that there is no commonly accepted definition of interdisciplinarity, nor a commonly
accepted set of outcomes, a study that describes perceptions of importance of skills,
preparation in these skill areas, and means by which trainees acquire capabilities in these areas
can be useful.
Expected Contributions

Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 7

The study will provide information to the IGERT program staff about how projects operationalize
interdisciplinary graduate training, specifically whether and how projects provide training to
develop trainees’ skills in the six areas hypothesized as important to conducting interdisciplinary
research, the frequencies with which various approaches are used, and the successes and
challenges reported by the study’s IGERT participants. The results of the study can also inform
the design of a more rigorous study that can assess whether and how specific training activities
are associated with the acquisition of identified skills.

References
Aboelela, S.W., Larson,E., Bakken, S., Carrasquillo, O., Formicola, A., Glied, S.A., Haas, J., and
Gebbie, K.M. (2007). Defining Interdisciplinary Research: Conclusions from a Critical Review of
the Literature. Health Services Research 42: 1, Part 1 (February), 329-346. Published by Health
Research and Educational Trust.
Boix Mansilla, V. (2006). Quality assessment of interdisciplinary research: Toward empirically
grounded validation criteria. Research Evaluation 15 (1), April, 69–74.
Borrego, M., Newswander, L. (2010). Definitions of Interdisciplinary Research: Toward GraduateLevel Interdisciplinary Learning Outcomes. The Review of Higher Education, 34(1), 61-84.
Carney, J.G., Chawla, D., Wiley, A., & Young, D. (2006). Evaluation of the Initial Impacts of the
National Science Foundation’s Integrative Graduate Education and Research Traineeships
(IGERT) Program. Prepared for the National Science Foundation.
Carney, J.G., Martinez, A., Dreier, J., Neishi, K., Parsad, A. (2010). Evaluation of the National
Science Foundation’s Integrative Graduate Education and Research Traineeships (IGERT)
Program: Follow-up Study of IGERT Graduates. Prepared for the National Science Foundation.
Chase, A. & Giancola, J. (2001). NSF Integrative Graduate Education and Research Traineeships
Monitoring Report: Boston University, The Bioinformatics Project. Prepared for the National
Science Foundation.
Chase, A., Giancola, J., Smith, C., Boulay, B., Gamse, B., Horst, L., Moss, M., Goldsmith, S.,
Haviland, D., Tushnet, N. (2002). IGERT Annual Cross-Site Report: 1998 Cohort. Prepared for the
National Science Foundation.
Coppola, B..P., Banaszak Holl, M.M., Karbstein, K. (2007). Closing the Gap between
Interdisciplinary Research and Disciplinary Teaching. American Chemical Society/Chemical
Biology, 2(8), 518-520.
Dauphine´e, D., & Martin, J.B. (2000). Breaking Down the Walls: Thoughts on the Scholarship of
Integration. Academic Medicine 75 (9) , 881-886.
Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 8

Feller, I. (2006). Multiple actors, multiple settings, multiple criteria: issues in assessing
interdisciplinary research. Research evaluation 15 (1), 5—15.
Jacobs, J., Frickel, S. (2009). Interdisciplinarity: A Critical Assessment. Annual Review of
Sociology. 35, 43-65.
Lattuca, L., Knight, D. (2010). In the Eye of the Beholder: Defining and Studying Interdisciplinarity
in Engineering Education. American Society for Engineering Education.
Martinez, A., Chase, A., Boulay, B., Chawla, D., Layzer, C., Litin, L., Zotov, N. (2006). Contractor
Annual Report and Summary of the Cross-Site Monitoring of the NSF Integrative Graduate and
Education Research Traineeship (IGERT) Program. Prepared for the National Science Foundation.
Nersessian, N.J. (2009). How do engineering scientists think? Model-based simulation in
biomedical engineering laboratories, Topics in Cognitive Science, 1:730-757.
Schilling, K. L. (2001). Interdisciplinary assessment for interdisciplinary programs. In
B. L. Smith & J. McCann (Eds.), Reinventing ourselves: Interdisciplinary education,
collaborative learning and experimentation in higher education (pp. 344–54). Bolton,
MA: Anker.
Van Hartesveldt, C., & Giordan, J. (2008). Impact of Transformative Interdisciplinary Research
and Graduate Education on Academic Institutions: National Science Foundation/Education and
Human Resources Directorate/Division of Graduate Education, Integrative Graduate Education
and Research Training Program Workshop Report (May).

Proposed IGERT Study Summary Prepared by Abt Associates August 2011

Page 9