0990-Training Ph D s FacultyViews325minusInstrument422

Contract No.: 233-02-0086

MPR Reference No.: 6294-300

Training Ph.D.s: Faculty Views on Their Role and Their Institution’s Role to Promote the Development of Responsible Researchers

Supporting Statement for Request for OMB Approval

April 22, 2008

Janice Ballou

Julie Ladinsky

Brian Roff

Submitted to: Office of Research Integrity U.S. Department of Health and Human Services 1101 Wootton Parkway Rockville, MD 20852 Project Officer: Sandra Titus

Submitted by: Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Janice Ballou

CONTENTS

Page

SUPPORTING STATEMENT FOR TRAINING PH.D.S: FACULTY VIEWS

ON THEIR ROLE AND THEIR INSTITUTION’S ROLE TO PROMOTE

THE DEVELOPMENT OF RESPONSIBLE RESEARCHERS 1

A. JUSTIFICATION 1

A.1. Overview of Need and Legal Basis 1

A.2. Purpose and Use of Information Collection 3

A.3. Use of Technology to Reduce Burden and Improve Data Collection 9

A.4. Efforts to Avoid Duplication and Use of Similar Information 10

A.5. Small Business 10

A.6. Consequences to Collecting the Information Less Frequently 10

A.7. Special Circumstances 11

A.8. Federal Register Notice and Outside Consultation 11

A.9. Payment or Gift to Respondents 12

A.10. Confidentiality of the Data 12

A.11. Additional Justification for Sensitive Questions 13

A.12. Burden Estimate (Total Hours & Wages) 13

A.13. Estimate of Total Annual Cost Burden to Respondents 15

A.14. Estimates of Annualized Cost to Federal Government 15

A.15. Reasons for Program Changes or Adjustments 15

A.16. Tabulations, Publication Plans, and Time Schedules 15

A.17. Approval Not to Display the Expiration Date for OMB Approval 17

A.18. Exception to the Certification Statement 17

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 19

B.1. Respondent Universe and Sampling Methods 19

B.2. Procedures for the Collection of Information 20

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 23

B.4. Tests of Procedures or Methods to Be Undertaken 24

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 24

REFERENCES 26

CONTENTS (continued)

APPENDIX A: FEDERAL REGISTER MAY 12, 2000 NOTICE

APPENDIX B: WEB WELCOME

QUESTIONNAIRE

FREQUENTLY ASKED QUESTIONS

APPENDIX C: EMAIL INVITATION HERE

EMAIL REMINDER HERE

REMINDER PHONE CALL SCRIPT HERE

APPENDIX D: DRAFT FEDERAL REGISTER NOTICE

SUPPORTING STATEMENT FOR TRAINING PH.D.S: FACULTY VIEWS

ON THEIR ROLE AND THEIR INSTITUTION’S ROLE TO PROMOTE

THE DEVELOPMENT OF RESPONSIBLE RESEARCHERS

A. JUSTIFICATION

A.1. Overview of Need and Legal Basis

The Office of Research Integrity (ORI) is conducting this study on research mentors and advisors because faculty who serve as mentors and advisors are in a pivotal position to promote the development of young scientists’ research skills in a responsible and ethical manner. The influence of faculty is believed to be critical to promoting research integrity and preventing research misconduct. This study is designed to learn what advisors and mentors believe their role is in educating doctoral students to conduct responsible research. Up to this time, there has been an assertion of the importance of the roles of mentors and advisors, but there have been no focused studies that demonstrate what they do to promote research integrity.

In 2000, the Division of Education and Integrity (DEI) at ORI was directed to “focus more on preventing misconduct and promoting research integrity through expanded education programs.” Specifically, DEI was directed to “conduct policy analyses, evaluations, and research to improve DHHS research integrity and build the knowledge base in research misconduct, research integrity and prevention;” (Federal Register: May 12, 2000 [Volume 65, Number 93]) (See Appendix A.)

In 2002, the Institute of Medicine (IOM) issued a report on integrity in scientific research. It advocates that “institutions should develop a multifaceted approach to promoting integrity in research appropriate to their research environments.” The IOM report also states that mentoring is the key to producing responsible researchers. Thus, we believe that institutional commitment to promoting research mentoring is a critical way to foster quality research and prevent misconduct.

In 2005, a research report by Martinson et al. demonstrated that National Institutes of Health (NIH) scientists report that they engage in behaviors that others consider to be poor practices and are currently termed “questionable research practices.” Specifically, the authors found that 15 percent of NIH scientists reported that they drop observation points and 27.5 percent reported that they have inadequate record keeping. In addition, the revised regulations on research misconduct (42 CFR93.516) added a clause indicating that the researcher has the burden of proof when the research records are deficient. Specifically, it states,

“The destruction, absence of, or respondent’s failure to provide research records adequately documenting the questioned research is evidence of research misconduct where ORI establishes by a preponderance of the evidence that the respondent intentionally, knowingly, or recklessly had research records and destroyed them, had the opportunity to maintain the records but did not do so, or maintained the records and failed to produce them in a timely manner and the respondent’s conduct constitutes a significant departure from accepted practices of the relevant research community.”

Thus, research groups that are lax about their rules for data documentation and retention are potentially training future scientists in a sloppy and deficient mode of conducting research, which is now considered to be an appropriate factor in determining research misconduct. Hence, rules about data documentation and retention are critical. This is one component of training doctoral students to conduct responsible research rules that we will be examining in our study.

We need to understand more about who is training new Ph.D. candidates in the responsible conduct of research. Do faculty advisors and mentors rely on the short Responsible Conduct of Research (RCR) training programs that are often created to fulfill the NIH training grant requirement? Are there different responsibilities for advisors and mentors? Who is in charge of the training? What responsibilities do faculty members perceive they and their institutions have for training doctoral students in the responsible conduct of research?

To focus our research efforts on training Ph.D. candidates, we have three key research questions:

1. How do faculty members perceive or define the roles of mentors and advisors?

2. What practices or activities do faculty members actually engage in to help doctoral students achieve successful outcomes?

3. How do universities promote or support mentoring and advising by faculty members?

42 U.S.C. § 289b Establishment of Office of Research Integrity, created to handle research misconduct. Subsequent to that statute, 42 CFR parts 50 and 93 directed ORI to handle research misconduct and also promote research integrity activities.

A.2. Purpose and Use of Information Collection

This study has been designed to gather information for ORI to use for conferences, workshops, and publications, and in the development of other training materials. The specific findings can be used to facilitate the improvement and promotion of best practices for doctoral student research training, such as guidelines for faculty activities and responsibilities in the development of responsible researchers. To the best of our knowledge, the data collected will provide the most comprehensive account thus far of faculty activities related to the development of doctoral students. We think it will be very valuable information which can be used to engage the educational community in a dialogue about something that appears to be taken for granted. It appears to be widely assumed that advisors and mentors know what their role is in relation to training students to be successful and conscientious researchers. We want to encourage a dialogue about that role and how to strengthen it. Faculty have enormous time constraints and cutting corners is common. Hence, it also seems likely that institutions must become more involved and foster the climate that leads to the development of sound scientific practices. We want to engage mentors, advisors, and institutions in a conversation about the processes used to educate doctoral students.

a. Overview of What Is Currently Known About Mentoring

Mentoring and advising comprise a set of relationships and activities between students and faculty that form in a university context to guide students toward successful outcomes. While mentoring is a product of the interactions among student, faculty, and institution, the existing research on mentoring has generally examined mentoring from the student perspective. The literature indicates that students perceive mentoring as highly critical to completing their graduate programs (Hartnett 1976; Blackwell 1987; Arce and Manning 1984). The study by Nettles and Millet (2006), based on student reports, looks at the level of incidence and nature of mentoring within universities. Little is known, however, about how faculty define the roles of advisor and mentor and how faculty members perform these roles in their daily work life. Furthermore, research on the institution’s role in promoting mentoring and advising is lacking.

b. Advisor versus Mentor

The role of faculty in preparing doctoral students is generally characterized in two ways: (1) an advisor provides formal links between the student and the institution regarding meeting academic requirements; or (2) a mentor provides professional development in areas not immediately pertinent to the curriculum or the dissertation, such as interpretation and presentation of research findings (Natural Academy of Sciences 1997). Little consensus exists on the use of the terms advisor and mentor, and a given faculty member may serve one or both roles (Nettles and Millet 2006). Yet, the way in which faculty members view these roles greatly shapes the nature of the mentoring and advising support they provide to doctoral students.

c. Conceptual Framework for the Study

As shown in Figure 1, Box A, we suggest that the roles of mentor and advisor can be defined with respect to three dimensions: (1) importance of the role, (2) objectives of the role,

FIGURE 1

A CONCEPTUAL MODEL TO UNDERSTAND HOW FACULTY MEMBERS

VIEW THEIR ROLE AND their INSTITUTION’S ROLE

in PROMOTing THE DEVELOPMENT OF

RESPONSIBLE RESEARCHERS

A. Faculty Views of Mentoring and Advising Roles

Importance of role

· Importance of role in achieving student

outcomes

∙ Importance of mentoring and advising

relative to other roles

Objectives – which student outcomes to

pursue

Implementation of role

· Responsibilities

· Qualifications for mentor and advisor

∙ Characteristics of mentee/advisee

· How mentor/advisor and student should be

matched

· Nature of interpersonal relationship (respect,

access)

B. Institutional Context

Importance of role

· Importance of role in achieving student

outcomes

∙ Importance of mentoring and advising

relative to other roles of faculty

Objectives – which student outcomes to

pursue

Policies

· Extent of formalization of policy (e.g.,

guidelines)

· Types of mentor and advisor training programs

provided

· Process to evaluate mentors and advisors

∙ Incentives for mentors and advisors

· System to identify mentors and advisors

· System to match mentors/advisors with students

· Policies to handle mentoring/advising problems

C. Faculty Mentoring and Advising Practices

· Number of students advising

· Number of students mentoring

· Number of students advising and mentoring

· How matching of mentor/advisor and

mentee/advisee works

· Type of activities implemented to help students

achieve outcomes

∙ Time commitment

D. Student Outcomes

· Graduate from doctoral program

· Acquire skills to identify research questions,

consider alternative explanations, develop

study design, maintain good data, and analyze

data

· Interpret research

· Publish and disseminate research

· Develop professional network

∙ Find a job with potential

· Obtain research grants

· Be knowledgeable about research ethics and

standards

and (3) implementation of the role. Faculty members may differ in how instrumental they consider the mentor and advisor to be in the overall development of successful researchers. It may be valuable to understand how faculty members assess the importance of the mentor and advisor roles relative to the importance of the other roles they play. Although the primary objective of mentoring and advising is to help students achieve successful outcomes (Box D), faculty may believe mentors and advisors seek different student outcomes. For example, the National Academy of Sciences (NAS) definitions of mentor and advisor might imply that an advisor is concerned with completion of the dissertation, whereas the mentor may help the student find publishing opportunities.

Implementation of mentoring and advising is based upon expectations regarding: (1) the responsibilities of these roles, (2) the qualifications needed to be a mentor or an advisor, (3) the characteristics of the mentor or advise, (4) the process for matching mentors/advisors and doctoral students, and (5) the nature of the social interaction with the doctoral student. Faculty members are likely to have different beliefs about the qualifications of a mentor, such as the level of research experience and personal characteristics needed to play this role. Guidelines for mentoring typically list substantial research knowledge and experience and communication skills as necessary qualifications for an effective mentor. Some faculty members may prefer to mentor a certain type of student; for example, they may believe that, for the relationship to be effective, the mentor and student should have a similar work ethic or research interest.

Our research questions link directly to the model. Our first question, “How do faculty members perceive or define the roles of mentors and advisors?,” corresponds to Box A. We are interested in learning about faculty views of the importance and objectives of their roles and how they go about implementing their roles. Our second question, “What practices or activities do faculty members actually engage in to help doctoral students achieve successful outcomes?,” relates to Box C of the model. The practices or activities a faculty member engages in are shaped by the number of students he/she is advising or mentoring, by how faculty members are matched with students, and by how much time the faculty member is able to invest in mentoring. All of these factors impact student outcomes. Box B in the model corresponds to our third research question, “How do universities promote or support mentoring and advising by faculty members?” The value universities place on faculty advising or mentoring activities, identification of outcomes that are the most important for faculty and students to pursue, and the policies in place to foster the faculty/student relationship are institutional characteristics that contribute to successful student outcomes.

d. Overview of Data Collection Methodology

“Training Ph.D.s: Faculty Views on Their Role and Their Institution’s Role in Promoting the Development of Responsible Researchers” will be a web survey of a random sample of faculty members who have received National Institutes of Health (NIH) grant funding. To determine their eligibility to participate in the survey, the sampled faculty will be asked several screening questions to identify those who have had responsibilities for doctoral students in the past five years.

e. Analysis

The analysis will be both descriptive and relational. The sample will include the following key segments: (1) medical schools that are stand-alone institutions and part of a larger academic institution, and (2) all other academic institutions. The descriptive analysis will identify topics such as faculty perceptions of mentor and advisor responsibilities, faculty practices, and policies within academic institutions. The relational analysis will identify the strength of the relationships among faculty views, faculty practices, and institutional views and policies. Analyses identifying the relationship between specific elements of the institutional context and faculty practice can provide information to develop best practices. This type of analysis can address important questions, such as: Do faculty members at universities that provide training programs offer a greater level and range of mentoring support and assistance to doctoral students? Other characteristics such as faculty demographics, their views on mentoring and advising, and their professional priorities will inform how these individual attributes relate to mentoring and advising.

Survey data will be used to inform the analysis of our research questions. The specific sections of our proposed web survey (Attachment B) relate to the three research questions as follows:

1. How do faculty members perceive or define the roles of mentors and advisors?

Sections: A. Faculty Roles and F. Faculty Responsibilities

2. What practices or activities do faculty members actually engage in to help doctoral students achieve successful outcomes?

Sections: B. Your Doctoral Students, C. Student Outcomes, E. Professional Activities, and F. Faculty Responsibilities

3. How do universities promote or support mentoring and advising by faculty members?

Section D. Institutions, Departments, and Programs

The questionnaire also includes items to provide background information such as faculty rank, tenure status, country of origin, age, race, and gender.

f. Report

After data collection ends, we will prepare a descriptive report outlining the survey methodology and key findings. A data file without any individual identifiers, along with supporting technical and methodological information, will be available for public use from the ORI website. We plan to actively identify opportunities for publishing and disseminating the survey results among the broader scientific community. Peer-reviewed journals, such as Accountability in Research, will be considered for publishing the research. The project team includes a research integrity expert who will be a lead author in publications using data from this study.

A.3. Use of Technology to Reduce Burden and Improve Data Collection

The survey will be conducted with a sample of 10,000 faculty members, using a web-based survey instrument. The web address for the questionnaire and the unique user ID and password for access will be provided to sampled faculty members in an email invitation to participate in the survey. The web questionnaire will be designed for ease of access and use. The visual format will emphasize readability. It will feature skip patterns that allow respondents to bypass non-applicable sections and will primarily use close-ended question structures that minimize respondent burden and facilitate the coding and analysis of responses. The design will maximize respondent usability while incorporating web features to minimize the rate of missing data. Also, the technology will be used to “read in” information from prior questions that informs questions that follow. To address the issue of different browsers and download speeds, we will test the web version thoroughly with several commonly used browsers. The web questionnaire will also be tested to verify proper navigation through the survey instrument. To facilitate response, we will include easy-to-use links to help screens and completion instructions, as well as an email help address and an 800 telephone number for respondents to call with questions about the web survey. The survey will be designed so that, if respondents cannot complete the survey at one time, their answers will be saved and they can access the survey later to complete it. Email reminders will be sent only to nonresponders, about every four days. In addition, telephone reminder calls to nonresponders will be used as needed to encourage participation.

A.4. Efforts to Avoid Duplication and Use of Similar Information

To the best of our knowledge, the proposed data collection effort has never been done before and there is not a similar set of data. A literature review was conducted to identify research on faculty roles to promote the development of responsible Ph.D.s. We found no reports on faculty roles that were directly relevant to this study. One study linked reduction of graduate school attrition to mentoring (Ph.D. Completion Project 2006) and an increase in student productivity (Cronan-Hillis et al 1986; Nettles and Millet 2006) The existing research on mentoring and advising has been primarily conducted from the student perspective. The literature indicates that students perceive mentoring as highly critical to completing their graduate programs (Hartnett 1976; Blackwell 1987; Arce and Manning 1984). The study by Nettles and Millet (2006), based on student reports, looks at the level of incidence and nature of mentoring within universities. Little is known, however, about how faculty define the roles of advisor and mentor, how faculty members perform these roles, and how they relate to the responsible conduct of research. Furthermore, research on the institution’s role in promoting and supporting the responsible conduct of research through faculty mentoring and advising is lacking. ORI is interested in filling critical information gaps related to the development of doctoral students.

A.5. Small Business

No small businesses will be involved in this study.

A.6. Consequences to Collecting the Information Less Frequently

The information collection is only planned for one time and has never been collected before. Without documenting faculty members’ roles and their perception of their institutions’ roles, little will be known about their participation in the scientific training of doctoral students. This survey will provide information to guide improved mentoring by identifying the relationships between faculty, students, and institutions that guide students to successful outcomes.

A.7. Special Circumstances

There are no special circumstances involved with this data collection.

A.8. Federal Register Notice and Outside Consultation

a. Federal Register Announcement

A 60-day Federal Register Notice was published in the Federal Register on December 12, 2007, vol. 72, No238; pp. 70597. One response was submitted to the OMB posting:

From: McNair, Tia B. [mailto:[email protected]]
Sent: Thursday, February 14, 2008 9:26 AM
To: FunnColeman, Sherette (HHS/ASRT)
Cc: Marrett, Cora
Subject: Comments on OMB No. 0990-New--ORI Training Ph.Ds
Importance: High

As requested, below are comments on the proposed study Training Ph.D.s:  Faculty Views on Their Role and Their Institution’s Role to Promote the Development of Responsible Researchers (OMB No. 0990-New—ORI).

The Federal Register Notice (FR Doc E7-24055) states that the proposed study will focus on collecting descriptive information from faculty members about their roles as advisor and mentor and how they carry out these advising and mentoring functions in their daily work with PhD candidates.  In addition, faculty members will be asked to describe the involvement of their institutions in promoting training or otherwise supporting research mentoring and advising.  The faculty members for the survey are to be recipients of grants in 2005 and 2006 from the National Institutes of Health (NIH) or the National Science Foundation (NSF) who have supervised doctoral students in the last five years. The Division of Education and Integrity (DEI) at the Office of Research Integrity (ORI) intends to use the results from the survey to build the knowledge base on research integrity.   

 

An interest in the effective mentoring of graduate extends to more Federal agencies than NIH and NSF.  For this reason, the subject of the study would be an appropriate matter for discussion by the Subcommittee on Education of the National Science and Technology Council (NSTC).  With co-chairs from the National Science Foundation, the Department of Education, and the National Institutes of Health, the Subcommittee brings together Federal agencies to address interests they share.  Those interests include the status of graduate education and of graduate students in science, technology, engineering and mathematics.  The request: permit the Subcommittee to review and discuss the planned study before it is submitted formally to the Office of Management and Budget for clearance.  The discussion could take account of requirements in the America COMPETES Act and other regulations that might affect aspects of the study design.

If you have any questions, please do not hesitate to contact me.

Sincerely,

Cora B. Marrett

Assistant Director

Education and Human Resources

National Science Foundation

Co-Chair of the NSTC Committee on Science

Subcommittee on Education

[email protected]

703-292-8600

Response to Above E-mail:

From: Titus, Sandra

Sent: Thursday, February 27, 2008 4:28 PM
To: [email protected];[email protected]

Subject: Re: OMB No. 0990-New--ORI Training Ph.Ds

Dear Dr. McNair and Dr. Marrett,

Thank you for your very kind interest in this study.  We greatly appreciate your proposed suggestions.  We agree that collaboration between our agencies has enormous potential for enriching the public trust in this important area of scientific advancement.

Subsequent to the posting of the project, our staff discerned a further need to limit the scope of work of the effort. This refinement, along with others we have decided, is essential so as to guarantee reasonable levels of success given the mission of our agency.  To this end we have decided to concentrate our efforts on NIH recipients alone. Since our project will only be limited to the NIH proportion of the original scope of work, collaborations on this specific project at this time with agencies extramural to DHHS will unfortunately not be possible.

However, your proposal for mutual collaboration is an extremely important and enthusiastic invitation regarding all of the present and emerging standards for the responsible conduct of research. The positive benefits from the COMPETES act and the ongoing leadership of ORI in promoting the responsible conduct of research have potential for promoting in a unified fashion a culture of integrity in the national research climate. While our collaborations on this specific project are not feasible for work effort efficiency due to our more refined scope of work, we would look forward very much to collaborative opportunities and inter-agency efforts as the future unfolds.

As I said to you in my conversation with you today, if you have any questions regarding the above, or wish to discuss further any specific aspects of our project, please contact me per below. At the same time we may be able to engage in preliminary discussions regarding future possible initiatives of mutual interest and benefit.

Very truly yours,

Sandra Titus, Ph. D.

Director, Intramural Research

240-453-8437

Larry Rhoades, Ph.D.

Director, DEI

Office of Research Integrity

OSOPHS, OS, DHHS

b. Consultation with Individuals Outside the Agency

During the preparation of the survey design and questionnaire, the main consultant was Dr. Francis L. Macrina, Ph.D., Vice President for Research at Virginia Commonwealth University (VCU) and an expert in academic mentoring and in training new scientists. Dr. Macrina’s phone number is 804-827-2262. Nationally recognized experts in sample design, data collection, and survey instrument design, professional statisticians, and survey researchers from MPR worked with ORI. MPR’s main contacts were Janice Ballou, (a senior fellow and nationally recognized survey research expert who has more than 30 years of experience conducting in-person, mail, telephone, and web-based surveys), and Frank Potter, (a senior fellow who specializes in the design and implementation of probability surveys and the implementation of statistical tasks such as weight adjustment, missing data imputation procedures, and data analysis). MPR drafted the survey instrument and developed the sample design. The consultation with both Dr. Macrina and MPR began in October 2006 and will continue until the project is completed.

c. Unresolved Issues

None.

A.9. Payment or Gift to Respondents

No payment or gift will be given to respondents who fill out the survey instrument “Training Ph.D.s: Faculty Views on their Role and their Institution’s Role in Promoting the Development of Responsible Researchers.”

In the development of the questionnaire, nine faculty members filled out the questionnaire and participated in the pilot testing about why they answered as they did. They were each paid $75 for their participation.

A.10. Confidentiality of the Data

The study will be conducted in accordance with all relevant regulations and requirements, including the Privacy Act of 1974 (5 USC 552a), the Privacy Act Regulations (34 CFR Part 5b), and the Freedom of Information Act (5 CFR 552) and related regulations (41 CFR Part 1-1, 45 CFR Part 5b, and 40 CFR 44502). In addition, the project will adhere to the guidelines outlined in the MPR Security Manual. Faculty members will be assigned a unique user ID and password to be able to access the web survey. MPR has rigorous requirements in place to protect the security of the information provided over the internet. The project programmer will inspect the database to ensure that data are stored properly, secure on the web server, and within MPR’s firewalls. The Institutional Review Board (IRB) submission reviewed by Public/Private Ventures approved these procedures. The IRB panel approved the project and data collection in May 2007. The data collected in this survey are not sensitive in nature. In addition, all of the data reported or available for public use will only be in the aggregate and no individuals will be identified.

A.11. Additional Justification for Sensitive Questions

Overall, none of the questions included in the survey instrument are commonly considered to be sensitive. The questionnaire does not request personal data that is not generally available to the public. However, some questions about activities related to responsibilities for doctoral student training may be perceived to be sensitive. Faculty members typically make information about their professional activities available on university websites. However, some faculty members may consider some of the questionnaire items sensitive.

One section of the questionnaire asks respondents about their professional background, including the number of extramural grants or contracts that fund their work and their current total amount of grant funding. Other questions ask faculty members about rewards related to their responsibilities for doctoral students. This information is necessary to determine if there is any relationship between faculty members’ mentoring roles and whether or not they perceive that their contributions to the development of responsible researchers are acknowledged. All other questionnaire items deal with descriptive and basic attitudinal information regarding mentor and advisor roles, faculty responsibilities, institutional and departmental policies and culture, and student outcomes.

A.12. Burden Estimate (Total Hours & Wages)

Burden will result from the proposed data collection for the 10,000 potential faculty respondents who are invited to participate one time in this web survey. From the initial sample size of 10,000 faculty, based on studies of graduate students who report mentors, it is expected that about 66 percent (6,600) will be eligible to participate in the survey because they have had primary responsibility for overseeing a doctoral student in the last five years. Among the 6,600 who are eligible, there is an expected 70 percent response rate for a total of 4,620 respondents.

Average burden per respondent will not vary because there is only one questionnaire. The average completion time is expected to be 20 minutes per questionnaire. These estimates are based primarily on the pilot testing with nine faculty members conducted in advance of the survey, which used an instrument similar in length and complexity to the one proposed for the final survey. The pilot testing suggests that some faculty members may take longer if they consult records to obtain information about doctoral students, and some may take less time if they are not eligible to respond to questions about students who received their doctorates in the past five years. But overall, we anticipate that 20 minutes is a reasonable estimate of time burden.

12A.

Table 1

RESPONDENTS AND BURDEN ESTIMATES FOR

THE TRAINING PH.D.S SURVEY

Instrument	Respondents	Response Time	Total Time
Faculty Survey	4,620 faculty who oversee doctoral students*	20/60 minutes	1,540 hours

* Of the original 10,000 sample members, 66 percent are expected to be eligible and among those who are eligible (6,600), 70 percent are expected to participate for a total of 4,620 respondents.

During the pilot testing, we asked about the content, length, and difficulty of the questionnaire. No negative comments were given about the questionnaire or the amount of time it took to complete. Several respondents commented that the questionnaire was similar in length to other questionnaires they had completed and that they did not see any problems with the amount of time required to complete it. Because of cost considerations, a web version of the questionnaire was not used for the pilot testing. It is expected that the electronic version of the questionnaire with the appropriate read-in information from prior items and automatic skip patterns will further reduce burden and the time for completion.

12.B

Estimate of Cost to Respondents

Type Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Cost
Science Faculty	1,540 hours	$60.00	$92,400

A.13. Estimate of Total Annual Cost Burden to Respondents

There are no capitol costs to individual survey respondents.

A.14. Estimates of Annualized Cost to Federal Government

The estimated annualized cost of administering the “Training Ph.D.s: Faculty Views on Their Roles and Their Institution’s Role in Promoting the Development of Responsible Researchers” is $173,201 (12 months). All the survey costs will be born by the contract between ORI and MPR (Contract No. 233-02-0086). The total cost for this 24-month project is $346,402. In addition, at ORI, the cost of the project officer is $30,000 for two years, which is 15 percent of an annual average salary of $100,000.

A.15. Reasons for Program Changes or Adjustments

This is a new0, one-time data collection plan resulting in a burden increase of an estimated 1540 hours.

A.16. Tabulations, Publication Plans, and Time Schedules

The discussion of tabulation and publication plans focuses on the analyses we will conduct and the reports we will produce. The focus of the analysis will be to address three key questions:

1. How do faculty members perceive or define the roles of mentors and advisors?

2. What practices or activities do faculty members actually engage in to help doctoral students achieve successful outcomes?

3. How do universities promote or support mentoring and advising by faculty members?

a. Tabulation Plans

For this analysis, we propose to conduct both descriptive and relational analyses. The descriptive analysis will identify the different types of perceptions, faculty practices, and organizational policies across the universities and between types of institutions (for example, Ph.D.-only graduate programs; Ph.D./M.D. graduate programs). The relational analysis will seek to identify the strength of the relationships between (1) faculty views, (2) faculty practices, and (3) institutional views and policies. Analyses identifying the relationship between specific elements of the institutional context and faculty practice can provide information to develop best practices. This type of analysis can address important questions, such as: Do faculty members in universities that provide training programs offer a greater level and range of mentoring support and assistance? And, what is the difference between medical school and graduate school research and faculty? In addition to describing the methodology to be used, the analysis will include tables with frequency distributions and cross-tabulations.

Since data will be collected electronically, the tabulations used for the analysis will automatically be checked by the program for inconsistencies and invalid format. A coding frame will be developed and entered into the data file for the qualitative information on the few open-ended questions. After the data have been reviewed and the data file reviewed for quality assurance, a CD containing the data file, codebook with variable descriptions, and descriptive frequency tabulations in SPSS format will be produced. A secure data file—stripped of identifiers—that includes supporting information with labeled data and analysis files will be prepared to offer for public use on the ORI website. A written analysis, along with the supporting paper and electronic data, will discuss the findings, tables, and interpretations of the data.

b. Publication Plans

We will write a report describing the main results from the data analysis and the methodological approach of the study in a readable and easily usable format. It will include an executive summary that will highlight the key findings. The final report will be professionally edited and it will be available in both paper and electronic format. We will also prepare a manuscript for submission to an appropriate peer-reviewed journal, such as Accountability in Research.

c. Schedule

The full timeline for the project is presented in Table 2. Survey design and instrument development are scheduled for October 2006 to December 2007 and data collection will begin in March 2008. The data analysis and report writing will occur between June and October 2008.

A.17. Approval Not to Display the Expiration Date for OMB Approval

Approval not to display the expiration date for OMB approval is not requested.

A.18. Exception to the Certification Statement

There are no exceptions taken to Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-1.

Table 2

Project Schedule

Activity	Schedule
1. Develop Study Design	October 2006
2. Develop Survey Instrument
Conduct literature review	October 2006 to January 2007
Draft survey instrument	November 2006 to May 2007
Pilot testing	June to July 2007
Revised survey instrument	August 2007
3. Pilot Instrument and Submission to OMB
IRB submission and approval	May 2007
OMB submission	November 2007
OMB approval	March 2008
4. Logistical issues to obtain samples
Finalize sampling design for OMB	October 2007
Finalize sampling issues based on NIH list	November 2007
5. Data Collection
Program and test web survey	November to December 2007
Email participation invitations	March 2008
Data collection field period	March to May 2008
6. Data Analysis	June to August 2008
7. Reporting Results
Final report	September to October 2008
Public data use file	October 2008
8. Journal Article	November 2008

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

The primary goal of the sampling design is to obtain a national random sample of NIH 2005 and 2006 grant recipients from publicly available information at each of these organizations. It is expected that these faculty will have graduate students in Ph.D, MD/Ph.D, and MD only programs, and most will be faculty members within natural science, engineering, and medical school departments of degree-granting institutions. There are approximately 14,000 NIH grantees per year. Our target population are grantees who are faculty of academic institutions or medical schools/medical centers who have primary responsibility for overseeing a doctoral student in the past 5 years. Based upon a review of the data from NIH grantees, approximately 75 percent of these grantees are faculty of academic institutions or medical schools/medical centers. A two-way stratification is proposed:

• Faculty belonging to medical schools both stand-alone and those that are part of large academic institutions

• Faculty belonging to all other academic institutions

The names, email addresses, and telephone numbers of 10,000 of these faculty members will be randomly selected within strata ( as indicated in Table 1). Since faculty members can be contacted directly using information from the sampling frame, no cost savings will be realized by clustering faculty members within institutions or any other clustering variable. Hence, individual faculty members can act as Primary Sampling Units (PSUs).

After data collection is complete, we will develop sampling weights to permit expansion of estimates to all faculty in the sampling frame. These sampling weights will be appropriately adjusted for nonresponse. Nonresponse adjustment is necessary to avoid bias in the estimates due to differential nonresponse. Total frequencies such as those for male and female faculty may also be poststratified to totals available in the frame, as deemed necessary after a review of the frequency distributions and finalization of data analysis. Analysis will be performed using software that accommodates the sampling design, such as SUDAAN, so that standard errors are estimated appropriately. The expected response rate using this sample design and the data collection procedures outlined below is 70 percent of those who are eligible because in the last five years they have had primary responsibility for overseeing a doctoral student.

B.2. Procedures for the Collection of Information

a. Statistical Methodology for Stratification and Sample Selection

To select a sample of faculty for the purposes of evaluating mentoring and advising of Ph.D students among those faculty, we will use a list of the NIH grantees in the past two years as a sampling frame. This will be used to identify strata from which the sample will be selected. The stratification variable identifying the type of grant the faculty member received can simply be created by merging the two lists and removing duplicates, awards to postdoctoral fellows, and awards to faculty in non-U.S. institutions. Using the Integrated Postsecondary Education Data System (IPEDS) Dataset Cutting Tool, provided by the National Center for Education Statistics (NCES), universities that contain either medical schools or veterinary schools will be distinguished from those that do not, for the purposes of stratification.

This study is focused on a variety of subdomains. A sample size of 10,000 was chosen to accommodate small subdomains, including, for example, faculty who are employed by medical schools that are not affiliated with a larger academic institution. According to the IPEDS Dataset Cutting Tool, there are only 53 such institutions. Previous studies of graduate students have shown about two-thirds report having a mentor. It seems conceivable that a similar proportion of faculty have mentored graduate students in the past 5 years. With a 66% eligibility expected (only those faculty with a recent graduate student in the past 5 years) along with an anticipated response rate of 70 percent, the original sample size is reduced to 4,620. Of the 1,863 institutions that grant graduate degrees, only 535 of which potentially have doctoral students, of which 53 is less than 10 percent. It is therefore conceivable that faculty from these institutions could constitute 10 percent of the sample. We may also be interested in differences in other subdomains such as between males and females in these schools which could result in a subsample of about 233.

b. Estimation procedure

The plan for the statistical analyses of the data, which is mainly limited to descriptive statistics, is presented in Section A. To summarize, this study attempts to get a snapshot view of the state of mentoring and advising in U.S. universities, using a variety of metrics. SUDAAN will be used to provide the standard error estimates to accommodate the sampling design.

c. Degree of accuracy for the purpose described in the justification

As explained in subsection (a) above, the large sample size is necessary to detect differences in study variables for small subdomains. We have shown how a subdomain with 233 respondents is conceivable for this study. Comparing two subdomains, with equal numbers in each stratum, for a comparison of proportions with 250 in each domain, with 80 percent power and a 5 percent level of significance, it would be possible to detect a difference of proportions of 0.145. This assumes a population size of 20,000.

d. Data Collection Procedures

The survey will be introduced by sending an email invitation to each faculty member who has been sampled (Appendix C). The literature suggests web surveys have a higher response rate when an email with the URL address is sent to the sample member. The email will introduce the study, stress its importance, review confidentiality, and provide a toll-free telephone number and an email address for the study’s help desk. The email will also include information about the web address of the survey (URL), and the user ID and password.

Following up on the initial invitation to participate in the survey, nonresponders will receive up to five email reminders, one every four days (Appendix C). As people complete the survey, the database used to track contacts will automatically delete them from the group scheduled to receive reminders. To address the possibility that spam filters may block the emails, the sender will be identified as the Office of Research Integrity. Emails will be sent on a staggered basis to reduce the possibility of some institutions receiving a large number of email invitations on one day.

Follow-up reminder telephone calls (Appendix C) will be made to sample members who have not responded after the email reminders. As needed to improve the response rate, sample members may also be offered the opportunity to complete the questionnaire with a telephone interviewer.

The data collection methods are designed to reach a targeted 70 percent response rate in a three-month period. This response rate projection is based on prior experience with similar data collection efforts.

The web questionnaire will be designed to maximize respondent usability, while incorporating web features to minimize the rate of missing data. The visual format will emphasize readability. To address the issue of different browsers and download speeds, we will test the web version thoroughly with several commonly used browsers. In addition, the web-based questionnaire will be thoroughly tested to verify proper navigation through the survey instrument. To facilitate response, easy-to-use links to help screens and completion instructions will be included. To ensure that respondents complete critical items, we will incorporate web functionalities, such as the ability to stop respondents from moving to another question without completing the critical item. For respondents’ convenience, the survey will be designed so that, if respondents cannot complete the survey at one time, their answers will be saved and they can access the survey later to complete it. Rigorous requirements will be in place to protect the security of the information over the internet. The project programmer will inspect the database to ensure that data are stored properly, secure on the web server, and within the firewalls.

e. Use of Periodic Data Collection Cycles to Reduce Burden

This survey has a single data collection cycle.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

To maximize the response rate, the email invitation with the URL and the easy-to-complete web questionnaire are designed to encourage participation. As faculty complete the survey, the database used to track contacts will automatically delete them from the group scheduled to receive email reminders. For those who do not respond, there will be follow-up email reminders and, as needed, follow-up reminder telephone calls to attain the projected response rate. Professional, experienced interviewers will make the follow-up calls; they will receive training on the background of the project, information on the sample members being contacted, and the survey instrument. During the final contact, we will offer sample members the opportunity to complete the questionnaire by telephone. We expect to achieve a 70 percent response rate using these methods.

To avoid bias in estimates, sampling weights will be adjusted for nonresponse. It is anticipated that it may not be possible to contact selected faculty, and if they are contacted, some may refuse to respond, or fail to respond for other reasons. Following standard practice, weight adjustments will occur in two steps, with the first adjustment for unlocated sample members, and the second adjustment for uncooperative sample members.

B.4. Tests of Procedures or Methods to Be Undertaken

During the questionnaire development, nine in-person interviews were conducted to pilot test the questionnaire with NIH grantees who have had primary responsibility for overseeing at least one doctoral student’s research within the last five years. These faculty members were employed at several different academic institutions and in various departments. A protocol was developed for the pilot testing that included an assessment of the survey instrument’s overall clarity, wording of specific questions, faculty member’s understanding of the questions and terminology, and their suggestions for improving the survey. Careful testing provides a quality review on instrument wording, skip logic, transitions, and response burden to participants. With the pilot testing methodology, we monitored and debriefed participants to assess comprehension, clarity of instructions, question flow, and organization. The testing was used to provide an estimate of respondent burden for completing the questionnaire.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following people were consulted on the statistical aspects of the study design:

• Sandra Titus, Office of Research Integrity, 240-453-8437

• Janice Ballou, Mathematica Policy Research, Inc., 609-750-4049

• Arthur Bonito, RTI International, 919-541-6377

• Laura Kalb, Mathematica Policy Research Inc., 617-301-8989

• Julie Ladinsky, Mathematica Policy Research Inc., 609-936-2764

• Eric Grau, Mathematica Policy Research Inc., 609-945-3330

• Fran Macrina, Virginia Commonwealth University, 804-827-2262

• Frank Potter, Mathematica Policy Research Inc., 609-936-2799

• Brian Roff, Mathematica Policy Research Inc., 609-750-4041

This group consists of sampling statisticians and survey methodologists who have extensive experience in the design and implementation of surveys. There is also a subject matter expert on the team.

REFERENCES

Arce, C., and W.H. Manning. “Minorities in Academic Careers: The Experience of Ford Foundation Fellows.” New York: Ford Foundation, 1984.

Blackwell, J. Mainstreaming Outsiders: The Production of Black Professionals. Second edition. Dix Hills, NY: General Hall, 1987.

Committee on Assessing Integrity in Research Environments, Board on Health Sciences Policy and Division of Earth and Life Studies, Institute of Medicine, National Research Council of the National Academies: “Integrity in Scientific Research: Creating and Environment That Promotes Responsible Conduct,” The National Academies Press, Washington, DC, 2002.

Council of Graduate Schools, Ph.D. Completion Project, “Ph.D. Completion and Attrition: Numbers, Leadership, and Next Steps,” Washington, DC, 2004.

Cronan-Hillix, T., L.K. Gensheimer, W.A. Cronan-Hillix, and W.S. Davidson. “Students’ Views of Mentors in Psychology Graduate Training.” Teaching of Psychology, vol. 13, 1986, pp. 123–127.

Guston, D.H. “Mentorship and the Research Training Experience.” In Responsible Science, Volume II: Background Papers and Resource Documents. Washington, DC: National Academy of Sciences, 1993. [www.nap.edu/openbook/0309047889/html/50.html]. Accessed June 11, 2006.

Haring-Hidore, M. “Mentoring as a Career Enhancement Strategy for Women.” Journal of Counseling and Development, vol. 66, 1985, pp. 147–148.

Hartnett, R.T. “Environment for Advanced Learning.” In Scholars in the Making: The Development of Graduate and Professional Students, edited by J. Katz and R.T. Hartnett. Cambridge, MA: Ballinger Publishing, 1976, pp. 49–84.

Langlais, P. “Ethics for the Next Generation.” Chronicle of Higher Education, January 13, 2006. [http://chronicle.com/weekly/v52/19/19b01.htm]. Accessed August 15, 2006.

Martinson, B., M. Anderson, and R. de Vries. “Scientists Behaving Badly,” Nature, 435, 737 738 (9 June 2005).

National Academy of Sciences. “Adviser, Teacher, Role Model, Friend: On Being a Mentor to Students in Science and Engineering.” Washington, DC: NAS, 1997. [www.nap.edu/readingroom/books/mentor/1.html]. Accessed June 11, 2006.

Nettles, M., and C. Millet. Three Magic Letters: Getting to Ph.D. Baltimore: Johns Hopkins Press, 2006.

Office of Research Integrity. “Survey of Research Integrity Measures Utilized in Biomedical Research Laboratories. Final Report.” [www.ori.hhs.gov/documents/research/ integrity_measures_final_report_11_07_03.pdg] Accessed July 17, 2006.

Office of Research Integrity. “Responsible Conduct in Research Mentoring.” [http://ori.hhs.gov/education/products/niu_mentorship/mentoring/relationship/relationship.html]. Accessed August 15, 2006.

“Ph.D. Completion Project.” [http://www.phdcompletion.org/ information]. Accessed June 11, 2006.

University of Toronto. “Graduate Supervision: Guidelines for Students, Faculty, and Administrators.” Toronto, Canada, 2002. [www.sgs.utorontoca/current/ supervision/guidelines.pdf]. Accessed on July 17, 2006.

Wright, David E., Sandra L. Titus, and Jered Cornelison. ^“Mentoring and Research Misconduct: An Analysis of Research Mentoring in Closed ORI Cases” under review 2007.

APPENDIX A

FEDERAL REGISTER MAY 12, 2000 NOTICE

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Office of the Secretary; Office of Public Health and Science

Statement of Organization, Functions, and Delegations of Authority

Part A, (Office of the Secretary) of the Statement of Organization, Functions, and Delegations of Authority for the Department of Health and Human Services, Chapter AC, Office of Public Health and Science (OPHS), paragraph ACA, Immediate Office, as last amended at 62 FR 5009–10, 2/3/97; and paragraph ACF, Office of Research Integrity (ORI), as last amended at 60 FR 56606–06, dated November 9, 1995, are being amended to make policy changes approved by the Secretary. Specifically, the Notice is to reflect that the Assistant Secretary for Health (ASH) will make proposed findings of research misconduct and administrative actions in response to allegations of research misconduct involving research conducted or supported by components of the Public Health Service (PHS); that direct investigations, previously conducted by ORI, will be conducted by components of the PHS for intramural research and by the Office of Inspector General for extramural research; and that role and structure of ORI will be changed to focus more on preventing misconduct and promoting research integrity through expanded education programs. The changes are as follows:

I. Amend Chapter AC.20 Functions, paragraph A. ‘‘Office of Public Health and Science,’’ paragraph titled, ‘‘The Immediate Office (ACA)’’ by adding the following new clause:

(1) Proposes findings of research misconduct and administrative actions in response to allegations of research misconduct involving research conducted or supported by the Public Health Service (PHS) OPDIVs, including reversal of an institution’s no misconduct finding or opening of a new investigation.

II. Under Section AC.20 Function, delete, paragraph E. ‘‘Office of Research Integrity (ACF)’’ in its entirety, and replace with the following:

E. Office of Research Integrity (ACF)— The Director reports to the Secretary and will: (1) Oversee and direct Public Health Service (PHs) research integrity activities on behalf of the Secretary with the exception of the regulatory research integrity activities of the Food and Drug Administration; (2) recommend to the Assistant Secretary for Health for decision, findings of research misconduct and administrative actions in connection with research conducted or supported by the PHS; (3) coordinate the development of research integrity policies designed to ensure that subjects of investigations and whistleblowers are treated fairly, including clear specification of what constitutes misconduct, a fair hearing process, appropriate time limits on pursuing allegations, and specific whistleblower protections; (4) manage the financial resources and provide overall administrative guidance in carrying out the activities; and (5) oversee and direct the research misconduct and integrity activities of the office, including the oversight of research misconduct inquiries and investigations, education and training in the responsible conduct of research, activities designed to promote research integrity and prevent misconduct, and research and evaluation programs.

1. Division of Education and Integrity (ACF2)—The Director and staff: (1) develop and implement, in consultation with the PHS OPDIVs, activities and programs for PHS intramural and extramural research to teach the responsible conduct of research, promote research integrity, prevent research misconduct, and to enable the extramural institutions and PHS OPDIVs to respond effectively to allegations of research misconduct; (2) coordinate the dissemination of research integrity policies, procedures, and regulations; (3) conduct policy analyses, evaluations, and research to improve DHHS research integrity policies and procedures and build the knowledge base in research misconduct, research integrity, and prevention; (4) develop (in consultation with the PHS OPDIVs) policies, procedures, and regulations for review by the Director, Office of Research Integrity, and recommendations to the Secretary; (5) administer programs for: approval of institutional assurances; response to Freedom of Information Act and Privacy Act requests; review and approval of intramural and extramural policies and procedures; and response to allegations of whistleblower retaliation.

2. Division of Investigative Oversight (ACF3)—The Director and staff: (1) review and monitor investigations conducted by applicant and awardee institutions and intramural research programs; (2) evaluate investigations and investigatory findings of awardee and applicant institutions, intramural research programs, and the Office of Inspector General and develop and recommend to the ORI Director, findings of research misconduct and proposal administrative actions against those who committed misconduct; (3) assist the Office of the General Counsel (OGC) in preparing and presenting cases in hearings before the Research Integrity Adjudications Panel of the DHHS Department Appeals Board; (4) provide information on DHHS policies and procedures, as requested, to individuals who have made an allegation or have been accused of research misconduct; and (5) establish and implement a program of advice and technical assistance to entities that conduct inquiries and investigations, or otherwise respond to allegations of research misconduct.

III. Under Chapter AC, Section ACF– 30, Delegations of Authority—All delegations and redelegations of authority to the Assistant Secretary for

Health and officials of the Office of Research Integrity that were in effect prior to the effective date of this reorganization shall continue in effect pending further redelegation.

Dated: April 14, 2000. Betsy D’Jamos,

Acting Assistant Secretary for Management and Budget.

[FR Doc. 00–11958 Filed 5–11–00; 8:45 am]

BILLING CODE 4150–04–M