Draft-Survey Instruments (3)

OMB Clearance # 0925-0534

Supporting Statement for the

Paperwork Reduction Act Submission

Survey Instruments (Attachment 2)

National Institutes of Health

A Process Evaluation of the NIH Director’s

Pioneer Award (NDPA) Program

Survey Questionnaires

Survey Instrument—NDPA

General Survey for Applicants (not Interviewees and Awardees)

Welcome to the NDPA Candidate Survey. Please provide responses to the following questions to the best of your ability. You may choose not to answer specific questions and it will not affect your ability to submit the survey. After choosing a response, please click "next" to view the next set of questions. If you would like to go back and change a response, you can use the "back" button on the survey or the pull down menu at the bottom of the page. Please do not use your browser's navigation buttons. If you would like to save and come back to the survey, click the "save" button at the bottom of any page. The survey should take 10-15 minutes to complete.

Please consult the NDPA website to review the Request for Applications (RFA), criteria, or processes:

NDPA Website

Public reporting burden for this collection of information is estimated to average 15 minutes per response (.25 hours), including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0534). Do not return the completed form to this address.

Please note that participation in this survey is entirely voluntary. Your decision to participate will have no effect on your current or future NIH funding status, and other risks for participation or non-participation are minimal.

Additionally, you may click on underlined words in the survey, which are hyperlinked to the appropriate document.

To begin the survey, scroll down and click "next."

1. Prior to NDPA, had you ever applied for an NIH award or grant as a Principal Investigator (PI) or Co-PI?

{Choose one}

( ) Yes

( ) No

( ) Do not know

Background/Funding History

Please list other sponsors below:

{Enter answer in paragraph form}

[

]

Other

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Hospitals, universities, or other non-profit institutions

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

For-profit companies

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Foundations (e.g. Howard Hughes Medical Institute, Ford Foundation, etc.)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Other US government sources

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

National Institutes of Health (NIH)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

2. Please provide a rough estimate of the percentage of your total research funding over the past five years represented by each source listed below. (Please make sure that your answers do not add up to more than 100%.)

3. In your opinion, given the innovative nature of your proposed NDPA project, what is the likelihood that it might be supported by other funders?

{Choose one}

( ) Very likely

( ) Somewhat likely

( ) Somewhat unlikely

( ) Very unlikely

In your opinion, how risky was your proposed idea?

( ) Very risky

( ) Somewhat risky

( ) Not risky

4. Please indicate which of the following statements (if any) are true of the research you proposed to NDPA in 200X:

{Choose all that apply}

( ) One or more of the fundamental ideas underlying my proposed research were at odds with prevailing wisdom

( ) My proposed research required use of equipment or techniques that have not been proven or are extraordinarily difficult

( ) My proposed research required knowledge of fields beyond my previously demonstrated area of expertise

( ) My research involved a unique and unprecedented combination of perspectives, disciplines, or approaches

( ) None of these statements is true of my proposed research

5. Do you plan to reapply for an NDPA in future years?

{Choose one}

( ) Yes

( ) No

Why not?

{Enter answer in paragraph form}

[

]

In future years, to what extent will you make changes to the basic idea you proposed in FY 200X?

{Choose one}

( ) I will submit a completely different idea

( ) I will make substantial changes to the basic idea I proposed

( ) I will make minor changes to the basic idea I proposed

( ) The basic idea I proposed will remain the same

The Application Process

Other:

{Enter answer in paragraph form}

[

]

6. Where did you hear about the NIH Director's Pioneer Award (NDPA)? (Select all that apply)

{Choose all that apply}

( ) Journal

( ) Departmental flyer or announcement

( ) Federal Register

( ) NDPA website

( ) Other website (please specify below)

( ) Word of mouth

( ) Do not recall

7. As part of the application process, you were asked to classify your research into one or more of seven areas:

1. behavioral and social sciences

2. clinical and translational research

3. instrumentation and engineering

4. molecular, cellular, and chemical biology

5. pathogenesis or epidemiology

6. physiological and integrative systems

7. quantitative and mathematical biology

or select "other." Were these seven areas adequate to choose from?

{Choose one}

( ) Yes

( ) No

Which other areas should be offered in future rounds?

{Enter answer in paragraph form}

[

8. Do you agree that you were given adequate opportunity to display your qualifications in the application phase?

{Choose one}

( ) Completely agree

( ) Somewhat agree

( ) Somewhat disagree

( ) Completely disagree

Please comment on what additional information you would have preferred to provide or if any information you provided seemed unnecessary.

{Enter answer in paragraph form}

[

]

9. In your opinion, how important was each of the following application components in allowing you to display your qualifications for the NIH Director's Pioneer Award?

Application materials

3-5 page essay that addresses your innovative vision

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

2 page biographical sketch

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

List of current research support

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

3 letters of reference

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

Your most significant accomplishment

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

300 word abstract describing project goals

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

If the criteria were not appropriate or adequate in your opinion, please propose additional or different criteria.

{Enter answer in paragraph form}

[

]

Criteria for selecting awardees

10. Do you agree that the stated criteria for selecting awardees (see link below), were adequate and appropriate to select "scientists of exceptional creativity who take innovative approaches to major challenges in biomedical research"?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

If it was unclear, please suggest ways that the directions could be made more clear in the future.

{Enter answer in paragraph form}

[

]

NDPA RFA

11. Was the 200X NDPA RFA (see link below) clear in describing the kind of person (e.g. "exceptionally creative") or the kind of idea (e.g. "high-impact") the program seeks to fund?

{Choose one}

( ) Yes, it was clear

( ) No, it was unclear

12. Did you have any difficulties with the electronic submission forms during the application phase?

{Choose one}

( ) Yes

( ) No

If yes, please describe your difficulties:

{Enter answer in paragraph form}

[

]

Overall Assessment

13. Please provide any additional comments or feedback related to the NIH Director's Pioneer Award.

{Enter answer in paragraph form}

[

]

Demographic Information

Please note: Responses to these questions are entirely voluntary.

Gender

{Choose one}

( ) Male

( ) Female

Ethnicity

{Choose one}

( ) Hispanic or Latino

( ) Not Hispanic or Latino

Race (Please mark all that apply)

{Choose all that apply}

( ) American Indian or Alaska Native

( ) Asian

( ) Black or African American

( ) Native Hawaiian or other Pacific Islander

( ) White

Age (Please select a range)

{Choose one}

( ) under 20

( ) 20 - 24

( ) 25 - 29

( ) 30 - 34

( ) 35 - 39

( ) 40 - 44

( ) 45 - 49

( ) 50 - 54

( ) 55 - 59

( ) 60 - 64

( ) 65 - 69

( ) 70 - 74

( ) 75 - 79

( ) 80 - 84

( ) 85 - 89

( ) 90+

Thank you for completing the 200X NDPA Survey. Please click "finish" below to submit your responses.

Special Survey for Interviewees

Welcome to the NDPA Candidate Survey. Please provide responses to the following questions to the best of your ability. You may choose not to answer specific questions and it will not affect your ability to submit the survey. After choosing a response, please click "next" to view the next set of questions. If you would like to go back and change a response, you can use the "back" button on the survey or the pulldown menu at the bottom of the page. Please do not use your browser's navigation buttons. If you would like to save and come back to the survey, click the "save" button at the bottom of any page. The survey should take 10-15 minutes to complete.

Please consult the NDPA website to review the Request for Applications (RFA), criteria, or processes:

NDPA Website

Public reporting burden for this collection of information is estimated to average 15 minutes per response (.25 hours), including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0534). Do not return the completed form to this address.

Please note that participation in this survey is entirely voluntary. Your decision to participate will have no effect on your current or future NIH funding status, and other risks for participation or non-participation are minimal.

Additionally, you may click on underlined words in the survey, which are hyperlinked to the appropriate document.

To begin the survey, scroll down and click "next."

1. Prior to NDPA, had you ever applied for an NIH award or grant as a Principal Investigator (PI) or Co-PI?

{Choose one}

( ) Yes

( ) No

( ) Do not know

Background/Funding History

Please list other sponsors below:

{Enter answer in paragraph form}

[

]

Other

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Hospitals, universities, or other non-profit institutions

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

For-profit companies

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Foundations (e.g. Howard Hughes Medical Institute, Ford Foundation, etc.)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Other US government sources

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

National Institutes of Health (NIH)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

2. Please provide a rough estimate of the percentage of your total research funding over the past five years represented by each source listed below. (Please make sure that your answers do not add up to more than 100%.)

3. In your opinion, given the innovative nature of your proposed NDPA project, what is the likelihood that it might be supported by other funders?

{Choose one}

( ) Very likely

( ) Somewhat likely

( ) Somewhat unlikely

( ) Very unlikely

In your opinion, how risky was your proposed idea?

( ) Very risky

( ) Somewhat risky

( ) Not risky

4. Please indicate which of the following statements (if any) are true of the research you proposed to NDPA in 200X:

{Choose all that apply}

( ) One or more of the fundamental ideas underlying my proposed research were at odds with prevailing wisdom

( ) My proposed research required use of equipment or techniques that have not been proven or are extraordinarily difficult

( ) My proposed research required knowledge of fields beyond my previously demonstrated area of expertise

( ) My research involved a unique and unprecedented combination of perspectives, disciplines, or approaches

( ) None of these statements is true of my proposed research

5. Our records indicate that you applied previously for an NDPA award. In 200X, to what extent did you make changes to the basic idea you had proposed in previous year(s)?

{Choose one}

( ) I submitted a completely different idea

( ) I made substantial changes to the basic idea I proposed

( ) I made minor changes to the basic idea I proposed

( ) The basic idea I proposed remained the same

Why not?

{Enter answer in paragraph form}

[

]

6. Do you plan to reapply for an NDPA in future years?

{Choose one}

( ) Yes

( ) No

In future years, to what extent will you make changes to the basic idea you proposed in FY 200X?

{Choose one}

( ) I will submit a completely different idea

( ) I will make substantial changes to the basic idea I proposed

( ) I will make minor changes to the basic idea I proposed

( ) The basic idea I proposed will remain the same

The Application Process

Other:

{Enter answer in paragraph form}

[

]

7. Where did you hear about the NIH Director's Pioneer Award (NDPA)? (Select all that apply)

{Choose all that apply}

( ) Journal

( ) Departmental flyer or announcement

( ) Federal Register

( ) NDPA website

( ) Other website (please specify below)

( ) Word of mouth

( ) Do not recall

Which other areas should be offered in future rounds?

{Enter answer in paragraph form}

[

]

8. As part of the application process, you were asked to classify your research into one or more of seven areas:

1. behavioral and social sciences

2. clinical and translational research

3. instrumentation and engineering

4. molecular, cellular, and chemical biology

5. pathogenesis or epidemiology

6. physiological and integrative systems

7. quantitative and mathematical biology

or select "other." Were these seven areas adequate to choose from?

{Choose one}

( ) Yes

( ) No

9. Do you agree that you were given adequate opportunity to display your qualifications in the application phase?

{Choose one}

( ) Completely agree

( ) Somewhat agree

( ) Somewhat disagree

( ) Completely disagree

Please comment on what additional information you would have preferred to provide or if any information you provided seemed unnecessary.

{Enter answer in paragraph form}

[

]

10. In your opinion, how important was each of the following application components in allowing you to display your qualifications for the NIH Director's Pioneer Award?

Application materials

3-5 page essay that addresses your innovative vision

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

2 page biographical sketch

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

List of current research support

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

3 letters of reference

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

Your most significant accomplishment

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

300 word abstract describing project goals

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

If the criteria were not appropriate or adequate in your opinion, please propose additional or different criteria.

{Enter answer in paragraph form}

[

]

Criteria for selecting awardees

11. Do you agree that the stated criteria for selecting awardees (see link below), were adequate and appropriate to select "scientists of exceptional creativity who take innovative approaches to major challenges in biomedical research"?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

If it was unclear, please suggest ways that the directions could be made more clear in the future.

{Enter answer in paragraph form}

[

]

NDPA RFA

12. Was the 200X NDPA RFA (see link below) clear in describing the kind of person (e.g. "exceptionally creative") or the kind of idea (e.g. "high-impact") the program seeks to fund?

{Choose one}

( ) Yes, it was clear

( ) No, it was unclear

The Interview Process

If not, what did you find particularly problematic or unclear?

{Enter answer in paragraph form}

[

]

13. Were the invitation and instructions clear about the intent, purpose, and content of the interview?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

Please feel free to comment on the interview instructions and guidelines:

{Enter answer in paragraph form}

[

]

14. Was the duration of the interview appropriate to convey your interest and enthusiasm in the NDPA program and to make your case as a candidate?

{Choose one}

( ) The interview was too long

( ) The interview was too short

( ) The interview was about right

Please comment on the interview process below:

{Enter answer in paragraph form}

[

]

15. Do you agree that the interviewers adequately understood your ideas and gave you a fair chance to convey your ideas?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

16. Did you have any difficulties with the electronic submission forms during the application phase?

{Choose one}

( ) Yes

( ) No

If yes, please describe your difficulties:

{Enter answer in paragraph form}

[

]

Overall Assessment

17. Please provide any additional comments or feedback related to the NIH Director's Pioneer Award.

{Enter answer in paragraph form}

[

]

Demographic Information

Please note: Responses to these questions are entirely voluntary.

Gender

{Choose one}

( ) Male

( ) Female

Ethnicity

{Choose one}

( ) Hispanic or Latino

( ) Not Hispanic or Latino

Race (Please mark all that apply)

{Choose all that apply}

( ) American Indian or Alaska Native

( ) Asian

( ) Black or African American

( ) Native Hawaiian or other Pacific Islander

( ) White

Age (Please select a range)

{Choose one}

( ) under 20

( ) 20 - 24

( ) 25 - 29

( ) 30 - 34

( ) 35 - 39

( ) 40 - 44

( ) 45 - 49

( ) 50 - 54

( ) 55 - 59

( ) 60 - 64

( ) 65 - 69

( ) 70 - 74

( ) 75 - 79

( ) 80 - 84

( ) 85 - 89

( ) 90+

Thank you for completing the 200X NDPA Survey. Please click "finish" below to submit your responses.

Special Survey for Awardees

Welcome to the NDPA Candidate Survey. Please provide responses to the following questions to the best of your ability. You may choose not to answer specific questions and it will not affect your ability to submit the survey. After choosing a response, please click "next" to view the next set of questions. If you would like to go back and change a response, you can use the "back" button on the survey or the pulldown menu at the bottom of the page. Please do not use your browser's navigation buttons. If you would like to save and come back to the survey, click the "save" button at the bottom of any page. The survey should take 10-15 minutes to complete.

Please consult the NDPA website to review the Request for Applications (RFA), criteria, or processes:

NDPA Website

Public reporting burden for this collection of information is estimated to average 15 minutes per response (.25 hours), including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0534). Do not return the completed form to this address.

Please note that participation in this survey is entirely voluntary. Your decision to participate will have no effect on your current or future NIH funding status, and other risks for participation or non-participation are minimal.

Additionally, you may click on underlined words in the survey, which are hyperlinked to the appropriate document.

To begin the survey, scroll down and click "next."

1. Prior to NDPA, had you ever applied for an NIH award or grant as a Principal Investigator (PI) or Co-PI?

{Choose one}

( ) Yes

( ) No

( ) Do not know

Background/Funding History

Please list other sponsors below:

{Enter answer in paragraph form}

[

]

Other

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Hospitals, universities, or other non-profit institutions

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

For-profit companies

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Foundations (e.g. Howard Hughes Medical Institute, Ford Foundation, etc.)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

Other US government sources

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

National Institutes of Health (NIH)

{Choose one}

( ) 0%

( ) 1-24%

( ) 25-49%

( ) 50-74%

( ) 75-100%

2. Please provide a rough estimate of the percentage of your total research funding over the past five years represented by each source listed below. (Please make sure that your answers do not add up to more than 100%.)

3. In your opinion, given the innovative nature of your proposed NDPA project, what is the likelihood that it might be supported by other funders?

{Choose one}

( ) Very likely

( ) Somewhat likely

( ) Somewhat unlikely

( ) Very unlikely

In your opinion, how risky was your proposed idea?

( ) Very risky

( ) Somewhat risky

( ) Not risky

4. Please indicate which of the following statements (if any) are true of the research you proposed to NDPA in 200X:

{Choose all that apply}

( ) One or more of the fundamental ideas underlying my proposed research were at odds with prevailing wisdom

( ) My proposed research required use of equipment or techniques that have not been proven or are extraordinarily difficult

( ) My proposed research required knowledge of fields beyond my previously demonstrated area of expertise

( ) My research involved a unique and unprecedented combination of perspectives, disciplines, or approaches

( ) None of these statements is true of my proposed research

5. Our records indicate that you applied previously for an NDPA award. In 200X, to what extent did you make changes to the basic idea you had proposed in previous year(s)?

{Choose one}

( ) I submitted a completely different idea

( ) I made substantial changes to the basic idea I proposed

( ) I made minor changes to the basic idea I proposed

( ) The basic idea I proposed remained the same

The Application Process

Other:

{Enter answer in paragraph form}

[

]

6. Where did you hear about the NIH Director's Pioneer Award (NDPA)? (Select all that apply)

{Choose all that apply}

( ) Journal

( ) Departmental flyer or announcement

( ) Federal Register

( ) NDPA website

( ) Other website (please specify below)

( ) Word of mouth

( ) Do not recall

Which other areas should be offered in future rounds?

{Enter answer in paragraph form}

[

]

7. As part of the application process, you were asked to classify your research into one or more of seven areas:

1. behavioral and social sciences

2. clinical and translational research

3. instrumentation and engineering

4. molecular, cellular, and chemical biology

5. pathogenesis or epidemiology

6. physiological and integrative systems

7. quantitative and mathematical biology

or select "other." Were these seven areas adequate to choose from?

{Choose one}

( ) Yes

( ) No

8. Do you agree that you were given adequate opportunity to display your qualifications in the application phase?

{Choose one}

( ) Completely agree

( ) Somewhat agree

( ) Somewhat disagree

( ) Completely disagree

Please comment on what additional information you would have preferred to provide or if any information you provided seemed unnecessary.

{Enter answer in paragraph form}

[

]

9. In your opinion, how important was each of the following application components in allowing you to display your qualifications for the NIH Director's Pioneer Award?

Application materials

3-5 page essay that addresses your innovative vision

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

2 page biographical sketch

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

List of current research support

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

3 letters of reference

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

Your most significant accomplishment

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

300 word abstract describing project goals

{Choose one}

( ) (Least Important) 1

( ) 2

( ) 3

( ) 4

( ) 5

( ) (Most Important) 6

If the criteria were not appropriate or adequate in your opinion, please propose additional or different criteria.

{Enter answer in paragraph form}

[

]

Criteria for selecting awardees

10. Do you agree that the stated criteria for selecting awardees (see link below), were adequate and appropriate to select "scientists of exceptional creativity who take innovative approaches to major challenges in biomedical research"?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

If it was unclear, please suggest ways that the directions could be made more clear in the future.

{Enter answer in paragraph form}

[

]

NDPA RFA

10. Was the 200X NDPA RFA (see link below) clear in describing the kind of person (e.g. "exceptionally creative") or the kind of idea (e.g. "high-impact") the program seeks to fund?

{Choose one}

( ) Yes, it was clear

( ) No, it was unclear

The Interview Process

If not, what did you find particularly problematic or unclear?

{Enter answer in paragraph form}

[

]

12. Were the invitation and instructions clear about the intent, purpose, and content of the interview?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

13. Was the duration of the interview appropriate to convey your interest and enthusiasm in the NDPA program and to make your case as a candidate?

{Choose one}

( ) The interview was too long

( ) The interview was too short

( ) The interview was about right

Please comment on the interview process below:

{Enter answer in paragraph form}

[

]

14. Do you agree that the interviewers adequately understood your ideas and gave you a fair chance to convey your ideas?

{Choose one}

( ) Completely Agree

( ) Somewhat Agree

( ) Somewhat Disagree

( ) Completely Disagree

15. Did you have any difficulties with the electronic submission forms during the application phase?

{Choose one}

( ) Yes

( ) No

If yes, please describe your difficulties:

{Enter answer in paragraph form}

[

]

Overall Assessment

16. Please provide any additional comments or feedback related to the NIH Director's Pioneer Award.

{Enter answer in paragraph form}

[

]

Demographic Information

Please note: Responses to these questions are entirely voluntary.

Gender

{Choose one}

( ) Male

( ) Female

Ethnicity

{Choose one}

( ) Hispanic or Latino

( ) Not Hispanic or Latino

Race (Please mark all that apply)

{Choose all that apply}

( ) American Indian or Alaska Native

( ) Asian

( ) Black or African American

( ) Native Hawaiian or other Pacific Islander

( ) White

Age (Please select a range)

{Choose one}

( ) under 20

( ) 20 - 24

( ) 25 - 29

( ) 30 - 34

( ) 35 - 39

( ) 40 - 44

( ) 45 - 49

( ) 50 - 54

( ) 55 - 59

( ) 60 - 64

( ) 65 - 69

( ) 70 - 74

( ) 75 - 79

( ) 80 - 84

( ) 85 - 89

( ) 90+

Page 10

Thank you for completing the 200X NDPA Survey. Please click "finish" below to submit your responses.

Interview Protocol

Last Name, First Name:

Title:

Date:

STPI staff:

STATEMENT OF INFORMED CONSENT

The Science and Technology Policy Institute (STPI), a federally funded research and development center based in Washington, DC, has been requested by the National Institutes of Health (NIH) to evaluate the process by which recipients of the NIH Director’s Pioneer Awards (NDPA) were chosen. The primary objectives of the evaluation are to: (1) assess the NDPA award selection process; (2) examine whether the program was implemented as planned; and (3) determine if the process was conducted in accordance with the overall mission of the NDPA program.

We are employing various data collection techniques to answer these questions; however, we believe that some of the most valuable information will come from those who were involved directly in the evaluation process. These informal interviews are one mechanism that will provide important information concerning the overall NDPA process, and will hopefully highlight aspects of the award process that may need to be revised or improved for future rounds of awards. We anticipate conducting approximately XX such interviews.

Please note that:

• Your responses will be kept strictly confidential: If you choose to participate, respondent confidentiality will be protected to the extent provided by law, and STPI will report only aggregate information concerning overall impressions of the process to the NIH.

• Your participation is entirely voluntary: You are under no obligation to interview with us, but we strongly encourage you to do so. A successful evaluation of the NDPA awards process depends on a high response rate to gather as much information and as many perspectives as possible. There are no consequences or risks for participating. Refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue the interview at any time without penalty or loss of benefits to which you are otherwise entitled.

• Whom to contact for additional information: For additional information about the study you may contact Bhavya Lal, STPI project director XX. If you have any questions that you would like to address to the NIH Office of the Director, please contact G. Stephane Philogene, Ph.D., the OD Program Officer responsible for this evaluation (e-mail: [email protected]).

Public reporting burden for this collection of information is estimated to average 15 minutes per response (.25 hours), including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0534). Do not return the completed form to this address.

Extramural Evaluators Questions

Introductory questions

1. How did you become involved with NDPA as an evaluator?

2. Was your evaluator training adequate – were the terms adequately defined and criteria made explicit?

3. Did you have adequate time to review the application packages?

Review Criteria

4. How did you decide if a (application/interviewee) was responsive/competitive? In other words, what were you looking for in a competitive (applicant, interviewee)?

5. What role did existing grant support play in your review? What role did career stage play in your review?

6. How often did you look individuals up online or consult with others during your review?

7. Reviewers were instructed to use 3 primary criteria to evaluate individuals. For each, what did you identify as the markers - can you please give an example of how you operationalized each of the criteria?

The evaluation Criteria for the 200X NIH Director’s Pioneer Award included:
1. Scientific problem to be addressed (Biomedical significance/importance, If successful, likelihood of high impact on biomedical problem, Creativity/innovativeness)
2. Investigator (Evidence for claim of innovativeness/creativity (innovation density); Demonstrated ability to devote 51% or more effort on NDPA project0
3. Suitability for NDPA mechanism (Evidence that proposed project is of sufficient risk/impact to make it more suitable for NDPA than for traditional NIH grant mechanism; Distinct from other research by investigator).

8. Which of the criteria was most important to you in your assessment of the application packages?

9. Do you believe these criteria are adequate to identify a "pioneer?" If not, what other criteria might be used in the future years?

10. How did you evaluate applications outside of your area of expertise? Were there instances where you were not comfortable with reviewing an application because of the subject area?

Application Material

11. How did the information provided to you in the application package help you to evaluate an individual's "pioneeringness?"

12. Was information missing that if there would have helped you make a better decision? Was certain information provided to you better left concealed (name, affiliation, etc.)?

Scoring System

13. Was the scale (1-5) used to rate a package the best way to provide feedback to NDPA?

General Characterization

14. How would you characterize the applications you reviewed? Were the ideas outside the realm of convention (conceptually or technically risky? (Rate risk from scale 1-5) Multidisciplinary? Outside the realm of the investigator's experience?

15. Have you participated in other traditional NIH study sections? How did this process differ? In your opinion, did the NDPA process allow you to choose applications that might not be funded under a traditional study section?

16. Do you have a sense of how did the nominations that you reviewed compare with those applying through other NIH mechanisms, e.g. R01, R21s? In your opinion, did NIH truly capture researchers and/or ideas that wouldn't otherwise be in the NIH system? To the best of your knowledge, to what extent is Pioneer adding value to the NIH portfolio?

17. One of the stated goals of the Pioneer Award Program is to bring in unique ideas, approaches, and/or people that are not being funded through NIH traditional peer review system. Based on your review of (nominations, applications, or interviewees), and any follow up you have had on the program (e.g. you’ve seen the winners), do you believe that Pioneer accomplished this goal? What are your thoughts on the awardees - are they "pioneers"?

General

18. (Repeat Applicants Only) In your opinion, how did this year’s review phase compare to that of previous years in terms of clarity, consistency, etc?

19. Did you have any difficulty with the electronic forms or data recording (e.g., grants.gov)?

20. Do you have any final feedback on the FY 200X NDPA process? Do you have any other recommendations for how the program could be improved?

21. Given your experiences this year, would you consider being involved in the program next year? (If they say no) What must change for you to participate again?