Reviewer Survey

Attachment A — Questionnaire:  
Reviewer Survey 

Sampling Specifications: 

Distributed to IMLS grant application reviewers following rounds of application reviews.

Research Objectives/Research Questions: 

1. Usefulness of resources created for peer reviewers

2. Experience using systems and forms

3. Motivating factors for serving as a peer reviewer

Overall Programming Instructions

[All closed-ended questions soft validated. All open-ended questions optional. One question per page.]

[Programmer: Capture reviewer type from population frame.]

Section I: Preamble 

[Text Box - Introduction]:  

Thank you for clicking through to this survey! 

Your thoughtful participation should take less than 10 minutes and will help IMLS continually improve the effectiveness of its review processes and the experience of our valued reviewers.

This survey is confidential, and any current or future relationship you may have with IMLS will not be affected by your participation. IMLS’ Office of Museum Services (OMS), Office of Library Services (OLS), and Office of Grants Policy and Management (OGPM) will not have direct access to the data collected.

Your personal identity, and your institution’s identity, will not be included in any data downloads or reports made accessible outside of IMLS’ Office of Research and Evaluation (ORE), who have no role in the agency’s grantmaking.

[CONTINUE BUTTON]

Access to this survey platform is password-protected and certified to meet the Federal Government's data security requirements. All of the data collected is maintained on firewalled servers that meet federal data security requirements. The collected data will be disposed of in compliance with all relevant federal statutes.

Taking this survey is voluntary, and you may skip any question that you do not wish to answer. You do not need to complete the survey in one session. If you close the survey window, you can pick up where you left off using your original link.

You are not required to respond to this collection of information unless it displays a currently valid U.S. Office of Management and Budget (OMB) control number. The OMB control number is 3137-0081, and the expiration date is 1/31/2027 If you have any questions, please contact the survey team at [email protected].

Let’s begin! 

Section II: Overall Experience

1. Overall, how satisfied or dissatisfied were you with your experience as an IMLS grant application reviewer?

   1. Very satisfied

   2. Somewhat satisfied

   3. Somewhat dissatisfied

   4. Very dissatisfied

2. How would you rate your experience with the following parts of the grant application review process?

[RANDOMIZE ROWS. SCALE: Very positive, Somewhat positive, Somewhat negative, Very negative, N/A]

1. Interacting with other reviewers

2. Interacting with IMLS staff

3. Reading the grant applications

3. Was the amount of time you were given to read your assigned grant applications…?

   1. More than enough

   2. About right

   3. Not enough

4. Overall, was the amount of time you spent in the grant application review process…?

[RANDOMIZE ROWS. SCALE: Much less than you expected, Somewhat less, About what you expected, Somewhat more, Much more than you expected, N/A]

1. Interacting with others in the panel

2. Reading your assigned materials

3. Signing up to participate in IMLS reviews

5. How easy or difficult was your experience…?

[RANDOMIZE ROWS. SCALE: Very easy, Somewhat easy, Somewhat difficult, Very difficult]

1. Finding the information you needed for your reviews

2. Interacting with IMLS systems

3. Interacting with IMLS staff

Section III: Usefulness of resources created for peer reviewers

6. Did you use the following resources as a part of your experience as a reviewer for IMLS?

[RANDOMIZE ROWS. SCALE: Yes, No, Not Sure]

1. Reviewer Handbook

2. eGMS Reach

3. Webinar for Peer Reviewers

4. Emails from program officer

[ASK IF A = YES AT Q6]

7. Was the information provided in the Reviewer Handbook…?

[RANDOMIZE ROWS. SCALE: Yes, No, Not Sure]

1. Easy to find

2. Clear

3. Accurate

4. Sufficiently detailed

[ASK IF B = YES AT Q6]

8. Was the information provided in eGMS Reach…?

[RANDOMIZE ROWS. SCALE: Yes, No, Not Sure]

1. Easy to find

2. Clear

3. Accurate

4. Sufficiently detailed

[ASK FOR ITEMS = YES AT Q6]

9. Overall, how useful were the following resources in aiding your experience as a reviewer for IMLS?

[RANDOMIZE ROWS. SHOW ROWS BASED ON YES AT Q6.]

[SCALE: Very useful, Somewhat useful, Not very useful, Not at all useful]

1. Reviewer Handbook

2. Webinar for Peer Reviewers

3. eGMS Reach

4. Emails from program officer

10. What information, if any, was missing from/should be added to the resources provided for reviewers by IMLS?

Please help protect your confidentiality and avoid providing personally identifying information in your response.

[OPEN ENDED PARAGRAPH-SIZED BOX]

11. Do you have any other feedback regarding the resources provided for reviewers by IMLS?

Please help protect your confidentiality and avoid providing personally identifying information in your response.

[OPEN ENDED PARAGRAPH-SIZED BOX]

Section IV: Experience using systems and forms

12. Did you complete the following forms as a part of your experience as a reviewer for IMLS?

[RANDOMIZE ROWS. SCALE: Yes, No, Not Sure]

1. Peer Reviewer Services Agreement (Microsoft Office secure fillable form)

2. Complying with Ethical Obligations and Avoiding Conflicts of Interest (PDF in eGMS Reach)

3. Evaluation forms in eGMS Reach (format for providing written comments and scores for each application)

4. IMLS Travel Forms and Documentation

13. How easy or difficult was it to fill out the following forms?

[SHOW ROWS = YES AT Q12. RANDOMIZE ROWS.]

[SCALE: Very easy, Somewhat easy, Somewhat difficult, Very difficult]

1. Peer Reviewer Services Agreement (Microsoft Office secure fillable form)

2. Complying with Ethical Obligations and Avoiding Conflicts of Interest (PDF in eGMS Reach)

3. Evaluation forms in eGMS Reach (form for providing written comments and scores for each application)

4. IMLS Travel Forms and Documentation

14. Do you have any other feedback regarding the forms provided by IMLS?

Please help protect your confidentiality and avoid providing personally identifying information in your response.

[OPEN ENDED PARAGRAPH-SIZED BOX]

Section V: Motivating factors for serving as a peer reviewer

15. Which of the below were your top three motivating factors for participating in IMLS grant reviews?

[RANDOMIZE ROWS, ANCHOR OTHER]

1. Helping peer institutions

2. Service to the field as a whole

3. Personal growth/professional development

4. Honorarium received

5. Gaining insight into IMLS’ grant application review process

6. Using my experience/expertise to benefit others

7. Opportunity to interact with other reviewers

8. Opportunity to interact with IMLS staff

9. Other (Please Specify:)

Section VI: Summary Questions & Close

16. Overall, how interested would you be in serving as an IMLS reviewer again in the future?

    1. Very interested

    2. Somewhat interested

    3. Not very interested

    4. Not at all interested

17. Is there any additional information you would like IMLS to know about your experience as a reviewer?

Please help protect your confidentiality and avoid providing personally identifying information in your response.

[OPEN ENDED PARAGRAPH-SIZED BOX]

[TEXT BOX]

That’s all of the questions. Thank you for taking the survey!

[COMPLETED SURVEY REDIRECTS TO IMLS.GOV]

IMLS Reviewer Survey

Page 0 of 8