Administration for Native Americans (ANA) Post Peer Panel Review Survey

2019 ANA Panel Survey

The purpose of this survey is to provide the Administration for Native Americans with information on your experience with the 2019 ANA Peer Panel Review processes so we can improve the process in the future. Your participation is voluntary, and the information provided will be kept private.

THE PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13): Public reporting burden for this collection of information is estimated to average 10 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The control number for this project is 0970-0401. The control number expires on 05/31/2021.

I. REVIEW MANAGEMENT

1. The quality of reviewer recruitment and confirmation communications.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. The helpfulness of the logistical information provided to conduct a successful review.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

3. Overall review management.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

4. The “user friendliness” and overall user experience of ARM.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

5. I received clear explanations of the conflict of interest and the confidentiality requirements regarding information about applications, and grant review comments and scores.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

6. The compensation I received was commensurate with the time and effort required.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

II. TRAINING

1. The quality of the Mandatory Online Training

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. How useful was the Pre-Panel Webinar and Opening Ceremony?

No Helpful

Helpful

Very Helpful

Comment

3. The quality of the Program Support Web Page (please consider all aspects including the accessibility, organization, and presentation of required documents, forms, training materials, and other items).

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

4. The helpfulness of the guidance materials provided on the Program Support Web Page.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

5. The helpfulness of the programmatic information focused on understanding the Funding Opportunity Announcement (FOA) and guidance for reviewing the grant applications.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

III. THE OVERALL REVIEW PROCESS

1. The format used for panel discussions (Teleconference, Webinar).

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. How would you rate your PM/RD for professionalism and knowledge of the FOA and ANA panel process?

Poor

No Opinion

Good

Excellent

Comment

3. Your overall satisfaction with the review experience.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

3. The 11 days allotted to read and panel the assigned applications was

Too Long

Just Right

Too Short

4. I would participate in another ANA Panel Review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

5. Overall, I had a positive experience with this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

FOR CHAIRPERSON ONLY

IV. CHAIRPERSON

Dear Panel Chairperson,

Your feedback is valuable. Please take a few minutes to complete our survey. We would especially appreciate comments for items that did not meet your satisfaction so we may further improve future grant review sessions. Your feedback is private and will not be shared with other members of your panel or the public. This survey should take approximately 10 minutes to complete. Thank you.

Chairperson - Overall Grant Review Process

1. The training I received effectively prepared me as a chair for the grant review process.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

2. The pre-panel introductory call with my PM was helpful in preparing me to develop a strong working relationship and establish expectations before the panel session began.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

3. The guidance I received from the program office (PM) was consistent with the FOA, including the FOA's evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

EVALUATION OF PANEL REVIEWERS:

Please complete for each reviewer.

1. Reviewer's Name (as written in ARM)

First:

Last:

2. The reviewer was knowledgeable about the subject matter related to this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

3. The reviewer had access to technology and adequate computer skills.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

4. The reviewer attended all panel review discussions.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

5. The reviewer was on time to panel * review discussions

None of the Time

Some of the Time

Most of the Time

All of the Time

N/A

6. The reviewer respected and cooperated with other panel members.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

7. The reviewer read and clearly understood the applications being reviewed, the FOA, and the evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

8. The reviewer's summary comments were well-written, specific, constructive, and based on the strengths and weaknesses of an application's response to the evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

9. The reviewer assigned applications scores that were consistent with * written comments.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

10. I would recommend this reviewer for future panel reviews.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

11. If you are not recommending this reviewer, please provide 2-3 sentences with additional information about problems or issues with the reviewer. If you are recommending the reviewer, please type N/A.

FOR PANEL REVIEWER ONLY

Dear Panelist,

Your feedback is valuable. Please take a few minutes to complete our performance and process survey. We would especially appreciate comments for items that did not meet your satisfaction so we may further improve future grant review sessions. Your feedback is private and will not be shared with other members of your panel or the public. This survey should take approximately 10 minutes to complete. Thank you.

1. Name of your Panel Chairperson (as written in ARM)

First:

Last:

2. The panel chairperson clearly established administrative and procedural rules for the review (e.g. established a schedule, ensured reviewers had received all needed materials, and established ground rules).

Strongly Disagree

Disagree

Agree

Strongly Agree

3. The panel chairperson instructed reviewers to use only the guidance from the FOA evaluation criteria in evaluating applications.

Strongly Disagree

Disagree

Agree

Strongly Agree

4. The panel chairperson effectively led panel discussions and facilitated meetings.

Strongly Disagree

Disagree

Agree

Strongly Agree

5. The panel chairperson effectively resolved any differences of opinion that arose between reviewers in regard to scoring applications.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

6. The panel chairperson effectively led the process of developing the Panel Summary Reports.

Strongly Disagree

Disagree

Agree

Strongly Agree

7. Overall, the panel chairperson was effective in managing this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

8. Please include any additional comments in the space below.