Administration for Native Americans (ANA) 2020 Discretionary Application Process and Peer Panel Review Surveys

ANA’s 2020 Peer Panel Review Survey

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to provide feedback to the Administration for Native Americans regarding the 2020 discretionary application peer panel review. Public reporting burden for this collection of information is estimated to average 10 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0401 and the expiration date is 05/31/2021. If you have any comments on this collection of information, please contact [email protected].

The purpose of this survey is to provide the Administration for Native Americans (ANA) with information on your experience with the 2020 ANA Peer Panel Review processes so we can improve the process in the future. Your participation is voluntary, and the information provided will be kept private.

I. REVIEW MANAGEMENT

1. The quality of reviewer recruitment and confirmation communications.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. The helpfulness of the logistical information provided to conduct a successful review.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

3. Overall review management.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

4. The “user friendliness” and overall user experience of ARM.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

5. I received clear explanations of the conflict of interest and the confidentiality requirements regarding information about applications, and grant review comments and scores.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

6. The compensation I received was commensurate with the time and effort required.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

7. What resources were most helpful through the panel review process?

II. TRAINING

1. The effectiveness of the Mandatory Online Training

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. Which modules did you find most helpful?

Module 1 (Insert titles)

Module 2

…

3. Since you have now completed ANA’s Panel Review, is there anything you would have wanted covered that was not?

Comment

4. If you attended the Pre-Panel Webinar how helpful was it ?

Not Helpful

Helpful

Very Helpful

5. Are there ways to improve the Pre-Panel Webinar?

6. If you attended the Opening Ceremony, how helpful was it?

Not Helpful

Helpful

Very Helpful

Did not attend

7. If you attended the Opening Ceremony, are there ways to improve it?

8. The quality of the Program Support Web Page (please consider all aspects including the accessibility, organization, and presentation of required documents, forms, training materials, and other items).

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

9. The helpfulness of the guidance materials provided on the Program Support Web Page.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

10. The helpfulness of the programmatic information focused on understanding the Funding Opportunity Announcement (FOA) and guidance for reviewing the grant applications.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

11. Please provide feedback on what could be changed to enhance the trainings?

III. THE OVERALL REVIEW PROCESS

1. The format used for panel discussions (Teleconference, Zoom???.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

2. How would you rate your PM/RD for professionalism and knowledge of the FOA and ANA panel process?

Poor

No Opinion

Good

Excellent

Comment

3. Your overall satisfaction with the review experience.

Unacceptable

Poor

No Opinion

Good

Excellent

Comment

4. The 12 days allotted to read and panel the assigned applications was

Too Long

Just Right

Too Short

5. I would participate in another ANA Panel Review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

6. Overall, I had a positive experience with this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

7. What section(s) of the FOA was/were the most difficult to understand from your perspective, and why?

Comment:

If you are interested in volunteering to be a part of a focus group on how to improve the FOAs in the future, please contact [email protected]

8. How can ANA better help you before, during or after Panel Review?

FOR CHAIRPERSON ONLY

IV. CHAIRPERSON

Dear Panel Chairperson,

Your feedback is valuable. Please take a few minutes to complete our survey. We would especially appreciate comments for items that did not meet your satisfaction so we may further improve future grant review sessions. Your feedback is private and will not be shared with other members of your panel or the public. This survey should take approximately 10 minutes to complete. Thank you.

Chairperson - Overall Grant Review Process

1. The training I received effectively prepared me as a chair for the grant review process.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

2. The pre-panel introductory call with my PM was helpful in preparing me to develop a strong working relationship and establish expectations before the panel session began.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

3. The guidance I received from the program office (PM) was consistent with the FOA, including the FOA's evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

4. On average, how long did it take to draft the initial PSR?

5. What worked best in solving differences between the panelists or comments?

EVALUATION OF PANEL REVIEWERS:

Please complete for each reviewer.

1. The reviewer was knowledgeable about the subject matter related to this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

2. The reviewer had access to technology and adequate computer skills.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

3. The reviewer attended all panel review discussions.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

4. The reviewer was on time to panel review discussions

None of the Time

Some of the Time

Most of the Time

All of the Time

N/A

5. The reviewer respected and cooperated with other panel members.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

6. The reviewer read and clearly understood the applications being reviewed, the FOA, and the evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

7. The reviewer's summary comments were well-written, specific, constructive, and based on the strengths and weaknesses of an application's response to the evaluation criteria.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

8. The reviewer assigned applications scores that were consistent with written comments.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

9. Was the reviewer unbiased and fair?

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

10. I would recommend this reviewer for future panel reviews.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

11. If you are not recommending this reviewer, please provide 2-3 sentences with additional information about problems or issues with the reviewer. If you are recommending the reviewer, please type N/A.

FOR PANEL REVIEWER ONLY

Dear Panel Reviewer,

Your feedback is valuable. Please take a few minutes to complete our survey. We would especially appreciate comments for items that did not meet your satisfaction so we may further improve future grant review sessions. Your feedback is private and will not be shared with other members of your panel or the public. This survey should take approximately 10 minutes to complete. Thank you.

IV. PANELIST

1. On average, how long did it take to review, score, and draft initial comments for applications?

2. The panel chairperson clearly established administrative and procedural rules for the review (e.g. established a schedule, ensured reviewers had received all needed materials, and established ground rules).

Strongly Disagree

Disagree

Agree

Strongly Agree

Comment

3. The panel chairperson instructed reviewers to use only the guidance from the FOA evaluation criteria in evaluating applications.

Strongly Disagree

Disagree

Agree

Strongly Agree

Comment

4. The panel chairperson effectively led panel discussions and facilitated meetings.

Strongly Disagree

Disagree

Agree

Strongly Agree

Comment

5. The panel chairperson effectively resolved any differences of opinion that arose between reviewers in regard to scoring applications.

Strongly Disagree

Disagree

Agree

Strongly Agree

N/A

Comment

6. The panel chairperson effectively led the process of developing the Panel Summary Reports.

Strongly Disagree

Disagree

Agree

Strongly Agree

Comment

7. Overall, the panel chairperson was effective in managing this review.

Strongly Disagree

Disagree

Agree

Strongly Agree

Comment

8. Please include any additional comments in the space below.