CSR Mini SSA

Mini Supporting Statement A

Stakeholder Perceptions of the NIH Peer Review Process Focus Groups

OMB# 0925-0474 Expiration Date 2/28/2018

April 29, 2015

Mary Ann Noecker Guadagno

301-435-1251

[email protected]

Mini Supporting Statement A

A.1 Circumstances Making the Collection of Information Necessary

The Center for Scientific Review (CSR) is the portal for NIH grant applications and their review for scientific merit. Our mission is to see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research. The proposed project is important and timely to gain a comprehensive and systematic view of stakeholder perceptions of the quality of peer review during a time of social and economic change. In the current economic climate of shrinking pay lines and increased competition for research dollars, the economic and personal burden of peer review needs to be examined and monitored. To identify areas of success and improvement in the quality of peer review, CSR will conduct a series of focus groups with stakeholder groups, through a contract awarded to Social Solutions International, Inc. (Social Solutions) under the OMB control number 0925-0474, with expiration date 2/28/2018.

A.2 Purpose and Use of the Information Collection

The focus groups will examine the extent to which current best practices are optimal for achieving CSR’s mission and identify ways in which improvements in the quality and/or cost of peer review can be achieved. Focus groups will be conducted with key stakeholders including Reviewers, Applicants, Program Officers, and Scientific Review Officers. Results will help CSR and NIH understand what peer review process elements are essential to maintaining the quality and integrity of peer review. Participation in the focus groups will be completely voluntary and information collected will help refine and improve the quality of CSR’s review operations. Identifiable information will be used by Social Solutions to make logistical arrangements, obtain consent for participation, and provide remuneration and expense reimbursement upon completion. Aggregate findings and de-identified transcripts will be provided to CSR. Participants will be assured that neither their participation/non-participation nor any responses in focus groups will be linked to their identity and will not have any effect on support or services at CSR or NIH in the future. All responses will be secure to the extent permitted by law.

A.3 Use of Information Technology to Reduce Burden

N/A

A.4 Efforts to Identify Duplication

There is no duplication of effort for this study. The NIH peer review process is specific to the National Institutes of Health. The NIH Center for Scientific Review is the portal for NIH grant applications and their review for scientific merit. This is the first time a comprehensive effort to examine stakeholder perceptions of the quality of peer review has been conducted in the history of CSR at the NIH.

A.5 Impact on Small Businesses or Other Small Entities

N/A

A.6 Consequences of Collecting the Information Less Frequently

This is a one-time data collection effort.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This survey will be implemented in a manner that fully complies with 5 C.F.R. 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

N/A

A.9 Explanation of Any Payment of Gift to Respondents

A gift card of $75 will be provided to respondents in appreciation for their time participating in the 90 minute focus group and 5 minutes required to complete an informed consent form. It is standard practice to offer respondents some form of remuneration for the time they spend participating in a focus group. Extensive literature supports the use of incentives, primarily monetary incentives, to supplement or complement other efforts of encouragement to potential participants. (See references 1,2 below).

Respondents will receive a gift card through a Contractor being paid to recruit participants. The gift card amount is based on pay scales the company follows and data from the Chronicle of Higher Education Faculty Salary Survey (See reference 3 below). Recruitment professionals and the Chronical of Higher Education data show that full professors across disciplines are earning an annual mean wage ranging from $215,500 to $73,600 based on a 9 month work year. The Chronical of Higher Education report states the salary range in annualized salaries extrapolated for a 12 month work year is quite large ($287,333 to $98,133) with an annual hourly wage rate of $138/hour to $47/hour, based on a 40 hour work week. This averages to about $93 per hour for U.S. faculty members. For this reason, the incentive value for focus group participants who are all faculty members at U.S. universities has been set at $75.

References:

(1) Bricker, Jesse. (2013). “Survey Incentive Fees, Data Quality, Nonresponse, and Survey Administration.” Presented at the Annual Conference of the American Association for Public Opinion Research, Boston, MA.

(2) Singer, E., and Ye, C. (2013). “The Use and Effects of Incentives in Surveys.” The Annals of the American Academy of Political and Social Science, 645, 112-141.

(3) The Chronicle of Higher Education (April 7, 2014). 2013-14 AAUP Faculty Salary Survey.

http://chronicle.com/article/2013-14-AAUP-Faculty-Salary/145679/#id=table

A.10 Assurance of Confidentiality Provided to Respondents

Respondents will be assured in prefatory comments at the focus group, and in an informed consent document they will read and sign before the session begins, that the information shared in focus groups will not be linked to their name in anyway. They will be assured that their participation or non-participation is entirely voluntary and will not have any effect on support or services at CSR or NIH in the future. All responses will be secure and private to the extent permitted by law. Personally Identifiable Information (PII) will not be collected.

A.11 Justification for Sensitive Questions

There are no questions of a sensitive or private nature that will be asked in these focus groups. Discussion topics will cover stakeholder sentiment, experiences, and opinion about policy driven and transparent elements of the NIH peer review process. Respondents will be asked to sign an informed consent document before the focus group session begins ensuring they understand that participation is completely voluntary and no comments will be personally identifiable.

A.12.1 Estimated Annualized Burden Hours

A.12-1 Estimated Annualized Burden Hours

	A.12 - 1 Estimated Annualized Burden Hours
Form Name	Type of Respondent	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Annual Burden Hour
Focus Groups Guide with Applicants	Applicants	72	1	90/60	108 hours
Focus Groups Guide with Reviewers	Reviewers	72	1	90/60	108 hours
Informed Consent Form	Applicants and Reviewers	144	1	5/60	12 hours
	Totals	144			228 hours

Respondents will include a mix of adult scientists connected to the NIH and CSR research communities. It is estimated that approximately 16 focus groups will be convened. Each focus group is expected to take 1.5 hours to complete and will include approximately 9 members per group. Reviewers and applicants will complete an informed consent form, making the total number of respondents 144. The total annual burden hours for the 16 annual focus groups is estimated to be 228 hours.

A.12-2 ANNUALIZED COST TO RESPONDENTS

A.12-2 Annualized Cost to Respondents

Type of Respondent	Number of Respondents	Frequency of Response	Average Burden Per Response (in hours)	Hourly Wage Rate*	Respondent Cost
Adult science professionals - Focus groups	144	1	95/60	$93.00*	$21,204.00

*Source of hourly wage rate estimates: A survey of 1,156 colleges published in April 2014 in the Chronicle of Higher Education http://chronicle.com/article/2013-14-AAUP-Faculty-Salary/145679?cid=megamenu#id=table showed that full professors across disciplines are earning an annual mean wage ranging from $215,500 to $73,600 per year based on a 9 month work year. The vast majority of respondents will be full professors. Based on this report, the range in annualized salaries extrapolated for a 12 month work year is quite large ($287,333 to $98,133) with an annual hourly wage rate of $138/hour to $47/hour, based on a 40 hour work week. This averages to about $93 per hour for U.S. faculty members.

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no additional costs other than respondents’ time and a small amount of money for remuneration for any travel, parking, or incidental costs associated with traveling to a central location for focus groups. The total annual cost burden to respondents resulting from the collection of information will be minimal and is estimated at $2,880.00. This estimate is based on estimating $20 for travel and/or parking for each of the 144 focus group participants.

A.14 Annualized Cost to the Federal Government

A.14 Annualized Cost to the Federal Government

Staff	Grade/Step	Salary	% of Effort	Fringe (if applicable)	Total Cost to Gov’t
Federal Oversight
Mary Ann Guadagno	15/8	$155,705	20%		$ 31,141
Contractor Cost
Staff Avg.		$ 72,570	100%	Included	$290,280
Travel					$7,584
Other Costs*					$148,932
Total Estimated Costs					$477,937

* Other Indirect Costs General & Administrative (G&A)

A.15 Explanation for Program Changes or Adjustments

N/A

A.16 Plans for Tabulation and Publication and Project Time Schedule

Focus group results will be collected, analyzed, and reported by an outside Contractor. Aggregate findings and de-identified transcripts will be provided to CSR. A Content analysis will be performed on the narrative responses by the Contractor who will produce an aggregate, anonymized summary report with accompanying charts. Qualitative data will be presented descriptively without underlying counts. Only percentages or percentage ranges will be reported for the aggregate, if at all. For internal NIH purposes only, if CSR finds important results from data with small cell sizes (N<11), the reports will be issued with a stamp stating “Confidential” and a footer indicating that the data are restricted to authorized users only. Results will be disseminated to key management officials at CSR, NIH management, CSR members and chairs, NIH principle investigators, and CSR employees within six months of focus group completion.

Task (Deliverable in italics)	Timeframe
Develop reporting template, code and categorize focus group data (Template and Coding manual)	April 1 – May 31, 2015
Analyze data	June 1 – July 31, 2015
Prepare preliminary results (Preliminary report on data and findings)	August 1 -15, 2015
Present draft of interim report and obtain feedback from CSR management and staff (Interim report on data and findings)	August 15 – 31, 2015
Finalize report including recommendations (Final report on findings and recommendations)	September 1-22, 2015
Prepare power point slides of findings (508 compliant) (Slides)	September 1-22, 2015
Presentation of results to CSR staff	September 1-22, 2015

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

We are not requesting an exemption to the display of the OMB Expiration date.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

This survey will comply with the requirements in 5 CFR 1320.9.