CSR SupportingStatementA

12

Supporting Statement A for

GENERIC CLEARANCE FOR SURVEYS

OF CUSTOMERS AND PARTNERS OF

THE CENTER FOR SCIENTIFIC REVIEW (CSR)

NATIONAL INSTITUTES OF HEALTH

Date: Aug 22, 2011

Name: George Chacko, Ph.D.

Address: 6701 Rockledge Drive, Room 3030, Bethesda, MD, 20892-7776

Telephone: 301-435-1133

Fax: 301-480-3965

Email: [email protected]

Table of Contents

A. JUSTIFICATION 4

A.1 Circumstances Making the Collection of Information

Necessary 4

A.2. Purpose and Use of the Information COLLECTION 5

A.3 Use of Information Technology and Burden Reduction 6

A.4 Efforts to Identify Duplication and Use of Similar Information 7

A.5 Impact on Small Businesses or Other Small Entities 7

A.6 Consequences of Collecting the Information Less Frequently 7

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 7

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency 7

A.9 Explanation of Any Payment of Gift to Respondents 8

A.10 Assurance of Confidentiality Provided to Respondents 8

A.11 Justification for Sensitive Questions 9

A.12 Estimates of Hour Burden Including Annualized Hourly Costs 9

Table A12- 1 -ESTIMATES OF HOUR BURDEN……………………….………………10

Table A12- 2 - ANNUALIZED COST TO RESPONDENT….…………………………...10

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record

keepers 11

A.14 Annualized Cost to the Federal Government 11

A.15 Explanation for Program Changes or Adjustments 11

A.16 Plans for Tabulation and Publication and Project Time Schedule 12

Table A16- 1 -TIMETABLE SUMMARY…………………………………..……… 12

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 12

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 13

List of Attachment(s):

• Attachment 1 - Example of Fielded Applicant Survey Instrument

• Attachment 2 – Example of Reviewer Survey Instrument

SUPPORTING STATEMENT

Voluntary Partner and Customer Satisfaction Surveys,

Center for Scientific Review (CSR), National Institutes of Health (NIH)

A. JUSTIFICATION

A.1 CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY

This is a request for reinstatement of an Office of Management and Budget (OMB) approval, under the Paperwork Reduction Act of 1995, for a generic clearance (OMB No. 0925-0474). This clearance approves voluntary Partner and Customer Satisfaction Surveys for the Center for Scientific Review (CSR), National Institutes of Health (NIH) to implement Executive Order 12862 within the agency.

Executive Order 12862 directs agencies that "provide significant services directly to the public" to "survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services". CSR/NIH provides significant services directly to the public. The mission of the National Institutes of Health (NIH) is to improve human health through biomedical and behavioral research. The NIH implements this mission by conducting intramural research in its own laboratories; supporting extramural research by scientists at universities, medical schools, hospitals, and research institutes; training basic and clinical research investigators; and fostering and supporting biomedical communication as well as public health information dissemination.

The Center for Scientific Review (CSR) is responsible for providing support to the Office of the Director, National Institutes of Health. This support is the receipt, referral, scientific, and technical peer review of applications for: grants that support the advancement of fundamental scientific knowledge to meet the NIH mission of extending healthy life and reducing the burdens of illness and disability, grant opportunities that support research-related training and career development, Small Business Innovation Research (SBIR) grants, and a variety of other awards. CSR plays a pivotal role in assuring NIH funds only the most meritorious biomedical research. Each year, CSR receives nearly 91,000 applications and reviews roughly 66% of them by recruiting roughly 19,000 experts to its study sections¹. Additional scientists serve on other NIH advisory councils, which provide a second level of peer review and make funding recommendations based on priorities set by Congress, DHHS, and the public. For nearly 60 years, this peer review system has enabled NIH to fund cutting-edge research.

The Center for Scientific Review of the National Institutes of Health requests the Office of Management and Budget (OMB) to reinstate generic clearance OMB No. 0925-0474 for surveys of our customers for the period of three years. The OMB approval expires 9/2011. We have ensured these survey activities, which were and are designed to gather and measure customer and partner satisfaction with the CSR's processes and operations, satisfy the requirements and the spirit of Executive Order (EO) 12862. CSR staff has reviewed the OMB “Resource Manual for Customer Surveys.” We are confident all our survey activities meet the requirements and follow the guidelines of this manual. Further, the CSR agrees to adhere to the guidelines set forth in both EO 12862 and in the OMB Manual.

CSR recognizes the generic clearance covers only voluntary surveys of our partners and customers. Participation in these surveys will be strictly voluntary. Personal identifiers will be excluded from all surveys, and additional measures will be taken to ensure anonymity of respondents. Our surveys will be both qualitative and quantitative. The generic clearance is limited to collection of information from current and immediate past customers of CSR.

CSR strives to maintain its leadership as an international model of impartiality, rigor, fairness, and excellence in the peer review of state-of-the-art biomedical research. Input from our customers is an essential component of our efforts to continuously improve our processes and operations. Results gathered in these surveys will be used to help us improve and refine our peer review organization, to refocus the strategies that we use to accomplish our objectives, and to identify areas in need of improvement.

Quality management principles have been integrated into CSR's culture and these surveys will be at the core of CSR's continuing assessment of peer review operations. The data collected from these surveys will assess our customers' satisfaction with CSR's reorganization of grant review study groups, operations, and processes. CSR will present data and outcomes from these surveys to the NIH Advisory Committee and based on feedback from this committee formulate improvement plans and action steps when necessary. To move our quality initiative forward, CSR needs input from its customers.

A.2 PURPOSE AND USE OF THE INFORMATION COLLECTION

This is an extension of an already approved clearance. The primary use for information gathered through voluntary partner and customer surveys is to identify strengths and weaknesses in current services provided by CSR/NIH and to make improvements that are practical and feasible. Obtaining information about the satisfaction of customers is consistent with EO 12862, as well as a good business practice. If CSR does not collect this information, vital feedback regarding customer and partner satisfaction or dissatisfaction with various aspects of revisions to the peer review services will be unavailable. This would inhibit CSR’s ability to develop, implement and refine programs, products, and services in a manner most consistent with the needs of our customers. The information collected in these surveys will be used by CSR personnel: 1) to assess the quality of the changed operations and processes used by CSR to review grant applications; 2) to assess the quality of service provided to our customers; 3) to assess the quality and relevancy of the reorganized scientific review groups; 4) to assist with the design of modifications of these operations, processes, and services, based on customer input; 5) to develop new modes of operation based on customer need; and 6) to obtain customer feedback about the efficacy of implemented modifications. These surveys will almost certainly lead to quality improvement activities that will enhance and/or streamline CSR's operations. The satisfaction surveys are tailored to the specific changes CSR/NIH is considering implementing. CSR will submit specific survey instruments as they become available and will report the results of completed surveys to DHHS and OMB. on a yearly basis.

These satisfaction surveys have been used to implement multiple enhancements to the NIH peer review system. A sample of the applicant survey instrument that has been fielded is included in Attachment 1.

Previously, our satisfaction surveys were used to inform our use of bi-coastal meetings and discounted airfares for participants. Since our last approval, a satisfaction survey was conducted to evaluate the use of our electronic review platform which results in significant cost savings and allows the Center for Scientific Review to enable the participation of those scientists who are unable to travel to peer review meetings (See Attachment 2 enclosed). The knowledge gained from that survey directed our efforts to improve the platform and make it more user friendly even as we increased the number of grant applications reviewed in this medium.

A.3 USE OF INFORMATION TECHNOLOGY AND BURDEN REDUCTION

As appropriate, automated information technology will be used to collect and process information for these surveys. In many instances the most appropriate methodology will involve Internet administration of surveys but some may be written.

A.4 EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION

Each survey will be designed to reflect the specifics of the review related change being assessed. Any potential duplication will be identified in the internal CSR review and approval process. Information about plans for customer and partner surveys will also be shared among CSR components at an early stage so activities can be coordinated.

A.5 IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES

It is not expected that small businesses will be involved in these partner and customer surveys.

A.6 CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY

Surveys will be conducted only at intervals that are considered appropriate to measure the impact of changes implemented as a result of initial satisfaction surveys and to monitor the continued level of performance. In most instances, a satisfaction survey is likely to be conducted on an annual or biennial basis after establishment of a baseline. Collection on a less frequent basis would reduce the practical utility of the information and inhibit the program's ability to monitor changes.

A.7 SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINES OF 5 CFR 1320.5

These surveys will be implemented in a manner fully consistent with 5 CFR 1320.5

A.8 COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE AGENCY

This proposed information collection was previously published in the Federal Register on July 22, 2011 (Vol. 76, No. 141, p. 44020. A single response was received in response to the 60 day notice from a person identifying herself as Jane Q Public. This person felt that ‘projects needed to be submitted to the public since omb approves all that comes before it..” A response was sent to this person indicating that the purpose of conducting these surveys was to solicit feedback so that ways improve the efficiency and effectiveness of the way NIH reviews grant applications could be found.

CSR may use focus groups and other qualitative information collection activities to identify areas of interest and concern to customers and then build the design and content of its quantitative surveys upon this qualitative input. CSR will call upon its in-house statistical staff and contractors in developing survey plans. During the 3 years since the previous submission was approved, CSR has utilized the services of a contractor: Luba Katz of ABT Associates of Cambridge, MA.

A.9 EXPLANATION OF ANY PAYMENT OF GIFT TO RESPONDENTS

There will be a need for nominal remuneration to focus group participants who are asked to leave their usual location and travel to a central location to compensate them for the time and inconvenience required. The level of remuneration will depend on the amount of respondent time and expense projected for each focus group.

A.10 ASSURANCE OF CONFIDENTIALITY PROVIDED TO RESPONDENTS

A sample of reviewers and applicants from each fiscal year will be asked to participate in one of the surveys. Those records are kept by CSR. However, the responses to the questionnaire surveys are entirely anonymous and have no identifiers to link them to individual respondents. Thus, their privacy is ensured by the anonymity. This confidentiality will be explained to respondents. The respondents will be given the Internet address and a unique access code and password to the survey web site. The system is programmed so only a respondent who has a password can access a survey form. The password will expire automatically after the form is completed and submitted. The survey will not contain any identifying information. An independent contractor will collect and collate the surveys electronically.

Even the email address, which could possibly identify individual respondents, will be separated from the survey responses as they are entered into the data set. As the contractor's computer server receives the completed surveys, the identity of the responding computer is also eliminated. The electronic responses will be downloaded automatically into the data set. Contractors will perform the analyses. CSR staff will see only aggregate data. No individual information will be able to be extracted. Additionally, respondents will be informed that their participation is voluntary and that no consequences will be associated with not responding or with responding. This procedure will be explained to respondents. Individuals contacted in the course of these surveys will be assured of the confidentiality of their replies under 42 USC 1306, 20 CFR 401 and 422, 5 USC 552 (Freedom of Information Act), 5 USC 552a (Privacy Act of 1974), Privacy Act System of Records Notice: 09-25-036, and OMB Circular No.A-130.

A.11 JUSTIFICATION FOR SENSITIVE QUESTIONS

These voluntary customer surveys will not involve personal information of a sensitive nature.

A.12 ESTIMATES OF HOUR BURDEN INCLUDING ANNUALIZED HOURLY COSTS

Respondents are expected to be a mix of adult scientists connected to the research community. It is estimated that e-mail/mail surveys will take 15 minutes to complete. The annual hour burden is, therefore, estimated to be 1,250 hours for approximately 5000 respondents in each of the 3 fiscal years. Focus groups: It is projected that in each year of this approval approximately 5 focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 2.5 hours and will include approximately 15 members. The focus groups are expected to take 2.5 hours to complete. The annual burden for the 5 annual focus groups is estimated to be 187.5 hours for each of 3 fiscal years. Estimated costs to the respondents consist entirely of their time. Costs for time were estimated using a rate of $40.00 per hour for adult science professionals. The estimated annual cost burden for respondents for each year for which the generic clearance is requested is $57,500 for FY 2011, $57,500 for FY 20012, and $57,500 for FY 2013. Thus, the total combined FY 2011 to FY 2013 potential hour burden on the respondents is estimated to be 4,312.5 hours for 15,225 respondents for all surveys that would be conducted under this generic clearance. If all planned surveys are conducted, the total three year cost is estimated to $172,500.

TABLE A12-1 ESTIMATES OF HOUR BURDEN

Type of Respondent	Number of Respondents	Frequency of Response	Average Time per Response (Hr)	Total Annual Hour Burden
Adult scientific professionals (via Mail/Telephone/Internet)	5000	1	0.25	1250
Adult scientific professional (via focus groups)	75	1	1	188
Total	5075	1		1438

TABLE A12-2 ESTIMATES OF ANNUALIZED COST TO RESPONDENTS

Type of Respondents	Number of Respondents	Frequency of Response	Average Burden Per Response	Hourly Wage Rate	Respondent Cost
Adult science professionals - Focus groups	75	1	1 hr	$40.00	$3,000.00
Adult science professionals -mail/telephone/ e-mail surveys	5,000	1	0.25 hr	$40.00	$50,000.00
Total…………………					$53,00.00

A.13 ESTIMATE OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS

Focus group participants will be reimbursed for any travel or incidental costs associated with traveling to a central location for interview. Except for focus groups, costs to respondents will be limited to their time to provide the requested information.

A.14 ANNUALIZED COST TO THE FEDERAL GOVERNMENT

The surveys and focus groups are likely to be carried out under contract. Assuming contract costs for each survey are $8,000 - $10,500 (approximately), and for each focus group are $937.50 (5 focus groups per fiscal year), total contract costs could average approximately $92,500.00 per year (10 x $9,250.). An additional annual cost of about $83,287.70 for agency staff would be associated with this, assuming 10 surveys and 5 focus groups per fiscal year with 2 GS14/5 program analysts or project officers ($96,572 annual salary) and 1.5 weeks of time per project (22.5 weeks per staff person each fiscal year). The total annual cost to the government is estimated to be $175,787.70.

A.15 EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS

This is the same satisfaction survey of Center for Scientific Review's customers approved by OMB in 1999 under OMB No. 0925-0474. CSR is requesting an extension of No. 0925-0474 to continue the activity of assessing customer satisfaction with the organizational and process changes made by CSR. This reorganization of the CSR peer review study groups is being accomplished incrementally. Reorganization began in 2000 and was largely completed by2010. The first phase of reorganization involved the peer review structure for the review of neuroscience applications. The original OMB approval was for 5.855 responses and 1,932 burden hours. However, there are more initiatives being undertaken by CSR each year (which results in additional respondents), the survey lengths will be comparable. For the e-mail surveys, the anticipated annual hour burden for FY 2011 through FY2013 is 1,250 hours for approximately 5000 respondents in each of the 3 fiscal years. Focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 1 hours per person and will include approximately 15 members. CSR expects to have 5 focus groups per year. The focus groups are expected to take 1 hour per respondent to complete. The annual burden for focus groups is estimated to be 187.5 hours for each of 3 fiscal years.

A.16 PLANS FOR TABULATION AND PUBLICATION AND PROJECT TIME SCHEDULE

The satisfaction surveys that will be collected under the generic clearance will be conducted only once per review cycle. The projected survey schedule is: 5-10 surveys per year. Topics over the 3 years will include: scientific review using Asynchronous Electronic Discussion (AED), virtual and Telepresence reviews , scientific review utilizing Editorial Boards, Shortening Review Applications, automated referral of grant applications, modes of review for orphan applications, etc.

The surveys will be distributed electronically. Information collected from the surveys will be analyzed within two months of receipt. Analysis plans will be specific to the goals and designs of the individual surveys. For all types of surveys, the analyses will be mostly descriptive, rather than inferential. The results of any analysis will be disseminated to key management officials at CSR, NIH management, CSR members and chairs, NIH principle investigators, and CSR employees within six months of the survey's completion. Relational databases will be designed and analysis-ready data inputted. Analyses will include cross tabulation of the questions that indicate consistency and validity of responses. Analysis result tables will be designed and produced to present the information in an aggregate format in order to maintain confidentiality. Content analysis will be performed on the narrative responses. A narrative report with accompanying charts will be provided to CSR management.

TABLE 16.1 TIMETABLE SUMMARY
Month 1	Construct relational databases and produce coded survey instruments.
Month 2	Construct survey collection Internet website
Month 3	Post survey instrument and send email invitations to participate
Month 4	Post survey instrument and send email invitations to participate
Month 5	Download survey responses and input data
Month 6&7	Analyze coded and narrative data and write reports for CSR management
Month 8	CSR management study report and decide if changes need to be implemented.
Month 9	CSR disseminates results and any changes that are to be made

A17 REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE

No exemption is being requested.

A.18 EXCEPTIONS TO CERTIFICATION FOR REDUCTION ACT SUBMISSIONS

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

1 * Data from recent years 2008-2010 and rounded to the nearest 1,000.

12