Supporting Statement A July 10

11

Supporting Statement A for

GENERIC CLEARANCE FOR SURVEYS

OF CUSTOMERS AND PARTNERS OF

THE CENTER FOR SCIENTIFIC REVIEW (CSR)

NATIONAL INSTITUTES OF HEALTH

Date: July 10, 2008

Name: Andrea Kopstein, Ph.D., MPH

Address: 6701 Rockledge Drive, Room 3030, Bethesda, MD, 20892-7776

Telephone: 301-435-1133

Fax: 301-480-3965

Email: [email protected]

Table of Contents

A. JUSTIFICATION 4

A.1 Circumstances Making the Collection of Information

Necessary 5

A.2. Purpose and Use of the Information COLLECTION 5

A.3 Use of Information Technology and Burden Reduction 6

A.4 Efforts to Identify Duplication and Use of Similar Information 6

A.5 Impact on Small Businesses or Other Small Entities 6

A.6 Consequences of Collecting the Information Less Frequently 6

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 6

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency 6

A.9 Explanation of Any Payment of Gift to Respondents 7

A.10 Assurance of Confidentiality Provided to Respondents 7

A.11 Justification for Sensitive Questions 8

A.12 Estimates of Hour Burden Including Annualized Hourly Costs 8

Table A12- 1 -ESTIMATES OF HOUR BURDEN……………………….………………8

Table A12- 2 - ANNUALIZED COST TO RESPONDENT….…………………………...9

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record

keepers 9

A.14 Annualized Cost to the Federal Government 9

A.15 Explanation for Program Changes or Adjustments 10

A.16 Plans for Tabulation and Publication and Project Time Schedule 10

Table A16- 1 -TIMETABLE SUMMARY…………………………………..…………10

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 11

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 11

List of Attachment(s):

• Attachment 1 - Examples of Customer Satisfaction Surveys

SUPPORTING STATEMENT

Voluntary Partner and Customer Satisfaction Surveys,

Center for Scientific Review (CSR), National Institutes of Health

A. JUSTIFICATION

A.1 CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY

This is a request for reinstatement of an Office of Management and Budget (OMB) approval, under the Paperwork Reduction Act of 1995, for a generic clearance (OMB No. 0925-0474). This clearance approves voluntary Partner and Customer Satisfaction Surveys for the Center for Scientific Review (CSR), National Institutes of Health (NIH) to implement Executive Order 12862 within the agency.

Executive Order 12862 directs agencies that "provide significant services directly to the public" to "survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services". CSR/NIH provides significant services directly to the public. The mission of the National Institutes of Health (NIH) is to improve human health through biomedical and behavioral research. The NIH implements this mission by conducting intramural research in its own laboratories; supporting extramural research by scientists at universities, medical schools, hospitals, and research institutes; training basic and clinical research investigators; and fostering and supporting biomedical communication as well as public health information dissemination.

The Center for Scientific Review (CSR) is responsible for providing support to the Office of the Director, National Institutes of Health. This support is the receipt, referral, scientific, and technical peer review of applications for: grants that support the advancement of fundamental knowledge about the nature and behavior of living systems to meet the NIH mission of extending healthy life and reducing the burdens of illness and disability, grant opportunities that support research-related training and career development, Small Business Innovation Research (SBIR) grants, and a variety of other awards. CSR plays a pivotal role in assuring NIH funds only the most meritorious biomedical research. CSR receives nearly 80,000 applications a year and recruits over 18,000 external experts to review its portion of them in its study sections. Additional scientists serve on other NIH advisory councils, which provide a second level of peer review and make funding recommendations based on priorities set by Congress, DHHS, and the public. For nearly 60 years, this peer review system has enabled NIH to fund cutting-edge research.

The Center for Scientific Review of the National Institutes of Health requests the Office of

Management and Budget (OMB) reinstate generic clearance OMB No. 0925-0474 for surveys of our customers for the period of three years. The original OMB approval expired 9/2007. We have ensured these survey activities, which were and are designed to gather and measure customer and partner satisfaction with the CSR's processes and operations, satisfy the requirements and the spirit of Executive Order (EO) 12862. CSR staff has reviewed the OMB “Resource Manual for Customer Surveys.” We are confident all our survey activities meet the requirements and follow the guidelines of this manual. Further, the CSR agrees to adhere to the guidelines set forth in both EO 12862 and in the OMB Manual.

CSR recognizes the generic clearance covers only voluntary surveys of our partners and customers. Participation in these surveys will be strictly voluntary. Personal identifiers will be excluded from all surveys, and additional measures will be taken to ensure anonymity of respondents. Our surveys will be both qualitative and quantitative. The generic clearance is limited to collection of information from current and immediate past customers of CSR.

CSR strives to maintain its leadership as an international model of impartiality, rigor, fairness, and excellence in the peer review of state-of-the-art biomedical research. Input from our customers is an essential component of our efforts to continuously improve our processes and operations. Results gathered in these surveys will be used to help us improve and refine our peer review organization, to refocus the strategies that we use to accomplish our objectives, and to identify areas in need of improvement.

Quality management principles have been integrated into CSR's culture and these surveys will be at the core of CSR's continuing assessment of peer review operations. The data collected from these surveys will assess our customers' satisfaction with CSR's reorganization of grant review study groups, operations, and processes. CSR will present data and outcomes from these surveys to the NIH Advisory Committee and based on feedback from this committee formulate improvement plans and action steps when necessary. To move our quality initiative forward, CSR needs input from its customers.

A.2 PURPOSE AND USE OF THE INFORMATION COLLECTION

The primary use for information gathered through voluntary partner and customer surveys is to identify strengths and weaknesses in current services provided by CSR/NIH and to make improvements that are practical and feasible. Obtaining information about the satisfaction of customers is consistent with EO 12862, as well as a good business practice. If CSR does not collect this information, vital feedback regarding customer and partner satisfaction or dissatisfaction with various aspects of revisions to the peer review services will be unavailable. This would inhibit CSR’s ability to develop, implement and refine programs, products, and services in a manner most consistent with the needs of our customers. The information collected in these surveys will be used by CSR personnel: 1) to assess the quality of the changed operations and processes used by CSR to review grant applications; 2) to assess the quality of service provided to our customers; 3) to assess the quality and relevancy of the reorganized scientific review groups; 4) to assist with the design of modifications of these operations, processes, and services, based on customer input; 5) to develop new modes of operation based on customer need; and 6) to obtain customer feedback about the efficacy of implemented modifications. These surveys will almost certainly lead to quality improvement activities that will enhance and/or streamline CSR's operations. The satisfaction surveys are tailored to the specific changes CSR/NIH is considering implementing. CSR will submit specific survey instruments as they become available and will report the results of completed surveys to DHHS and OMB on a yearly basis.

These satisfaction surveys have been used to implement multiple enhancements to the NIH peer review system. An enhancement implemented since the last approval is bi-coastal review. A large number of CSR review meetings are now held on the West Coast as opposed to always holding these meetings in the Washington D.C. area. This resulted in tremendous financial saving for the government because of more competitive hotel rates and lower per diems for the many reviewers CSR travels to meetings. Another change which utilized satisfaction surveys is the use of discounted airfare tickets. Surveying our partners allow identification of a modification to this new policy that would make using discount tickets acceptable to the reviewers. There is very little need for rescheduling airline tickets for reviewers because going to meetings is on a very fixed schedule. On occasion, review meetings end early enough to allow reviewers to fly on an earlier flight. With this information, NIH allows reviewers one paid change to their flights. The use of discounted tickets has also resulted in significant financial savings to the government. A sample of the travel satisfaction survey is included in Attachment 1.

A.3 USE OF INFORMATION TECHNOLOGY AND BURDEN REDUCTION

As appropriate, automated information technology will be used to collect and process information for these surveys. In many instances the most appropriate methodology will involve Internet administration of surveys but some may be written.

A.4 EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION

Each survey will be designed to reflect the specifics of the review related change being assessed. Any potential duplication will be identified in the internal CSR review and approval process. Information about plans for customer and partner surveys will also be shared among CSR components at an early stage so activities can be coordinated.

A.5 IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES

It is not expected that small businesses will be involved in these partner and customer surveys.

A.6 CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY

Surveys will be conducted only at intervals that are considered appropriate to measure the impact of changes implemented as a result of initial satisfaction surveys and to monitor the continued level of performance. In most instances, a satisfaction survey is likely to be conducted on an annual or biennial basis after establishment of a baseline. Collection on a less frequent basis would reduce the practical utility of the information and inhibit the program's ability to monitor changes.

A.7 SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINES OF 5 CFR 1320.5

These surveys will be implemented in a manner fully consistent with 5 CFR 1320.5

A.8 COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE AGENCY

The 60 day notice required in 5 CFR 1320.8(d) was published in the Federal Register on April 17, 2008 (Volume 73, Number 75), pages 20927-20928. No comments were received regarding this 60 day notice.

The 30 day notice required in 5 CFR 1320.8(d) was published in the Federal Register on July 2, 2008 (Volume 73, Number 128), pages 37971-37972. On July 3, 2008, a comment was received from an angry citizen who views partner surveys to be a waste of federal dollars. In addition to feeling taxpayers pay for federal employee “vacations across the country” the author of the comment wants all the management at the Center for Scientific Review (NIH) to be fired. She expressed concerns about out of control federal spending because she feels no satisfactory inroads to health are being achieved in the United States. The author was particularly concerned about autism and wants “to shut the whole thing down.” A response was sent to the author of this comment. The reply talked about the importance of peer review for funding the most promising research and the best scientists. The role of partner surveys as the National Institutes of Health (NIH) is piloting changes to its peer review system was also explained. The response also mentioned some of the important medical advances supported with federal dollars (in the form of peer-reviewed grants from the NIH) in recent years.

CSR may use focus groups and other qualitative information collection activities to identify areas of interest and concern to customers and then build the design and content of its quantitative surveys upon this qualitative input. CSR will call upon its in-house statistical staff and contractors in developing survey plans. During the 3 years since the previous submission was approved, CSR has utilized the services of a contractor:

Teresa R. Kowalczyk, senior associate at McManis & Monsalve in Manassas, Virginia.

A.9 EXPLANATION OF ANY PAYMENT OF GIFT TO RESPONDENTS

There will be a need for nominal remuneration to focus group participants who are asked to leave their usual location and travel to a central location to compensate them for the time and inconvenience required. The level of remuneration will depend on the amount of respondent time and expense projected for each focus group.

A.10 ASSURANCE OF CONFIDENTIALITY PROVIDED TO RESPONDENTS

A sample of reviewers and applicants from each fiscal year will be asked to participate in one of the surveys. Those records are kept by CSR. However, the responses to the questionnaire surveys are entirely anonymous and have no identifiers to link them to individual respondents. Thus, their privacy is ensured by the anonymity. This confidentiality will be explained to respondents. The respondents will be given the Internet address and a unique access code and password to the survey web site. The system is programmed so only a respondent who has a password can access a survey form. The password will expire automatically after the form is completed and submitted. The survey will not contain any identifying information. An independent contractor will collect and collate the surveys electronically.

Even the email address, which could possibly identify individual respondents, will be separated from the survey responses as they are entered into the data set. As the contractor's computer server receives the completed surveys, the identity of the responding computer is also eliminated. The electronic responses will be downloaded automatically into the data set. Contractors will perform the analyses. CSR staff will see only aggregate data. No individual information will be able to be extracted. Additionally, respondents will be informed that their participation is voluntary and that no consequences will be associated with not responding or with responding. This procedure will be explained to respondents. Individuals contacted in the course of these surveys will be assured of the confidentiality of their replies under 42 USC 1306, 20 CFR 401 and 422, 5 USC 552 (Freedom of Information Act), 5 USC 552a (Privacy Act of 1974), Privacy Act System of Records Notice: 09-25-036, and OMB Circular No.A-130.

A.11 JUSTIFICATION FOR SENSITIVE QUESTIONS

These voluntary customer surveys will not involve personal information of a sensitive nature.

A.12 ESTIMATES OF HOUR BURDEN INCLUDING ANNUALIZED HOURLY COSTS

Respondents are expected to be a mix of adult scientists connected to the research community.

It is estimated that e-mail/mail surveys will take 15 minutes to complete. The annual hour burden is, therefore, estimated to be 1,250 hours for approximately 5000 respondents in each of the 3 fiscal years. Focus groups: It is projected that in each year of this approval approximately 5 focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 2.5 hours and will include approximately 15 members. The focus groups are expected to take 2.5 hours to complete. The annual burden for the 5 annual focus groups is estimated to be 187.5 hours for each of 3 fiscal years. Estimated costs to the respondents consist entirely of their time. Costs for time were estimated using a rate of $40.00 per hour for adult science professionals. The estimated annual cost burden for respondents for each year for which the generic clearance is requested is $57,500 for FY 2008, $57,500 for FY 2009, and $57,500 for FY 2010. Thus, the total combined FY 2008 to FY 2010 potential hour burden on the respondents is estimated to be 4,312.5 hours for 15,225 respondents for all surveys that would be conducted under this generic clearance. If all planned surveys are conducted, the total three year cost is estimated to $172,500.

TABLE A12-1 ESTIMATES OF HOUR BURDEN

Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Response	Annual Hour Burden
Adult Science Professionals - Focus Groups	75	1	2.5 hours	187.5 hours
Adult Science Professionals - Mail/Telephone/e-mail Surveys	5,000	1	0.25 hours	1,250 hours
Totals	5,075			1,437.5 hours per year

TABLE A12-2 ESTIMATES OF ANNUALIZED COST TO RESPONDENTS

Type of Respondents	Number of Respondents	Frequency of Response	Average Burden Per Response	Hourly Wage Rate	Respondent Cost
Adult science professionals - Focus groups	75	1	2.5 hours	$40.00	$7,500.00
Adult science professionals -mail/telephone/ e-mail surveys	5,000	1	0.25 hours	$40.00	$50,000.00
Total…………………					$57,500.00

A.13 ESTIMATE OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS

Focus group participants will be reimbursed for any travel or incidental costs associated with traveling to a central location for interview. Except for focus groups, costs to respondents will be limited to their time to provide the requested information.

A.14 ANNUALIZED COST TO THE FEDERAL GOVERNMENT

The surveys and focus groups are likely to be carried out under contract. Assuming contract costs for each survey are $8,000 - $10,500 (approximately), and for each focus group are $937.50 (5 focus groups per fiscal year), total contract costs could average approximately $92,500.00 per year (10 x $9,250.). An additional annual cost of about $83,287.70 for agency staff would be associated with this, assuming 10 surveys and 5 focus groups per fiscal year with 2 GS14/5 program analysts or project officers ($96,572 annual salary) and 1.5 weeks of time per project (22.5 weeks per staff person each fiscal year). The total annual cost to the government is estimated to be $175,787.70.

A.15 EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS

This is the same satisfaction survey of Center for Scientific Review's customers approved by OMB in 1999 under OMB No. 0925-0474. CSR is requesting a reinstatement of No. 0925-0474 to continue the activity of assessing customer satisfaction with the organizational and process changes made by CSR. This reorganization of the CSR peer review study groups is being accomplished incrementally. The reorganization began in 2000 and will not be complete until 2010. The first phase of reorganization involved the peer review structure for the review of neuroscience applications. The original OMB approval was for 5.855 responses and 1,932 burden hours. There are more initiatives being undertaken by CSR each year (which results in additional respondents), the survey lengths will be comparable. For the e-mail surveys, the anticipated annual hour burden for FY 2008 through FY2010 is 1,250 hours for approximately 5000 respondents in each of the 3 fiscal years. Focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 2.5 hours and will include approximately 15 members. CSR expects to have 5 focus groups per year. The focus groups are expected to take 2.5 hours per respondent to complete. The annual burden for focus groups is estimated to be 187.5 hours for each of 3 fiscal years.

A.16 PLANS FOR TABULATION AND PUBLICATION AND PROJECT TIME SCHEDULE

The satisfaction surveys that will be collected under the generic clearance will be conducted only once per review cycle. The projected survey schedule is: 5-10 surveys per year. Topics over the 3 years will include: scientific review using Asynchronous Electronic Discussion (AED), virtual review, scientific review utilizing Editorial Boards, Shortening Review Applications, automated referral of grant applications, modes of review for orphan applications, etc.

The surveys will be distributed electronically. Information collected from the surveys will be analyzed within two months of receipt. Analysis plans will be specific to the goals and designs of the individual surveys. For all types of surveys, the analyses will be mostly descriptive, rather than inferential. The results of any analysis will be disseminated to key management officials at CSR, NIH management, CSR members and chairs, NIH principle investigators, and CSR employees within six months of the survey's completion. Relational databases will be designed and analysis-ready data inputted. Analyses will include cross tabulation of the questions that indicate consistency and validity of responses. Analysis result tables will be designed and produced to present the information in an aggregate format in order to maintain confidentiality. Content analysis will be performed on the narrative responses. A narrative report with accompanying charts will be provided to CSR management.

TABLE 16.1 TIMETABLE SUMMARY
Month 1	Construct relational databases and produce coded survey instruments.
Month 2	Construct survey collection Internet website
Month 3	Post survey instrument and send email invitations to participate
Month 4	Post survey instrument and send email invitations to participate
Month 5	Download survey responses and input data
Month 6&7	Analyze coded and narrative data and write reports for CSR management
Month 8	CSR management study report and decide if changes need to be implemented.
Month 9	CSR disseminates results and any changes that are to be made

A17 REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE

No exemption is being requested.

A.18 EXCEPTIONS TO CERTIFICATION FOR REDUCTION ACT SUBMISSIONS

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

11