Revised SSA for NLM Generic Clearance to Conduct Voluntary Customer/Partner Surveys

Supporting Statement A for

Request for Generic Clearance to Conduct Voluntary Customer/Partner Surveys (NLM)

June 5, 2012 (Revised Sept 2014 for Non-Material / Non-Substantive Change Request)

Name: David Sharlip

Address: National Library of Medicine

8600 Rockville Pike

Bldg. 38A, Rm. B2N12

Bethesda, MD 20894-3844

Telephone: 301-402-9680

Fax: 301-402-0871

Email: [email protected]

Generic Clearance to Conduct Voluntary Customer/Partner Surveys (NLM)

Table of contents

A. JUSTIFICATION 3

A.1 Circumstances Making the Collection of Information Necessary 3

A.2. Purpose and Use of the Information collection 4

A.3 Use of Information Technology and Burden Reduction 5

A.4 Efforts to Identify Duplication and Use of Similar Information 6

A.5 Impact on Small Businesses or Other Small Entities 6

A.6 Consequences of Collecting the Information Less Frequently 6

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 6

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency 6

A.9 Explanation of Any Payment of Gift to Respondents 6

A.10 Assurance of Confidentiality Provided to Respondents 7

A.11 Justification for Sensitive Questions 7

A.12 Estimates of Hour Burden Including Annualized Hourly Costs 7

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record

keepers 8

A.14 Annualized Cost to the Federal Government 8

A.15 Explanation for Program Changes or Adjustments 8

A.16 Plans for Tabulation and Publication and Project Time Schedule 8

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 9

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 9

List of Attachments:

Attachment 1 - Example of previously approved NLM Customer/Partner Surveys Request

Attachment 2 - Example of previously approved NLM Survey Instrument

Generic Clearance to Conduct Voluntary Customer/Partner Surveys (NLM)

A. JUSTIFICATION

A.1 Circumstances Making the Collection of Information Necessary

This is a request that the Office of Management and Budget (OMB) approve, under the Paperwork Reduction Act, an extension of generic clearance OMB 0925-0476 for the National Library of Medicine (NLM) to conduct a number of customer satisfaction surveys to implement Executive Order 12862.  This order directs federal agencies that provide significant services directly to the public to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services.  The order also requires future surveys to continually assess the level of customer satisfaction.  The current OMB authorization to conduct surveys approved under this generic clearance will expire on June 30, 2012.   

In addition to the Executive Order, the NLM is legislatively mandated by 42 USC 286 to assist the advancement of medical and related sciences and to aid the dissemination and exchange of scientific and other information important to the progress of medicine and to the public health. This mandate specifically directs the NLM to promote the use of computers and telecommunications by health professionals (including health professionals in rural areas) for the purpose of improving access to biomedical information for health care delivery and medical research.

In 1994, the NLM was designated a "Federal Reinvention Laboratory" with a major objective of improving its methods of delivering information to the public.  At a minimum, necessary elements in improving the delivery of information include; (1) development of easy-to-use access and delivery mechanisms that promote the public's understanding of health information, drawing on research in lay terminology, graphical and multimedia presentations;   (2) assisting those providing health information to the public to make effective use of electronic services through Internet connections, training, and other means, with an emphasis on those serving minority groups, low income populations, and seniors;  (3) promoting integrations of NLM services with other electronic services covering regional, state, or local health information; and  (4) conducting and supporting research, development, and evaluation of the public's health information needs, information seeking behavior and learning styles, information systems that meet the public's needs, and the impact of access to information.

NLM has become an international leader in health informatics research and development, especially in consumer health informatics. As a result, NLM needs to remain contemporary in consumer health informatics research by utilizing research methods that yield a better understanding of the predictors of consumer satisfaction. Without ongoing insights into the predictors of consumer satisfaction, NLM will lack the research findings to make evidence-based changes in the content, design and editorial management of its consumer Web sites and will not optimally serve the public.

On January 21, 2009, the President issued a memorandum calling for the establishment of “a system of transparency, public participation, and collaboration.” The memorandum required an Open Government Directive to be issued by the Director of the Office of Management and Budget (OMB). Following the President’s memorandum, OMB’s Open Government Directive required a series of measures to promote the commitments to transparency, participation, and collaboration to include the availability and uses of “generic” Information Collection Requests (ICRs).

In later guidance issued under the Open Government Directive, it was made clear that clearances of generic ICRs do provide a significantly streamlined process by which agencies may obtain OMB’s approval for particular information collections including customer satisfaction surveys, focus groups, contests, and website satisfaction surveys.

One of the most effective means of determining if NLM is providing satisfactory services to its customers or partners is through periodic voluntary satisfaction surveys that can be activated quickly.  Getting customer/partner input has and will continue to facilitate resource allocation.  

A.2 Purpose and Use of the Information Collection

NLM will collect and use information gathered through these surveys to identify strengths and weaknesses in current service provisions and to make improvements that are practical and feasible.  Information from the customer and/or partner surveys will be used to plan and direct resources and efforts to improve or maintain NLM's high quality of service to affected public.  To assess the needs of rurally situated health care providers, NLM needs to conduct periodic surveys to ensure that these providers have the ability to access the wealth of information in the Library's many databases and programs.  If this information is not collected, vital feedback, regarding customer and/or partner satisfaction with various aspects of NLM's services will be unavailable. As part of NLM’s mission to provide ongoing outreach programs in collaboration with our customers and partners, distance education learning has been developed for school students interested in careers in healthcare professions.

Without ongoing insights into the predictors of consumer satisfaction, NLM will lack the research findings to make evidence-based changes in the content, design and editorial management of its consumer Web sites and cannot optimally serve the public. Some of the predictors of consumer satisfaction identified include content quality, usability and design; consumer health information seeking habits on the Internet; prior online experience and health knowledge; as well as demographic information. The ongoing assessments of broad consumer attitudes about health, wellness and motivational inclinations remain highly important in predicting consumer satisfaction of consumer informatics websites as more traditional demographic variables such as gender, education level, age and geographic location. To better assess the effectiveness and customer/partner satisfaction in administering our distance education outreach engagements, NLM will collect baseline career information from students during the course of their education mentoring programs. These measures or survey instruments will typically be student career essays, a health career knowledge-interest inventory, and general career inventories.

To explore consumer attitudes as predictors of consumer satisfaction, NLM may employ some social science methods that are widely advocated within the health communication, mass communication and consumer health informatics research literature. These methods may include occasionally asking respondents to sort through opinion statements with which they agree or disagree and then, specify what opinion statements they agree or disagree with the most. NLM will use a second level discrimination in surveys where the selected sample is derived from experienced computer users.  The discrimination approach to measure attitudes is most simplified by web-based software, therefore NLM will use this approach for online or computer work station-based consumer satisfaction surveys. The use of this approach provides better insights into the attitudes that predict consumer satisfaction of health information Web sites, which keeps NLM at the forefront of consumer health informatics research and development.

One example of how NLM staff used feedback from customers and partners to improve services was the NLM Cervigram Finder Physician Survey where voluntary results gathered from focus groups helped determine the interest of specialized physicians in accessing the NLM Cervigram Finder and Boundary Marking Tool, as well as determine the points of interest most relevant to work conducted by these specialists.

The decision to conduct this survey was made because it was possible to reach out to a targeted audience of specialized physicians attending meetings held biennially and heavily attended by those physicians who would be most likely to benefit from the services provided through the NLM Cervigram Finder and Boundary Marking Tool project, as well as determine the points of interest most relevant to work conducted by these specialists. Receiving specific feedback from these specialists on the functionality of the program proved to be invaluable to the success of the projects.

An example of this approved generic survey conducted under the current clearance is provided (see Attachment 1) along with the sample survey instrument (Attachment 2). 

According to OMB guidelines for generic clearances for voluntary customer/partner satisfaction surveys, the NLM will establish an independent review process to assure the development and implementation of high quality customer/partner surveys within the NLM.  It is understood that if OMB approves this request for extension of generic clearance, individual survey instruments must be submitted to OMB prior to actual use.

A.3 Use of Improved Information Technology and Burden Reduction

As appropriate, automated information technology will be used to collect and process information for these surveys to reduce the burden on the public.  For example, the use of secondary discrimination techniques to better assess consumer attitudes is dependent on the use of information technology to reduce the burden on the respondent and as aforementioned, often shortens the length of survey instruments. The majority of the surveys will be web-based or using focus groups involving written or oral responses to brief questionnaires.

A.4 Efforts to Identify Duplication and Use of Similar Information

NLM information collection surveys conducted after the approval of previous generic clearance under OMB 0925-0476 were designed to address specific programs and databases and to obtain feedback for desired improvements to present systems and programs.  There is a need for continual input from customers and partners and as NLM has incorporated suggestions from earlier surveys into procedures, it is necessary to ask if the improvements are efficacious.  The extension of this generic clearance will enable NLM to determine if there have been improvements in service that resulted from the previous feedback customers provided.

A.5 Impact on Small Businesses or Other Small Entities

Small businesses or other small entities may be involved in these efforts, but NLM will keep the burden on them to a minimum with short, user-friendly surveys that will be strictly voluntary.  Therefore, there will not be a significant impact on small businesses.

A.6 Consequences of Collecting the Information Less Frequently

NLM will conduct surveys only at intervals considered appropriate to measure the impact of changes because of initial satisfaction surveys and to monitor the level of performance. In most cases, the NLM likely will conduct satisfaction surveys annually or biennially after the establishment of a baseline. Collection on a less frequent basis than annually or biennially might reduce the practical utility of the information and inhibit the NLM's ability to monitor changes.

 A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

These surveys will be implemented in a manner that fully complies with 5 CFR 1320.5.

 A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The information collection proposed in this statement was originally published in the Federal Register on April 2, 2012 (Vol. 77, No. 63, p.19673) and allowed 60-days for public comment. One public comment was received which was not germane to the specific purpose of the notice.

NLM programs will use routine contacts with customers and partners and other qualitative information collection activities to identify areas of interest and concern to customers.  NLM will utilize in-house staff with expertise in statistics and the staff of contractors in developing survey plans.  As needed, NLM may also utilize the statistical resources of the National Center for Health Statistics, which has a questionnaire design laboratory.  Additional support may be sought from outside experts to help in design and implementation of the surveys.

A.9 Explanation of Any Payment or Gift to Respondents

There may be a need for nominal remuneration to survey participants (e.g., focus groups).

The level of remuneration will depend on the amount of respondent time and projected expenses for each survey participant.

A.10 Assurance of Confidentiality Provided to Respondents

Individual respondents will not be identified and participation will be strictly voluntary. Respondents will be assured that neither their participation/non-participation nor any responses to items will have any effect on their eligibility for or receipt of services.

A.11 Justification for Sensitive Questions

No questions will be asked of a personal or sensitive nature.

 A.12 Estimates of Hour Burden Including Annualized Hourly Costs

The total estimated burden hours imposed by the proposed collections is provided in Table A. 12-1 below. NLM projects up to 30 customer surveys will be conducted over the duration of the 3-year clearance period with a sample size ranging between 10 and 5000 customers and a total of 15,000 respondents.   

A.12-1 Estimates of Hour Burden
Types of Respondents	Number  of Respondents	Frequency of  Response	Average time per response	Annual Hour Burden
Researchers, Physicians, Other Health Care Providers, Librarians, Students, General Public	15,000	1	.150	2,250

The proposed surveys will be completed by a range of members of the public, including researchers, physicians, librarians, other health care providers, students, and the general public.

Cost has been estimated using the U.S. Department of Labor (DOL) Bureau of Labor Statistics occupational employment statistics http://www.bls.gov/oes/data.htm

The total estimated cost to respondents is $62,010 over a 3-year period. Table A.12-2 below provides the estimated annualized cost to respondents.

A-12-2 Annualized Cost to Respondents

Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Respondents	Hourly Wage Rate	Respondent Cost
Researchers, Physicians, Other Health Care Providers, Librarians, Students, General Public *	5,000	1	0.150	$27.56	$20,670
Totals	5,000				$20,670

* Not all respondents have an associated cost

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There is no other annual cost burden to report for respondents or record keepers for this collection.

There are no capital and startup costs/operation and maintenance costs associated with these surveys.

 A.14 Annualized Cost to the Federal Government

The approximate annualized cost to the government for this data collection effort is $36,000.   An average of ten (10) surveys are projected to be conducted per year with the total costs estimated to be $6,000 for administrative and technical support in processing individual survey instruments through clearance. 

A.15 Explanation for Program Changes or Adjustments

This is a request for an extension of NLM’s current generic clearance authorization.

A reduction of 1,350 burden hours is being requested to accommodate the proposed surveys to be conducted during the next 3-year period.

 
A.16 Plans for Tabulation and Publication and Project Time Schedule

NLM tabulates results immediately after completion of future surveys and strives to publish selected findings in pertinent refereed journals and other edited publications within a timely fashion in order to contribute to the literature in consumer health informatics.

 A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

We are not requesting an exemption to the display of the OMB Expiration date.

 A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

8