Supporting Statement Part A

SUPPORTING STATEMENT FOR LEARNERS’ PERCEPTIONS SURVEY, Cont’d

This submission is for extension of a currently approved information collection.

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection of information.

Under the authority of Federal Law 38 U.S.C. Part V, Chapter 73, Section 7302, the Department of Veterans Affairs (VA) provides education and training to a national cohort of health care trainees per year to assist in providing an adequate supply of health personnel for VA and the Nation. VA is further required to evaluate this program on a continuing basis and determine its effectiveness in achieving its goals (Federal Law, 38 U.S.C. Part I, Chapter 5, Section 527). In addition, the Government Performance and Results Act (GPRA) of 1993, requires Federal agencies to set goals, measure performance, and report on the accomplishments.

Health professional training is one of the Veterans’ Health Administration’s (VHA) core missions for the past 60 years, since Policy Memorandum No. 2 established an association of Veterans Administration medical centers (VAMC) and schools of medicine. VA medical centers throughout the nation have served this mission by affiliating with accredited training programs in undergraduate and graduate medical education, nursing and associated health training programs such as pharmacy, psychology, social work and dentistry. Over this time period, the number of VA facilities with medical school affiliations has grown to 127 with 2,424 accredited physician training programs. The majority of these programs are accredited by the Accreditation Council on Graduate Medical Education (ACGME) training physicians. In addition, VA has affiliations with over 1,200 colleges and universities for over 5,000 accredited programs in over 40 associated health disciplines.

In addition to the growth of programs within the VA, external forces have created a changing healthcare environment. Accrediting bodies, such as the ACGME have imposed sweeping changes, including resident duty hour limits in the training environment that affect the VA environments and veteran patients served. This changing landscape in the clinical education environment comes at a time when Veterans Health Administration (VHA) must rise to the challenge of treating a new generation of veterans. The interface with the clinical and the educational arenas necessitates a system for assessment of the education environment. Consistent with VHA leadership in other areas of measurement, such as performance measures to track and improve quality of patient care in the clinical realm, VHA has served to lead in measurement of learner satisfaction through the creation and implementation of the Learners’ Perceptions Survey.

The Learners’ Perceptions Survey (LPS) is a tool that was created by the VA to allow ongoing assessment of clinical training environments in order to help accomplish VA’s teaching mission. The survey identifies both strengths and opportunities for improvement for VHA in the clinical education environment and measures the satisfaction of VA clinical trainees who come in direct contact with our veteran patients and who contribute to their care on a daily basis. Clinical trainees who are satisfied with their clinical training can impact how patients view their care and patient satisfaction. Implicit in the survey is the identification of key drivers of clinical trainees’ satisfaction so as to develop and implement targeted improvements that will benefit both learners and patients in VHA.

This survey is consistent with VA’s oversight responsibilities and Government Performance and Results Act (GPRA), and VA has identified trainee satisfaction with clinical training experience as one of the VA national performance measures. The survey results are reported internally in VA and in the Annual Performance and Accountability Report submitted to Office of Management and Budget (OMB).

2. Indicate how, by whom, and for what purposes the information is to be used; indicate actual use the agency has made of the information received from current collection.

The results of the survey are used at every level in VHA from the national level to local in understanding elements and domains that impact the clinical training environment. At the VA medical center level, the survey data is available to inform local VA education leaders and program officials regarding areas of strength and opportunities for improvement in specific domains (quality of the faculty, learning environment, working environment, physical environment and personal experience) and allows facilities to trend data so that changes over time can be monitored. In order to maintain VA as a preferred training site for future health care professionals, it is important to know how trainees view VA training versus training in non-VA settings. The survey results provide data on the perceptions of trainees comparing their clinical training experiences at both VA and non-VA facilities.

At the VISN level, the results of the survey can focus VISN-leadership on the critical needs pertaining to the teaching mission of VA and provide information as to how the teaching environment can be improved. Survey results also indicate that the likelihood that trainees will consider employment with VA improves (almost doubles) if they had training at VA. Careful evaluation of areas of satisfaction and relative dissatisfaction may provide useful information to aid in recruitment and retention of quality health care professionals.

At the national level, the survey results call attention to the rigor with which the VA addresses the education mission as the results are reported widely in VA and in the Annual Performance and Accountability Report submitted to OMB. Over the past several years, data has been collected regarding the degree to which individual facilities and disciplines have used the survey results to make changes in their programs. Since drivers of satisfaction are different for different disciplines, special studies are completed for certain disciplines (i.e., pharmacy, nursing and physician residents) to determine system-wide trends and issues.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

Electronic, online completion of the survey reduces respondent and government burden, by providing a convenient, faster and less labor intensive survey collection and data analysis than paper submission. The respondents do not have to mail paper questionnaires, and they can complete the survey from any Internet accessible computer, either at home or at a VA medical center. Data can be rolled up by facility, VISN, and nationally for analysis using a number of trainee categories (e.g., types of training programs and level of training).

To reduce burden, survey implementation will be limited to trainees having training rotations at the VA during the month of March. For physician trainees, it is known that satisfaction varies over the course of the year, with biases both at the start (e.g. July/August) and end (e.g. May/June) of the academic year. Measurement in the month of March will allow assessment of the clinical education environment at a time when physician trainee perceptions are not influenced by start of the year stressors (new interns) or end of the year fatigue. For allied health trainees, March is the optimal month for measurement as this is the time of year with the largest number of trainees who have had ample experience in the system to provide useful feedback on the system.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

Information at the facility or system level concerning trainees’ perceptions of their VA clinical training experience is not available elsewhere.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

The continuous improvement cycle requires local facilities to measure, intervene and re-measure. If the survey were to be conducted less frequently, VA could not institute timely changes and program oversight would be impaired.

There are no technical or legal obstacles to reducing burden.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

Not applicable. There are no such special circumstances or requirements.

8. a. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The notice of Proposed Information Collection Activity was published in the Federal Register on April 18, 2007, Volume 72, Number 74, Page 19586. We received no comments in response to this notice.

b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.

A contract was awarded (in 1999) to Schulman, Ronca & Bucuvalas, Inc. (SRBI), a contractor with expertise in conducting large-scale surveys. SRBI assisted in designing the survey instrument, conducting the survey and analyzing and reporting data. SRBI was consulted regarding frequency of collection, clarity of instructions, internal record keeping and reporting format. Additionally, public comments were sought through Federal Register notices. The contract with SRBI is no longer active, and all work on the survey is performed by VA employees, who have the appropriate expertise.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift is provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The survey includes the following statement: “Every effort will be made to maintain confidentiality of the respondents while sharing results of the Learners’ Perceptions Survey with VA facilities.”

Only aggregated data is used in reporting results to VA facilities. The completed surveys can be withheld if requested under the Freedom of Information Act (FOIA) under Exemption 5 of the FOIA according to VA General Counsel. The clinical trainees’ opinions in the survey are considered individual opinions gathered in a “pre-decisional” deliberation.

11. Provide additional justification for any questions of a sensitive nature (Information that, with a reasonable degree of medical certainty, is likely to have a serious adverse effect on an individual's mental or physical health if revealed to him or her), such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Not applicable. There are no questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information:

a. The annual burden is estimated to be

Respondents	Minutes	Equals	/ by 60	Annual Burden Hours
9,000	15	135,000	60	2,250

b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB 83-I.

This request is for a web-based survey questionnaire for VHA healthcare trainees.

c. Provide estimates of annual cost to respondents for the hour burdens for collections of information. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14 of the OMB 83-I.

The total cost to the respondents annually is based on time spent on completing the survey and is estimated to be $33,750.

Burden Hours	Cost	Total
2,250	$15	$33,750

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

The only cost to the respondent is the time spent in completing the survey. There is no capital, start-up, operational or maintenance costs to the respondent for completing the survey.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total estimated cost to the Government is estimated at $30,629.

.

Staff Salary	Function	Salary	Hours
GS 13/5	On-Line Survey Development	$40.95	80	$3,276
GS 13/5	Report Preparation	$40.95	200	$8,190
GS 13/5	SPSS Programming	$40.95	160	$6,552
GS 13/5	Web Design	$40.95	40	$1,638
GS 14/5	Statistical Analysis	$48.39	120	$5,807
GS 12/5	Help Desk Assistance	$34.44	150	$5,166
TOTAL				$30,629

15. Explain the reason for any burden hour changes since the last submission.

The public burden hours in item 12 on a per person basis remained unchanged at 15 minutes.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The primary mission of conducting the LP survey is to identify strengths and opportunities for improvement in the VA clinical training program. Information collected is disseminated to VA facilities and VISNs so that they can take corrective actions. The aggregated information is disseminated nationally in VA reports and may be published. OAA has published results based upon the LPS and may do so in the future using appropriate statistical methodology.

For example, in 2003 OAA published survey results in the journal Academic Medicine, Volume, 78, No. 9/September 2003, entitled, “The Veterans Affairs Learners’ Perceptions Survey: The Foundation for Educational Quality Improvement.” In this study the focus was on results for physician residents and differences in learner perceptions among residents in internal medicine, surgery, subspecialty training, and psychiatry.

List of articles published regarding recent surveys:

Keitz, S., Holland, G.J., Melander, E.H., Bosworth, H., and Pincus, S.H. for the Learners’ Perceptions Working Group (Gilman, S.C., Mickey, D.D., Singh, D., et al). The Veterans Affairs Learners’ Perceptions Survey: The Foundation for Educational Quality Improvement. Academic Medicine 78(9):910-917, 2003.

Singh, D.K., Holland, G.J., Melander, E.H., Mickey, D.D., Pincus, S.H.: VA’s Role in U.S. Health Professions Workforce Planning. Proceedings of the 13th Federal Forecasters Conference of 2003:127-133, 2004.

Singh, D. K., Golterman, L., Holland, G. J., Johnson, L.D., and Melander, E. H., Proposed Forecasting Methodology for Pharmacy Residency Training, Proceedings of the 15^th Federal Forecasters Conference of 2005: 39-42, 2005.

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

VA continues to seek an exemption that waives the displaying of the expiration date on the VA Forms by seeking to minimize its cost to itself of collecting, processing and using the information by not displaying the expiration date. Inclusion of the expiration date places an unnecessary burden on the respondent as we have found that the term “expiration date” confuses respondents.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

Page 8