SUPPORTING STATEMENT NH SOPS Part A 2-21-07

SUPPORTING STATEMENT

Voluntary Questionnaire and Data Collection Testing

to Pilot Test Proposed Nursing Home Survey on Resident Safety

A. JUSTIFICATION

1. Need for Information

The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) approve, under the Paperwork Reduction Act of 1995, AHRQ’s intention to pilot test survey questions for a new survey instrument, the Nursing Home Survey on Resident Safety, in preparation of work to be performed by Westat in collaboration with AHRQ.

As mandated in the Healthcare Research and Quality Act of 1999 (P.L. 106-129), the mission of AHRQ is to enhance the quality, appropriateness, and effectiveness of health services and access to such services, through the establishment of a broad base of scientific research [Section 901 (b)]. With respect to its role as the principal agency for health care research and quality and providing scientific and technical support for private and public efforts to improve health care quality, AHRQ shall promote health care quality improvement by conducting and supporting research that develops and presents scientific evidence regarding all aspects of health care-- including the outcomes, effectiveness, and cost-effectiveness of health care practices, including preventive measures and long-term care; and methods for measuring quality and strategies for improving quality, as well as initiatives to advance private and public efforts to improve health care quality. [Section 901, (b) (1) (B) and (F) and (3)].

Furthermore, AHRQ shall conduct and support research to identify the causes of preventable health care errors and patient injury in health care delivery; develop, demonstrate, and evaluate strategies for reducing errors and improving patient safety; and disseminate such effective strategies throughout the health care industry. [Section 912, (c) (1) (2) and (3)].

To help AHRQ meet these goals as they pertain to the nursing home setting, pilot data collection under this request will be done for the following purposes:

a) Developing a Nursing Home Survey on Resident Safety—A draft survey will be pilot tested with staff in 40 nursing homes.

b) Psychometric Analysis—Provide information for the revision and shortening of the final survey based on the assessment of the reliability and construct validity of the survey items and composites.

The goal of the final survey is to make it available for use in the public domain by researchers, healthcare systems, and nursing homes to assess resident safety culture.

These preliminary research activities are not required by regulation, and will not be used by AHRQ to regulate or sanction its customers. They will be entirely voluntary and the confidentiality of respondents and their responses will be preserved.

2. How, by Whom, and for What Purpose Information Will Be Used

The information collected will be used to test draft survey items from the Nursing Home Survey on Resident Safety described in item #1. The end result will be improvement in the final survey instrument which will be made available to the public for use in nursing homes to assess resident safety culture. Researchers are also likely to use the final nursing home survey to assess the impact of resident safety culture improvement initiatives. This nursing home survey is an expansion of AHRQ’s Hospital Survey on Patient Safety Culture (HSOPSC) which was pilot tested (OMB # 0935-0115, exp 1/31/2004) and made available to the public in November 2004 (www.ahrq.gov/qual/hospculture). It has since been used by hundreds of hospitals across the U.S. to assess patient safety culture. The nursing home survey contains new and revised questions and composites that more accurately apply to the nursing home setting.

Psychometric analysis will be conducted on the pilot data to examine item non-response, item response variability, reliability, and construct validity of the items included in the survey. Because the survey items are being developed to measure specific aspects of nursing home resident safety culture, the factor structure of the survey items will be evaluated through multi-level confirmatory factor analysis. After dropping items or factors based on the data analysis, a final version of the survey will be released for use in the public domain.

3. Use of Improved Information Technology

Data collection will not involve the use of any information technology. This data collection effort is a pilot study designed to gather information to examine the psychometric properties (internal consistency reliability, response variability, etc.) of a paper-and-pencil nursing home safety culture survey instrument. The use of information technology is not needed for this study and is not likely to result in a reduction of burden at this time.

4. Efforts to Avoid Duplication

A review of the literature including published, unpublished, and internet sources (excluding proprietary sources) identified a number of nursing home surveys. Many surveys are at the facility level focusing on the types of services provided, results of inspections, quality of care, or information about residents at the facility. Other surveys focus on staff job satisfaction and working conditions. There are even surveys under development that focus on resident and family/caregiver perspectives about the care provided in nursing homes. However, we have not found a survey assessing staff opinions about resident safety issues except for a few researchers who have adapted the AHRQ Hospital Survey on Patient Safety Culture (referred to in item #2) for the nursing home setting. We believe that simple adaptations to the AHRQ hospital survey (such as changing the word “hospital” to “nursing home”) are not sufficient to capture the range of safety culture issues that are specific to the nursing home setting. Therefore, existing surveys would not accomplish the purposes of the present project.

The current study will allow for the development and pilot testing of a nursing home-wide resident safety culture survey that nursing home staff will complete. The pilot data collection will provide much needed reliability and statistical information about the instrument.

5. Small Businesses

The survey instrument and procedures for completing the instrument are designed to minimize burden on individual nursing home staff respondents and will not have a significant impact on small businesses or other small entities.

6. Consequences of Less Frequent Collection

This effort is a one-time pilot test.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

The data collection efforts will be consistent with the guidelines at 5 CFR 1320.5(d)(2).

8. Applicability to 5 CFR 1320.8(d)

a. Notice was published in the Federal Register on Tuesday, Dec. 12, 2006—Attachment C. Thus far, the American Association of Retired Persons (AARP) has requested a copy of the draft survey and pilot data collection plan, but no comments have been received from them. One individual comment has been received but was received too late to be addressed in the final 30 day Federal Register notice:

“most nursing homes are hellholes. the patients are subtly browbeaten in most of them, unless they are very very expensive. this bad treatment of patients is not apparent to the casual observer. taking the time to fill out these stupid surveys takes time away from

patient care. the most important element in these nursing homes is patient care, not filling out these stupid surveys that accomplish NOTHING AT ALL. instead get out and inspect these homes and see what goes on. this survey is irresponsible in wasting tax dollars to accomplish absolutely nothing of any help.”

In response to this comment, the purpose of the survey data collection from staff in nursing homes will be to pilot test a newly developed survey on resident safety culture in nursing homes. The pilot test will be done on only 40 nursing homes for the purposes of pretesting the instrument, identifying which items work the best to assess resident safety culture, and developing a final survey that will be made available in the public domain. The goal of the pilot test is to not to have the government conduct a national survey of nursing homes and report on the results, but to collect data for the purposes of testing and finalizing a survey that can be used by nursing homes and nursing home chains for their own self-assessment, as well as providing a tool for researchers interested in examining resident safety culture in nursing homes.

The surveys are intended to educate staff in nursing homes about important resident safety issues they need to be aware of, and to provide a tool for self-assessment so nursing homes can determine how staff view resident safety and quality and identify areas for improvement. When the final survey is used by nursing homes, it will provide valuable information to identify areas for resident safety and quality improvement.

A similar patient safety culture survey has already been developed for hospitals and has been widely used by hundreds of hospitals across the U.S. and has even been translated into many different languages for use in hospitals internationally. Hospitals have found the hospital patient safety culture survey a very useful tool in their patient safety improvement efforts. AHRQ believes the nursing home survey on resident safety is very much needed as the population of older Americans continues to grow and nursing homes play a more important role in their healthcare.

b. A number of long-term care, patient safety researchers, and nursing home stakeholders were consulted on the need for a resident safety culture survey as well as for review and comment on the draft survey domains and items—see Attachment A for a list of those consulted outside the Agency thus far.

9. Remuneration of Respondents

Based on information received from consultations with researchers who have recently implemented surveys to nursing home staff, to ensure high response rates for our paper survey it appears necessary to provide individual nursing home staff members with a cash incentive to complete the survey. Because this population tends to have lower education levels and reading ability, it requires some effort for them to complete, and they may initially refuse to complete the survey. We therefore plan to provide a $5.00 cash prepaid incentive along with the survey.

In addition, because data collection will require the assistance of a nursing home point of contact (POC) to manage data collection at their nursing home (compile sample information, distribute surveys, promote survey response, etc.), we plan to provide the POC with a one-time individual incentive of $150. These types of individual and point-of-contact remuneration for survey participation are a recognized practice when administering surveys in nursing home settings, without which, it would be difficult to achieve appropriate and adequate participation.

We provide information from the following researchers and experts regarding their use of incentives as a way to obtain good response rates:

1) Nicholas G. Castle, PhD, Associate Professor, University of Pittsburgh. Data were collected from a nationally representative sample of nursing homes (N=2840) to evaluate safety culture using an adapted version of the AHRQ Hospital Survey on Patient Safety Culture. His response rate was 71% using a $5 incentive payment in the mailed envelope directly to the certified nursing assistant.

2) Steven M. Handler, MD, MS, Assistant Professor, University of Pittsburgh School of Medicine. To assess safety culture in nursing homes, 151 nursing home administrative personnel were sent survey packets containing a cover letter, survey, prepaid return envelope and a $10 gift certificate. Response rate was 68.9%.

3) Jill Scott-Cawiezell PhD, Sinclair School of Nursing, University of Missouri-Columbia. Various incentives were used for an Agency for Healthcare Research and Quality grant in 32 selected nursing homes. Researchers paid the site coordinator $100 to coordinate data collection. A $2 cash incentive was inserted in each sealed survey packet. Researchers provided $150 to every nursing home that met or exceeded an 80% return rate, which was successful.

4) David Cantor PhD, Associate Director at Westat. Based on his extensive experience fielding surveys, in general, promised incentives do not do well, especially small amounts, and particularly for professional staff like MDs. For example, a $2 pre paid incentive is better than a $5 promised incentive. Therefore, prepaid incentives are best.

5) Laura Brandon, Senior Study Director at Westat, Project Manager for the National Nursing Assistant Survey designed to produce national nursing home estimates.

• For a 69% response rate: Researcher gave $5 in the materials to recruit nursing assistant participation; then gave $30 after they completed a computer assisted personal interview.

• Had a Point of Contact (POC) for each facility, researchers met with administrator, DON and/or Human Resources person to explain project, get POC in place, the POC distributed packets, and collected signed receipts for incentive distribution. Of 1500 homes contacted, approximately half of them participated.

10. Assurance of Confidentiality

Individuals and organizations contacted will be assured of the confidentiality of their replies under Section 924(c) of the Healthcare Research and Quality Act of 1999.

11. Questions of a Sensitive Nature

No questions of a sensitive nature are included in the survey.

12. Estimates of Annualized Hour Burden

The survey will be distributed to a total of approximately 5,500 employees from 40 nursing homes, with a target response rate of 70%, or 3,850 returned surveys. Standard techniques like using a cover letter of support from the nursing home, a reminder postcard, and distribution of a second survey will be used to achieve the target response rate. Respondents should take approximately 15 minutes to complete the survey. Therefore, we estimate that the respondent burden for completing the survey will be 963 hours (3850 completes multiplied by 0.25 hours per completed survey)(See Table 1).

TABLE 1 ANNUAL BURDEN ESTIMATE FOR PILOT OF NURSING HOME SAFETY CULTURE SURVEY

Type of Respondent	Estimated Number of Respondents	Number of Responses per Respondent	Estimated Time per Respondent (hours)	Estimated Total Respondent Burden (Total Hours)	Avg. Nursing Home Wage per Hour	Annualized Cost Burden to Respondents
Nursing home staff member	3850	1	.25	963	$10.67 per hour	$10,275

Using the U.S. Department of Labor, Bureau of Labor Statistics 2005 National Occupational Employment and Wage Estimates for nursing aides, orderlies, and attendants of $10.67 per hour (http://www.bls.gov/oes/current/oes311012.htm) as an estimated mean hourly wage for nursing home employee respondents.

13. Estimates of Total Annual Cost Burden to Respondents

The only cost to the respondent will be that associated with their time to respond to the information collection, as shown in Table 1. There will not be any costs for capital equipment or operational expenses.

14. Estimates of Annualized Cost to the Government

The total one-time cost for this contracted survey development and analysis effort is approximately $320,000 which includes the cost of survey development, pretesting, preparation of survey administration procedures, data collection, incentives, data analysis, and preparation of a final report. The estimated cost of only the data collection component is approximately $150,000, which includes incentive costs, labor costs, fringe expenses, administrative expenses, and costs associated with copying, postage, and telephone expenses.

15. Change in Burden

This is a new activity.

16. Plans for Analyses

The purpose of this survey effort is to develop and pilot test a new survey—Nursing Home Survey on Resident Safety—and to perform a psychometric analysis of the pilot test data. Psychometric analyses of the pilot data will involve an examination of item-level non-response and variability. In addition, since the survey items were developed to measure apriori domains of safety culture, multi-level confirmatory factor analyses will be conducted to determine whether the survey items appear to assess these domains at both the individual and nursing home levels. ICC(1)s and goodness-of-fit indices will be examined. Any problematic items or factors will be dropped. The final survey will be made available in the public domain for use by researchers, healthcare systems, and nursing homes to assess safety culture.

17. Exemption for Display of Expiration Date

No exemption is being requested.

18. Certifications

These activities will comply with the requirements of 5 CFR 1320.9.

Attachment A-List of Long-Term Care & Patient Safety Expert Reviewers & Consultants

Attachment B-Survey Materials

Cover letter

Reminder postcard

Survey

Attachment C- Federal Register Notice (separate file attachment)

Alice Bonner, RN, GNP

Director of Clinical Quality

Massachusetts Extended Care Federation

Laura Brandon

Senior Study Director

Westat

Thomas E. Brown, DrPH

President & CEO

Lutheran Homes of South Carolina

Pam Carroll-Solomon, MJ, RHIA, CPHQ

Director, Quality Services

Catholic Health East

Nicholas G. Castle, PhD

Assistant Professor of Health Policy and Management;

Graduate School of Public Health

University of Pittsburgh

Catholic Healthcare Partners

Brian Forschner, Robert Breeden, Susan Hayes, Wayne Bohenek

Long Term Care Administrators and Quality Managers

Genevieve Gipson RN MEd RNC

Director

Career Nurse Assistants Programs, Inc.

National Network of Career Nursing Assistants

Jerry H. Gurwitz, MD

Executive Director, Meyers Primary Care Institute

Chief, Division of Geriatric Medicine

University of Massachusetts Medical School

Pam Guth

Operations Manager and Safety Officer

Skilled Nursing Center at St. John’s Mercy Medical Center

St. Louis, MO

Bryant Hall, Jr.

Board of Directors of ACHCA

Administrator at The Wesley Assisted Living Residence & Skilled Nursing Facility in Baltimore

Steven M. Handler, MD, MS

Assistant Professor

University of Pittsburgh School of Medicine,

Division of Geriatric Medicine and

Center for Biomedical Informatics

Lauren Harris-Kojetin, PhD

Chief, Long-term Care Statistics Branch

National Center for Health Statistics

Terry Hill, MD

Medical Director, Quality Improvement

Lumetra

San Francisco, CA

Sandra L. Hughes, Long Term Care Policy Coordinator and

Bill Borwegen, Health and Safety Director

The Service Employees International Union (SEIU)

Marshall B. Kapp, J.D

Garwin Distinguished Professor of Law and Medicine

Southern Illinois University

Founding editor of the Ethics, Law, and Aging Review

Fellow of the Gerontological Society of America

Kate Lapane, PhD

Associate Professor
Community Health

Brown University

Victor Lee, MD

Catholic Healthcare Partners

Becky Miller, MHA, CPHQ, CHE

Executive Director

Missouri Center for Patient Safety

Columbia, MO

Susan Pfefferle, PhD

Brandeis University

Susan Polniaszek, MPH

Office of Disability, Aging, and LTC Policy

Office of Assistant Secretary of Planning and Evaluation

Jules Rosen, MD

Institute On Aging

Professor of Psychiatry

University of Pittsburgh

Jill Scott-Cawiezell PhD

Sinclair School of Nursing

University of Missouri-Columbia

Christine Sheehy, PhD

Director of Quality for the INOVA HealthSystem nursing homes

Fairfax, VA

Steve Shields

Executive Director of Meadowlark Hills Continuing Care Retirement Community Manhattan, Kansas

Jo Abbott Taylor, B.S.N., M.P.H.

Preventing Falls in the Elderly Project

Christie Teigland, PhD

Director of Health Informatics and Research

New York Assn Homes & Services for Aging (NYAHSA)

Laura M. Wagner, PhD, RN

Gerontological Nursing Research Scientist

Kunin-Lunenfeld Applied Research Unit

Baycrest Centre for Geriatric Care

Toronto, Ontario

Attachment B

Cover Letter of Support from Nursing Home

[Insert nursing home facility logo here]

Hello,

[Insert name of facility] is participating in an important survey sponsored by the Agency for Healthcare Research and Quality. The survey asks for your opinions about resident safety in our nursing home. Westat, a private research organization located in Rockville, Maryland, is conducting the survey.

The survey should take about 15 minutes to complete. Your feedback will help us find ways to improve resident safety in our nursing home. If you do not wish to answer a question, you may leave it blank. Westat will keep your individual responses to this survey strictly confidential. Only group results will be reported.

Five dollars is enclosed to thank you for participating in this survey. Please complete the survey as soon as possible and return it to Westat in the enclosed envelope. If the return envelope is missing contact [insert NH Administrator name here] for another envelope.

If you have any questions or concerns, please contact me at [insert NH POC phone here]. We value and appreciate your participation in this important research!

Sincerely,

[Insert NH Administrator name here]

[Administrator]

Attachment B

Reminder Postcard Text

We want to hear from you!

 

Nursing Home Survey on Resident Safety

 

Recently, a survey was distributed to you. The survey is part of a research project sponsored by the Agency for Healthcare Research and Quality (AHRQ) to assess staff views on resident safety issues in nursing homes.

 

If you have already completed your survey and mailed it back to Westat, THANK YOU VERY MUCH! 

 

If you have not yet had a chance to complete your survey, please take a few minutes to fill it out and mail it back to Westat in the postage-paid envelope that was provided. Your opinions are important to us.  Thank You! 

 

If you have any questions, please call

«NAME» at «PHONE»

 

 

 

 

 

 

 

7