SUPPORTING STATEMENT
Part A
Demonstration of Health Literacy Universal Precautions Toolkit
Version: September 14, 2012
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances That Make the Collection of Information Necessary 3
2. Purpose and Use of Information 7
3. Use of Improved Information Technology 8
4. Efforts to Identify Duplication 8
5. Involvement of Small Entities 8
6. Consequences if Information Collected Less Frequently 8
7. Special Circumstances 8
8. Federal Register Notice and Outside Consultations 8
9. Payments/Gifts to Respondents 9
10. Assurance of Confidentiality 9
11. Questions of a Sensitive Nature 10
12. Estimates of Annualized Burden Hours and Costs 10
13. Estimates of Annualized Respondent Capital and Maintenance Costs 15
14. Estimates of Annualized Cost to the Government 15
15. Changes in Hour Burden 15
16. Time Schedule, Publication and Analysis Plans 15
17. Exemption for Display of Expiration Date 17
List of Attachments 17
References 18
The mission of the Agency for Healthcare Research and Quality (AHRQ), set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999, is to enhance the quality, appropriateness, and effectiveness of health services as well as access to such services. The agency addresses these objectives through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ promotes health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care;
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
AHRQ also conducts and supports research and evaluation projects as well as demonstration projects, with respect to (1) the delivery of health care in inner-city areas and in rural areas (including frontier areas); and (2) health care for priority populations, which include (a) low-income groups, (b) minority groups, (c) women, (d) children, (e) the elderly, and (f) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
A goal of Healthy People 2020 is to increase Americans’ health literacy, defined as, “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”1 The effects of limited literacy are numerous and serious, including medication errors resulting from patients’ inability to read labels; underuse of preventive measures such as Pap smears and vaccines; poor self-management of conditions such as asthma and diabetes; and higher rates of hospitalization and longer hospital stays.2,3
According to the 2003 National Assessment of Adult Literacy (NAAL), more than one-third of Americans – 77 million people – have limited health literacy.4 Although some adults are more likely than others to have difficulty understanding and acting upon health information (e.g., minority Americans, elderly), providers cannot tell by looking which patients have limited health literacy.5, 6, 7 Experts recommend that providers assume all patients may have difficulty understanding health-related information. Known as adopting “health literacy universal precautions,” providers create an environment in which all patients benefit from clear communication.8
AHRQ contracted with the University of North Carolina at Chapel Hill to develop the Health Literacy Universal Precautions Toolkit (Attachment A) to help primary care practices ensure that systems are in place to promote better understanding of health-related information by all patients. As part of Toolkit development, testing of a “prototype Toolkit” was conducted in eight primary care practices over an eight-week period. Testing provided important information about implementation and resulted in refinement of the Toolkit, which AHRQ made publically available in Spring 2010. At this time, the Toolkit includes 20 tools to prepare practices for health literacy-related quality improvement activities and to guide them in improving their performance related to four domains: (1) improving spoken communication with patients, (2) improving written communication with patients, (3) enhancing patient self-management and empowerment, and (4) linking patients to supportive systems in the community. The tools included in the Health Literacy Universal Precautions Toolkit are listed below:
Tools to Start on the Path to Improvement
Tool 1: Form a Team
Tool 2: Assess Your Practice
Tool 3: Raise Awareness
Tools to Improve Spoken Communication
Tool 4: Tips for Communicating Clearly
Tool 5: The Teach-Back Method
Tool 6: Follow up with Patients
Tool 7: Telephone Considerations
Tool 8: Brown Bag Medication Review
Tool 9: How to Address Language Differences
Tool 10: Culture and Other Considerations
Tools to Improve Written Communication
Tool 11: Design Easy-to-Read Material
Tool 12: Use Health Education Material Effectively
Tool 13: Welcome Patients: Helpful Attitude, Signs, and More
Tools to Improve Self-Management and Empowerment
Tool 14: Encourage Questions
Tool 15: Make Action Plans
Tool 16: Improve Medication Adherence and Accuracy
Tool 17: Get Patient Feedback
Tools to Improve Supportive Systems
Tool 18: Link Patients to Non-Medical Support
Tool 19: Medication Resources
Tool 20: Use Health and Literacy Resources in the Community
AHRQ will now conduct a demonstration of the Health Literacy Universal Precautions Toolkit. The purpose of this demonstration project is to examine Toolkit implementation and identify refinements that would improve the value of the Toolkit for future users. Twelve primary care practices will be recruited to implement at least four tools from the Health Literacy Universal Precautions Toolkit. The project team will provide participating practices with limited technical assistance throughout the implementation period. Data regarding the assistance provided will contribute to the team’s assessment of the ease with which specific tools can be implemented and will provide insight into additional resources and guidance that might be valuable to add to the Toolkit.
To achieve the goals of this project the following activities and data collections will be implemented:
Practice Screening and Follow-up Recruitment Calls: As described in Part B, recruitment emails will be sent to all members of two large practice-based research networks (see Attachment B). The project team will then conduct screening calls with all interested practices, typically with the lead physician or practice administrator. The introductory script presents an overview of the project. For those practices that agree to participate, some basic data about the practice will be collected, such as the type of practice, the number of full and part time clinicians, the number of patients seen in a typical week and the percentage of patients enrolled in Medicaid (see Attachment C). Following the completion of these screening calls, the project team will identify the most promising practices. We will conduct follow-up recruitment calls with each of those practices to review the requirements of study participation and ensure practices are interested and able to participate in the Demonstration (see Attachment D).
2) Health Literacy Assessment Questions: In implementing Tool 2, which guides practices in conducting a self-assessment of their health literacy-related systems and procedures, practices will complete the Health Literacy Assessment Questions at the beginning of the project (see Attachment E). We will request that they complete the same items again following implementation so that we may examine whether these items suggest change over time. Practices will collect responses from staff members representing different components of the practice (e.g., clinicians, front desk staff). A member of the practice staff, who will be designated the project coordinator, will oversee collection of survey data.
3) Implementation Tracking Form: The Implementation Tracking Form will be completed by the leader of the Health Literacy Team at the beginning of the project period and updated prior to each check-in phone call with project staff (see item 13 below). (As part of implementation of Tool 1, participating practices will establish a Health Literacy Team to oversee Toolkit implementation.) This form elicits information about the timing with which different steps in the implementation process were completed (e.g., when was the first training conducted). The form is included as Attachment F.
4) Webinar/Orientation: Prior to beginning data collection, we will conduct a Webinar with all practices to review the pre-implementation data collection requirements and provide an overview of Tools 1 and 2, which practices are to complete prior to our conducting site visits (see Attachment G). Up to four members of the Health Literacy Team or other practice members will attend.
5) Project Team Review of Practice’s Health Literacy Environment: At pre- and post-implementation, the project team will conduct an onsite observational review of the practice environment to assess health literacy-related features, such as readability of patient materials in the waiting room and ease of patient navigation (see Attachment H). This data collection activity involves no burden to participating practices and their patients and, therefore, is not included in the burden estimates in Section 12.
6) Patient Survey: The Patient Survey will be collected at pre- and post-implementation and is designed to obtain patient input on health literacy-related performance of providers and staff (e.g., “did your provider use medical words you did not understand”). The Survey will only be collected in practices that choose to implement Tool 17. We expect approximately six practices to select Tool 17 as one of their supplementary tools. Each practice will recruit 50 patients at each time point to complete the survey. The survey will include the same items at the two time points. The on-site project coordinator will oversee recruitment and collection of survey data (see Attachment I.1). Surveys will either be collected using touch screen tablets or paper forms. In either case, a cover memo will be provided to patients explaining the voluntary nature of their participation. Two copies of this memo are included in Attachment I.2 and I.3, one for patient completing the Survey on a touch screen tablet and one for patients using hardcopy forms.
8) Survey Using Items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS): In two of the participating practices that chose to implement Tool 17, selected health literacy-related items from the CAHPS Clinician and Group Survey will be administered at pre- and post-implementation (see Attachment J). Surveys will be sent by mail, with phone follow up. Across practices and the two time points (pre- and post-implementation), we will collect surveys for 1200 patients.
9) Medication Review Form: Each practice that chooses to implement Tool 8 (Brown Bag Medication Review) will conduct medication reviews with 20 patients at pre-implementation and 20 at post-implementation, completing the Medication Review Form for each review (see Attachment K). (We estimate that 3 of the 12 participating practices will choose to implement Tool 8.) During these reviews, the Medication Review Form will be completed to record errors found in the medication regimen (e.g., expired medications, incorrect dosing, patient misunderstanding of regimen). So that this data collection activity will be of value to practices and patients, reviews will be conducted with patients identified through routine clinical practice (e.g., the prescription refill process, regular follow-up visits) to require a full review of current medications.
10) Practice Staff Survey: We will request that all staff members of participating practices complete the Practice Staff Survey, which elicits staff perceptions regarding health literacy-related practices (e.g., staff use of effective communication techniques and confirmation of patient comprehension). Surveys will be completed at pre-implementation and post-implementation, with items varying slightly at the two time points. Attachments L and M contain the pre- and post-implementation versions of the survey, along with the accompanying cover memos which provide information about participation. The project coordinator for each practice will oversee collection of survey data.
11) Health Literacy Team Leader Survey: The leader of the Health Literacy Team will complete this survey at pre- and post-implementation to provide data regarding health literacy-related policies and details regarding Toolkit implementation (e.g., has the reading level of written patient materials been assessed, how does the practice remind patients to bring in medication bottles to facilitate medication reviews). The pre- and post-implementation versions of the survey, which differ slightly, are included in Attachments N and O, respectively. During the pre-implementation site visit, a project staff member will review an informed consent form with the Team Leader to obtain his/her written consent to participate in the Health Literacy Team Leader Survey, the pre- and post-implementation interviews, and the Check-in Phone Calls. Written informed consent will be obtained prior to the Team Leader completing the survey (Attachment P).
12) Health Literacy Team Leader Interview: The leader of the Health Literacy Team will be interviewed in person at pre- and post-implementation. At the beginning of the project, this qualitative interview will focus on expectations regarding implementation (e.g., expected barriers) and technical assistance needs. The post-implementation interview is designed to elicit detailed information about the implementation process, suggested revisions to the Toolkit, and an assessment of the technical assistance provided. Attachments Q and R include the pre- and post-implementation versions of the Health Literacy Team Leader Interview, respectively. Attachment P includes the consent form.
13) Check-in Phone Calls: To ensure that practices stay on track, the project team will contact practices on a regular schedule to assess progress and provide facilitation that might be needed to help practices address barriers they may be experiencing. Calls will take place two weeks, one month, two months, and four months into implementation and will involve the leader of the Health Literacy Team. The questions to be asked during these calls are provided in Attachment S.
14) Health Literacy Team Member Interview: So that we may obtain information about the implementation process as well as functioning of the Health Literacy Team (e.g., how difficult was it to reach decisions about which tools to implement), we also will interview a member of the Team other than the Team leader at post-implementation. Interviews will be conducted on site at the practice and will follow the interview protocol provided in Attachment T. Written informed consent will be obtained prior to the beginning of the interview (Attachment P).
15) Practice Staff Member Interview: So that we can obtain input about Toolkit implementation and project participation from someone outside of the Health Literacy Team, we will conduct on-site interviews at post-implementation with one or two staff members who were not involved in the Health Literacy Team (see Attachment U). Written informed consent will be obtained prior to the beginning of the interview (Attachment P).
This study is being conducted by AHRQ through its contractors, the University of Colorado, the American Academy of Family Physicians National Research Network and Synovate, Inc. d/b/a Ipsos-Reid Public Affairs under its statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness, and value of health care services and with respect to quality measurement and improvement (42 U.S.C. 299a(a)(1) and (2)).
Data collected will be used for the following purposes:
To thoroughly examine implementation of and refine the Health Literacy Universal Precautions Toolkit. AHRQ will issue a new edition of the Toolkit based on insights from this study.
To examine whether measures, including items from the CAHPS Item Set for Addressing Health Literacy, are sensitive to health literacy-related quality improvement activities. AHRQ will use the findings to modify the document entitled “About the CAHPS Item Set for Addressing Health Literacy,” which discusses use of the items for quality improvement.
To conduct preliminary analyses to explore whether/how the Toolkit assists motivated practices to take a systematic approach to reducing the complexity of health care and ensuring that patients can succeed in the health care environment. Based on the data collected, AHRQ will issue a Technical Assistance Guide for use by practice facilitators that work with Toolkit implementers and Case Studies that highlight lessons learned.
3. Use of Improved Information Technology
Each practice selected for the study will be encouraged to designate two individuals to take on the responsibility of creating the Health Literacy Team, which will then be responsible for overseeing implementation of the Toolkit. These individuals (and other members of the Health Literacy Team) will take part in a 1½ hour webinar, during which the project team will review the details of the project and the Toolkit. This webinar will be made available online so that interested practice members can access it for future reference.
The Patient Survey will be collected using a combination of touch screen tablets and paper surveys. The approach used will depend on factors related to the practice (e.g., the practice’s comfort with the technology) and the patients (e.g., some patients might prefer to complete a paper survey at home).
A comprehensive literature review was conducted by project staff and found no studies with a similar combination of objectives, design, setting, and study participants. Hence, the proposed study is not duplicative.
This project does not involve small entities.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on March 9, 2012 for 60 days.
Darren DeWalt, MD: Health Literacy Expert (Class I): Dr. DeWalt is Associate Professor in the Division of General Internal Medicine at the University of North Carolina Chapel Hill. He was the leader of the team that developed the Health Literacy Universal Precautions Toolkit for AHRQ and was a member of the scientific team that performed a systematic review of the impact of literacy on health outcomes for AHRQ. Dr. DeWalt is a content expert consultant to the project. In year one of the project, he focused on consulting on matters surrounding the practice self-assessment process and other measurement issues. In year two, he will provide input on implementation issues. In year three, he will work with the team, as required by the terms of the Task Order, to reach agreement about what revisions should be made to the Toolkit.
Additional Consultants: Five additional, nationally-recognized health literacy experts served as consultants to the project. They each spent two days on the project, providing guidance on issues related to measurement of health literacy and health literacy-related systems and procedures in primary care. These consultants were: (1) Dr. Ruth Parker, health literacy researcher, co-developer of the Test of Functional Health Literacy in Adults (TOFHLA), and co-chair of the Institute of Medicine's Committee on Health Literacy, (2) Dr. Dean Schillinger, health literacy researcher and director of the Center for Vulnerable Populations at the University of California, San Francisco, (3) Dr. Terry Davis, health literacy researcher and developer of the Rapid Estimate of Adult Literacy in Medicine, (4) Dr. Mark Williams, health literacy researcher, co-developer of the TOFHLA, and co-developer of the American Medical Association’s health literacy train-the-trainer program, and (5) Kim Broucksou, health literacy and education researcher.
Dr. Lucy Savitz served as an additional consultant for the project. Dr. Savitz is a health care delivery and health services research expert and Associate Professor in Clinical Epidemiology at the University of Utah. She provided guidance regarding measurement and evaluation design.
This study does not include any payments or gifts to respondents. However, participating practices will receive a payment of $3,000 as vendors to compensate them for time spent on this project.
Individuals and organizations will be assured of the confidentiality of their replies to interview and survey questions under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
All research involving interactions or interventions with human subjects that pose no more than minimal risk to those subjects is within the purview of the American Academy of Family Physicians (AAFP) Institutional Review Board (IRB) and Colorado Multiple Institutional Review Board (COMIRB). AAFP and COMIRB’s Federal-wide Assurance (FWA) with the Office for Human Research Protections of the U.S. Department of Health and Human Services provides that the organizations will assure compliance with the Terms of Assurance for Federally-supported research. We will submit data collection protocols and data management plans to AAFP’s IRB for review and will request that COMIRB cede regulatory control over the project to the AAFP IRB.
To reduce the burden on the practices, we will obtain a waiver of written consent for all data collection activities coordinated by practice staff (i.e., Patient Survey, CAHPS Survey, Practice Staff Survey, Medication Review Form, Health Literacy Assessment Questions). Our data collection activities present no more than minimal risk of harm to participants. No personally identifying or sensitive information will be obtained about the respondents or other living individuals during the surveys. We do not anticipate any challenges to obtaining this waiver from the AAFP IRB.
We will obtain written consent for all qualitative interviews, which will be conducted by project staff who will obtain consent prior to commencing the interviews (i.e., Health Literacy Team Leader Interview, Health Literacy Team Member Interview, Practice Staff Member Interview, Check-In Phone Calls). For the Health Literacy Team Leader, the consent form also will cover completion of the Health Literacy Team Leader Survey and the Implementation Tracking Form.
We will ensure compliance with all federal regulations addressing patient confidentiality. All information obtained from respondents as part of this project will be kept confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c) and stored in secure locations. No patient-level clinical data will be collected and all survey data from both patients and practice staff will be anonymous. Responses provided during qualitative interviews will be held in strict confidence. All completed surveys and transcriptions from qualitative interviews will be stored in locked filing cabinets or in a secure, limited-access location accessible only to the project team. All project data will be entered or scanned into one or more electronic files for analysis. These files will be housed on one of the University of Colorado Denver’s secure computer servers. Only project staff will have access to these electronic files.
This project includes no questions of a sensitive nature.
Exhibit 1 shows the estimated annualized burden hours for the respondents' time to participate in this research.
Practice Screening Calls will be conducted as part of the recruitment process with one person from 20 different practices, with 12 practices expected to “screen-in” and be included in this project. The screening calls will take 20 minutes.
Following the completion of the Practice Screening Calls, the project team will identify 15 practices for potential participation. As part of the recruitment process, we will conduct follow-up recruitment calls with two members of each practice and expect each call to last 45 minutes.
The Health Literacy Assessment Questions will be completed twice; once at pre-implementation and again at post-implementation. These data will allow us both to assess implementation and examine the possible value of these items in assessing change as a result of quality improvement work. We estimate that five staff members from each of the 12 practices will complete the questionnaire at each time point, for a total of 120 respondents, and will require 30 minutes to complete. (The same staff members will not be targeted to complete the survey at both time points.) A staff member will distribute and collect the survey, which we estimate will take approximately five minutes per survey.
The Implementation Tracking Form will be completed at the beginning of the project and updated before each of the four Check-in Phone Calls and again at the end of the intervention. The form will be completed by the Leader of each practice’s Health Literacy Team, which will take approximately 5 minutes to complete each time, is designed solely to allow us to monitor the implementation process.
The Webinar/Orientation will take place at the beginning of the intervention and will include, on average, 4 staff members from each of the 12 practices and may take up to 2 hours.
The Patient Survey will be completed in each practice choosing to implement Tool 17 (we estimate six practices), with the goal of collecting patient feedback data as required in Tool 17. Data will be collected at pre-implementation and post-implementation. Fifty patients from each time period will be surveyed at each of the practices for a total of 600 patients. The same patients will not be targeted to complete both surveys and the surveys will be identical at the two time points. We expect that the Patient Survey will take longer to complete for patients with limited literacy skills. Although about 36% of U.S. adults have significant limitations in health literacy, we expect that practices with significant populations of low-literate patients will be interested in participating in the project. As such we expect that about 60% of participating patients will have literacy limitations and estimate that it will take them approximately 25 minutes to complete the survey. We expect the remaining 40% of patients to have stronger literacy skills. For them, completing the survey is likely to take 10 minutes. Thus, on average, survey completion should take 20 minutes. . These will be administered by a practice staff member (recruiting patients, distributing surveys, collecting surveys). It is estimated that it will take 10 minutes of the staff member’s time to administer each survey.
The Survey Using Items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) will be completed by mail or phone and will take approximately 12 minutes to complete. It will be completed by a total of about 1200 patients total at two of the practices implementing Tool 17, for the purposes of collecting patient feedback about the practice environment; 600 patients will complete it at pre-implementation and 600 at post-implementation. The same patients will not be targeted to complete both surveys.
The Medication Review Form will not be used by all of the participating practices. We estimate that 3 of the 12 practices will choose to implement Tool 8 from the Toolkit (Brown Bag Medication Review). Practices implementing Tool 8 will collect these data as part of their implementation of that Tool. For practices that do complete the Medication Review Form, we expect that about four clinic staff per practice will complete this form and each will complete it approximately five times at each time point (pre-implementation and post-implementation). Therefore, a total of 12 clinical staff will complete a total of 120 Medication Review Forms and each form will take about 30 minutes to complete.
The Practice Staff Survey will be completed twice by each staff member to allow monitoring of practice changes associated with Toolkit implementation; about 18 staff at each of the 12 practices. The pre-implementation version of the survey will take 15 minutes to complete, whereas the post-implementation version of the survey will take 20 minutes to complete. The surveys will be disseminated and collected by a member of the practice, a role which we expect to take about five minutes for each survey.
The Health Literacy Team Leader Survey is completed by the Team Leader at each of the practices at pre-implementation and post-implementation. The pre-implementation version of the survey will take 15 minutes to complete, whereas the post-implementation version of the survey will take 20 minutes to complete. These surveys will allow monitoring of changes made as part of Toolkit implementation.
During the course of the intervention, there will be four Check-in Phone Calls with the Health Literacy Team Leader at each practice to monitor implementation progress. Each call will last approximately 30 minutes.
The Health Literacy Team Leader from each practice will be interviewed at pre-implementation and post-implementation. The pre-implementation version of the interview will take about 30 minutes, whereas the post-implementation interview will take 90 minutes. These interviews as well as the Health Literacy Team Member and Practice Staff Member Interviews (described below) will provide rich data on the implementation process and needed Toolkit revisions.
The Health Literacy Team Member interview will target one member of the Health Literacy Team from each practice (other than the Team Leader) and will be conducted at the post-intervention time period. The interview is expected to last 90 minutes.
For the Practice Staff Member Interview, two other staff members per practice (24 total) will be interviewed post-implementation and these will take 30 minutes to complete.
The total annualized burden hours are estimated to be 1,049 hours.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Practice Screening Calls |
20 |
1 |
20/60 |
7 |
Follow-up Recruitment Calls |
15 |
2 |
45/60 |
23 |
Health Literacy Assessment Questions Staff Staff Administration |
120 12 |
1 10 |
30/60 5/60 |
60 10 |
Implementation Tracking Form |
12 |
6 |
5/60 |
6 |
Webinar/Orientation |
48 |
1 |
2 |
96 |
Patient Survey Patients Staff Administration |
600 6 |
1 100 |
20/60 10/60 |
200 100 |
Survey Using Items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) |
1200 |
1 |
12/60 |
240 |
Medication Review Form |
12 |
10 |
30/60 |
60 |
Practice Staff Survey – Pre-implementation Staff Staff Administration |
216 12 |
1 18 |
15/60 5/60 |
54 18 |
Practice Staff Survey – Post-implementation Staff Staff Administration |
216 12 |
1 18 |
20/60 5/60 |
72 18 |
Health Literacy Team Leader Survey – Pre-implementation |
12 |
1 |
15/60 |
3 |
Health Literacy Team Leader Survey – Post-implementation |
12 |
1 |
20/60 |
4 |
Check-in Phone Calls |
12 |
4 |
30/60 |
24 |
Health Literacy Team Leader Interview – Pre-implementation |
12 |
1 |
30/60 |
6 |
Health Literacy Team Leader Interview – Post-implementation |
12 |
1 |
1.5 |
18 |
Health Literacy Team Member Interview – Post-implementation |
12 |
1 |
1.5 |
18 |
Practice Staff Member Interview – Post-implementation |
24 |
1 |
30/60 |
12 |
Total |
2,597 |
NA |
NA |
1,049 |
Exhibit 2 shows the estimated annual cost burden to respondents, based on their time to participate in this research. The annual cost burden is estimated to be $25,709.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total burden hours |
Average hourly wage ratea |
Total cost burden |
Practice Screening Calls |
20 |
7 |
$18.52c |
$130 |
Follow-up Recruitment Calls |
15 |
23 |
$18.52c |
$426 |
Health Literacy Assessment Questions Staff Staff Administration |
120 12 |
60 10 |
$29.15d $18.52c |
$1,749 $185 |
Implementation Tracking Form |
12 |
6 |
$18.52c |
$111 |
Webinar/Orientation |
48 |
96 |
$29.15d |
$2,798 |
Patient Survey Patients Staff Administration |
600 6 |
200 100 |
$22.48b $18.52c |
$4,496 $1,852 |
Survey Using Items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) |
1600 |
240 |
$22.48b |
$5,395 |
Medication Review Form |
12 |
60 |
$29.15d |
$1,749 |
Practice Staff Survey – Pre-implementation Staff Staff Administration |
216 12 |
54 18 |
$29.15d $18.52c |
$1,574 $333 |
Practice Staff Survey – Post-implementation Staff Staff Administration |
216 12 |
72 18 |
$29.15d $18.52c |
$2,099 $333 |
Health Literacy Team Leader Survey– Pre-implementation |
12 |
3 |
$29.15d |
$87 |
Health Literacy Team Leader Survey – Post-implementation |
12 |
4 |
$29.15d |
$117 |
Check-in Phone Calls |
12 |
24 |
$29.15d |
$700 |
Health Literacy Team Leader Interview – Pre-implementation |
12 |
6 |
$29.15d |
$175 |
Health Literacy Team Leader Interview – Post-implementation |
12 |
18 |
$29.15d |
$525 |
Health Literacy Team Member Interview – Post-implementation |
12 |
18 |
$29.15d |
$525 |
Practice Staff Member Interview –Post-implementation |
24 |
12 |
$29.15d |
$350 |
Total |
2,597 |
1,469 |
NA |
$25,709 |
a Mean hourly and wage costs for Colorado were derived from the Bureau of Labor and Statistics National Compensation Survey for May 2010 (http://www.bls.gov/oes/current/oes_co.htm).10
b Hourly rate for all workers (occupation code 00-0000) estimates the cost of time for patients.
c Hourly rate for medical records and health information technician (29-2071).
d Hourly rate for Healthcare Practitioners and Technical Workers, All Other (29-9799).
Capital and maintenance costs include the purchase of equipment, computers, or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3 shows the estimated total and annualized cost to the Federal Government for conducting this research. These estimates include the costs associated with the project such as the preparation of survey administration procedures, labor costs, administrative expenses, costs associated with copying, postage, and telephone expenses, data management and analysis, preparation of final reports, and dissemination of findings/results/products. The annualized and total costs are identical since the data collection period will last for one year. The total cost is estimated to be $784,910.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total |
Annualized Cost |
Administration |
$81,654 |
$81,654 |
Research Activities |
$446,201 |
$446,201 |
Dissemination Activities |
$57,222 |
$57,222 |
Final Report |
$57,864 |
$57,864 |
Overhead |
$141,969 |
$141,969 |
Total |
$784,910 |
$784,910 |
This is a new data collection effort and does not build on a previous submission.
Time Schedule
Assuming that we receive OMB clearance by November 17, 2012, we will adhere to the following schedule of project activities:
Task |
Start Date |
End Date |
Obtain IRB approval |
4/3/2012 |
9/14/2012 |
Recruit primary care practices |
10/1/2012 |
3/29/2013 |
Collect pre-implementation data |
6/3/2013 |
6/28/2013 |
Site visit #1 |
6/3/2013 |
7/14/2013 |
Implementation period |
7/1/2013 |
1/31/2014 |
Conduct Check-in Phone Calls |
7/15/2013 |
11/1/2013 |
Collect post-implementation data |
2/1/2014 |
2/28/2014 |
Site visit #2 |
1/13/2014 |
2/28/2014 |
Analyze data |
7/15/2013 |
5/1/2014 |
Revise Toolkit |
8/1/2013 |
9/1/2014 |
Prepare Technical Assistance Guide |
4/1/2014 |
7/1/2014 |
Prepare Case Studies |
4/1/2014 |
7/1/2014 |
Prepare Manuscripts |
3/1/2014 |
8/1/2014 |
Prepare Final Report |
5/1/2014 |
9/21/2014 |
Publication and Use of Findings
AHRQ plans to submit at least one manuscript to a peer-reviewed journal that describes the findings on the implementation of the Health Literacy Universal Precautions Toolkit. In addition, AHRQ plans to publish three products on its Web site:
A second edition of the Health Literacy Universal Precautions Toolkit.
A Technical Assistance Guide to inform use of the Toolkit by practice facilitators. The Guide will be sensitive to differences in practice settings (e.g., size and composition of practice) and provide suggestions for adapting the Toolkit for these different circumstances.
Case studies that highlight lessons learned from the demonstration sites.
Analysis Plans
This project represents a small pilot demonstration and emphasizes the goal of identifying potential improvements to the Toolkit and assessing the value of the measures for use in health literacy-related quality improvement work. We will have small sample sizes and will be involving a purposefully selected sample of practices to implement the Toolkit. As a result, data analyses will be exploratory in nature and will not be intended to provide results that can be generalized to the full population of primary care practice settings. The data will be analyzed in the following ways:
Qualitative Analyses
Using ATLAS.ti software, the project team will code qualitative interviews with practice staff to identify key concepts and themes related to Toolkit implementation. Analyses will be used to (1) document the extent to which each practice implemented the two primary tools (Tools 1 and 2) and their two selected tools; (2) explore the challenges, successes, barriers, and facilitators faced by practices while implementing the tools; (3) identify common themes and issues related to the role of health literacy within the practice and how it may have changed over the course of the project, and (4) collect information about recommended changes to the Toolkit. Data from the Implementation Tracking Form will be used to assess the timeline with which practices were able to implement specific tools.
Quantitative Analyses
Quantitative data will be summarized and described using univariate measures – means and standard deviations for continuous data, and proportions for categorical data. For assessing bivariate associations, we will use standard statistical techniques (e.g., Chi-square correlation, linear regression, ANOVA), depending on included items’ levels of measurement. We will compare pre- and post-implementation data to explore whether the measures appear to be sensitive to change over time in the health-literacy related systems and procedures of participating practices.
Patient-Level Data: Data from the Patient Survey and the CAHPS item set will be used to assess whether these measures are able to show change in patient perceptions of the quality of communication with providers and staff over time. We will compare these data with qualitative data on changes made by the practices to see if these different data sources show consistent results We also will compare CAHPS data collected in two practices with the same practices’ Patient Survey data to see whether the two data sources produce similar findings regarding patient perceptions. Overall participation rates and the proportion of completed surveys from fielding the CAHPS Item Set for addressing health literacy will be calculated. Data from the Medication Review Form will be used to assess possible change over time in the type and severity of medication regimen errors over time.
Practice-Level Data: Data from the Practice Staff Survey, Health Literacy Team Leader Survey, and the Health Literacy Assessment Questions will be used to examine changes to systems and procedures related to health literacy (e.g., systems for referring patients to health literacy resources in the community). We will compare these data to both patient and qualitative data to see whether these different data sources provide consistent findings regarding possible change over time in the health literacy environments of participating practices.
AHRQ does not seek this exemption.
List of Attachments
Attachment A Health Literacy Universal Precautions Toolkit
Attachment B Practice Recruitment Email
Attachment C Practice Screening Calls
Attachment D Follow-up Recruitment Calls
Attachment E Health Literacy Assessment Questions
Attachment F Implementation Tracking Form
Attachment G Webinar/Orientation Presentation
Attachment H Project Team Review of Practice’s Health Literacy Environment
Attachment I.1 Patient Survey
Attachment I.2 Cover memo for Patient Survey Completed on Touch Screen Tablet
Attachment I.3 Cover memo for Patient Survey Completed on Hardcopy Form
Attachment J Survey Using Items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS)
Attachment K Medication Review Form
Attachment L Practice Staff Survey – Pre-Implementation
Attachment M Practice Staff Survey – Post-Implementation
Attachment N Health Literacy Team Leader Survey – Pre-Implementation
Attachment O Health Literacy Team Leader Survey – Post-Implementation
Attachment P Consent Form for Qualitative Interview Participation
Attachment Q Health Literacy Team Leader Interview – Pre-Implementation
Attachment R Health Literacy Team Leader Interview – Post-Implementation
Attachment S Check-in Phone Calls
Attachment T Health Literacy Team Member Interview
Attachment U Practice Staff Member Interview
References
US Department of Health and Human Services. 2000. Healthy people 2010: understanding and improving health: U.S. Government Printing Office.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, Holland A, Brasure M, Lohr KN, Harden E, Tant E, Wallace I, Viswanathan M. Health Literacy Interventions and Outcomes: An Updated Systematic Review. Evidence Report/Technology Assessment No. 199. (Prepared by RTI International–University of North Carolina Evidence-based Practice Center under contract No. 290-2007-10056-I. AHRQ Publication Number 11-E006. Rockville, MD. Agency for Healthcare Research and Quality. March 2011.Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika. 16, 297-334.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, Holland A, Brasure M, Lohr KN, Harden E, Tant E, Wallace I, Viswanathan M. Health Literacy Interventions and Outcomes: An Updated Systematic Review. Evidence Report/Technology Assessment No. 199. (Prepared by RTI International–University of North Carolina Evidence-based Practice Center under contract No. 290-2007-10056-I. AHRQ Publication Number 11-E006. Rockville, MD. Agency for Healthcare Research and Quality. March 2011.
Kutner M., Greenberg E, Jin Y, et al. 2006. The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. Washington DC: National Center for Educational Statistics. NCES 2006-483.
Powell CK, Kripalani S. Brief report: Resident recognition of low literacy as a risk factor in hospital readmission. J Gen Intern Med. Nov 2005;20(11):1042-1044.
Bass PF, 3rd, Wilson JF, Griffith CH, Barnett DR. Residents' ability to identify patients with poor literacy skills. Acad Med. Oct 2002;77(10):1039-1041.
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | ctac |
| File Modified | 2012-08-21 |
| File Created | 2012-08-21 |