SUPPORTING STATEMENT
Part A
Collection of Information for: Agency for Healthcare Research and Quality’s (AHRQ) Guide to Improving Patient Safety in Primary Care Settings by Engaging Patients and Families – Evaluation
Version February, 2017
Agency for Healthcare Research and Quality (AHRQ)
Table of Contents
1. Circumstances that make the collection of information necessary - 3 -
2. Purpose and Use of Information - 8 -
3. Use of Improved Information Technology - 10 -
4. Efforts to Identify Duplication - 10 -
5. Involvement of Small Entities - 10 -
6. Consequences if Information Collected Less Frequently - 10 -
7. Special Circumstances - 10 -
8. Federal Register Notice and Outside Consultations - 10 -
8.b. Outside Consultations - 11 -
9. Payments/Gifts to Respondents - 11 -
10. Assurance of Confidentiality - 12 -
11. Questions of a Sensitive Nature - 14 -
12. Estimates of Annualized Burden Hours and Costs - 14 -
13. Estimates of Annualized Respondent Capital and Maintenance Costs - 16 -
14. Estimates of Total and Annualized Cost to the Government - 16 -
15. Changes in Hour Burden. - 17 -
16. Time Schedule, Publication and Analysis Plans - 17 -
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. Research that develops and presents scientific evidence regarding all aspects of health care; and
2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. Initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Background for this collection
There is a substantial evidence base showing that engaging patients and families in their care can lead to improvements in patient safety. Since the 2001 release of To Err is Human(Kohn, Corrigan, & Donaldson, 2000)1, there has been an undeniable focus on improving patient safety and eliminating patient harm within acute care. What is not as well documented is how to achieve these improvements in primary care settings.
Patient and Family Engagement (PFE) strategies for acute care settings include, among others: patient and family advisory committees; membership on patient safety oversight bodies at both operations and governance levels; consultation in the development of patient information material; engaging patients in process improvement or redesign projects; rounding with patients and families; patient and family participation in clinical education programs, and welcoming patients and families to work alongside providers and health systems employees on transparency, culture change and high reliability organization initiatives.2–5
Although the field of PFE in patient safety for hospitals and health systems is maturing, leveraging PFE to improve patient safety in non-acute settings is in its infancy. Building sustainable processes and practice-based infrastructure are crucial to improving patient safety through patient and family engagement in primary care.
In response to the limited guidance available for primary care practices to improve safety through patient and family engagement, the Agency for Healthcare Research and Quality (AHRQ) has sponsored the development of a Guide to Improving Safety in Primary Care Settings by Engaging Patients and Families (hereafter referred to as the Guide). The comprehensive guide will provide primary care practices with interventions that they can use to engage patients and families in ways that lead to improved patient safety. It will include explicit instructions to help primary care practices, providers, and patients and families adopt new behaviors. Additionally, staffing models and infrastructure to support both PFE and patient safety within primary care environments differs significantly from the hospital setting. Thus, the output of the Guide differs significantly from the AHRQ Guide to Patient and Family Engagement in Hospital Quality and Safety in that the primary focus is on engaging patients and families to improve patient safety (e.g. close gaps in communication, reduce the risk of fragmentation of care and diagnostic errors, and improve medication self-management. The interventions within the Guide are tailored for the ambulatory care environment, particularly for primary care patients, physicians and practice staff.
The Guide will also build upon the AHRQ’s prior work on PFE in the patient centered medical home.7 In its 2010 report, several approaches to PFE were identified within the office based practices to support patient safety and improved patient outcomes. The Guide moves to the next step to provide implementation guidance and supportive tools for primary care practices to adopt strategies to improve patient safety through PFE.
The Guide and its development are prefaced on several key insights relevant to primary care including:
Active engagement requires organizational commitment to hearing the patient and family voice and action by leadership to include them as central members of the healthcare team.
Patients and families expect and increasingly demand meaningful engagement in harm prevention efforts. Without this, engagement efforts may not achieve optimal outcomes.
Institutional courage is required to openly share patient safety vulnerabilities and proactively engage patients in developing solutions that prevent harm.
Supportive infrastructure is needed to design sustainable processes to support integration of PFE into all facets of care delivery across the care continuum.
When done well, patient engagement yields important and measurable results.5 When not done well, PFE activities may disenfranchise patients, contribute to misunderstanding about risk, result in lack of trust between providers and their organizations, and create fissures among members of the clinical care team.8–11
With these insights as a basis, three precepts undergird our approach to development for the Guide. The Guide interventions must yield:
Meaningful relationship-based engagement for patients and families and primary care providers.
Innovation and enabling technologies to support engagement, shared decision making and patient safety.
Workable processes yielding sustainable engagement opportunities for patients, families, providers, and practice staff.
The Guide will be principally (but not exclusively) meeting the needs of practices that have not already implemented effective PFE structures or processes. An environmental scan revealed several promising interventions for consideration for inclusion in the Guide. The four interventions selected as part of the Guide include:
Teach-back
Be Prepared to Be Engaged
Medication Management
Warm Handoff
The interventions will be compiled into a Guide for adoption by primary care practices. The environmental scan also yielded several important implications for Guide development including:
Engagement efforts in primary care to date have focused on the patient as the agent of change with limited guidance to providers on how to support patients in these efforts.
Many interventions are focused heavily on educational efforts alone, either for the patient, the provider, or the practice.
Few of the tools and interventions identified are immediately usable without the need for additional development or enabling materials to support sustainable adoption.
Health equity and literacy considerations are limited. Tools for patients are often at a relatively high level of literacy, and/or health literacy is required for use.
Current interventions, tools, and toolkits have a high level of complexity that may impede adoption.
Existing evidence-based interventions are being refined to reduce complexity and enhance the opportunity for implementation. Implementation development activities including guidance for each intervention and the Guide as a whole are currently underway. Guide field testing will evaluate the implementation challenges faced by primary care practices whereby offering an opportunity to revise the Guide materials for optimal implementation success prior to widespread dissemination.
The Guide will be made publicly-accessible through the AHRQ Web site for easy referral, access, and use by other healthcare professionals and primary care practices. AHRQ recognizes the importance of ensuring that the Guide will be useful, well implemented and effective in achieving the stated goals of improving patient safety by engaging patients and families. Thus, the purpose of the Field Testing evaluation is to gain insight on the implementation challenges identified by the twelve primary care practices field testing the Guide. The Guide materials will be revised in an effort to overcome these implementation challenges prior to broad dissemination.
The specific goals of the proposed Guide field testing evaluation are to examine the following:
The feasibility of implementing a minimum of two of the four Guide interventions within twelve medium or large primary care practices?
The challenges to implementing the interventions at the patient, clinician, practice staff, and practice level?
The uptake and confidence among primary care practices to improve patient safety through patient and family engagement.
How the implementation of two of the four Guide interventions changes the perception of patient safety among patients, clinicians, and practice staff.
How the implementation of two of the four Guide interventions changes the perception of patient and family engagement among patients, clinicians, and practice staff.
Whether primary care practices will continue to use the Guide (or its interventions) beyond the period of field testing and evaluation (i.e. examine sustainability).
How patients, clinicians, and practice staff would recommend changes to the interventions and the Guide to enhance sustainability.
The conceptual frame of delivery system science grounds this data collection effort. Delivery system science provides a systematic means and opportunity to promote learning, to answer questions, and to test hypotheses around organizational factors and design, infrastructure, policies, and payment mechanisms. The question we must address is not “Does it work?” but rather “How and in what contexts does the new intervention work or can it be amended to work?” This requires a formative evaluation approach that considers the full path of the intervention from activities to engage participants (i.e., patients and family members, providers, practice staff, and practices as units of measurement) and change how they act to the expected changes in clinical processes and outcomes (i.e., interventions and engagement activities). To this end, a primarily qualitative approach will be conducted for this formative evaluation.
To achieve the goals of the project, the following data collections will be implemented during the Field Testing evaluation:
Baseline Practice Assessment of Primary Care Practices (Appendix A). This pen and paper survey will be administered to the twelve primary care practice champions immediately following the recruitment as part of the Guide Field Test and prior to commencing implementation of the Guide. Information collected includes: i) practice name and location (e.g., city and State); ii) non-identifying demographic information about the practice (e.g., number of clinicians by type, number of patients served by the practice, payer mix of patients served by practice, race and ethnicity of patients served by practice); iii) general descriptive information on the practice’s experience with patient safety and quality improvement activities (e.g., current experience with Guide interventions, patient safety culture routinely measured); iv) information related to the practice’s affiliation with larger health system; and v) information related to any competing priorities or practice improvement initiatives (e.g., patient centered medical home designation, etc.).
Post-Implementation Focus Groups for Patients and Families (Appendix B). Information from patients on their experiences with the Guide and its interventions will be solicited twice during the Field test – once at 3-months and again at 6-months post-implementation of the Guide. Each patient and family focus group will aim to recruit between 6-8 participants and solicit feedback from patients and family members on their experiences with the Guide materials. Information collected will include: i) perceptions on patient safety in primary care practices; ii) perceptions of patient and family engagement in primary care practices; iii) feedback from the patient perspective on the Guide materials and their general use; iv) feasibility of adopting the patient and family focused intervention materials in practice; v) feedback on the patients and family experiences of the Guide and its relation to patient safety.
Baseline Practice Readiness Assessment (Appendix C). Information from primary care practices about their readiness to adopt patient and family engagement strategies will be solicited through telephone interviews with practice staff champions. Information collected will include: i) descriptive information on the person completed the interview (e.g., position in the practice, length of employment, experience in implementing patient safety improvements); ii) description of the patient safety culture of the primary care practice (e.g., teamwork, communication, patient safety culture, etc.,); iii) perceptions of patient and family engagement within the practice; iv) perceptions of change management strategies, challenges, and barriers (e.g., leadership support, competing initiatives, other production pressures); v) capacity for ongoing internal measurement and assessment of intervention. This process will also solicit general information the interviewee would like to share about the practice’s readiness to implement the Guide strategies.
Post-Implementation Interviews of Primary Care Clinicians (Appendix D). Information from primary care clinicians (e.g., physicians, nurses, nurse practitioners, social workers, etc.) on their experiences with the Guide and its interventions will be solicited twice during the Field test – once at 3-months and again at 6-months post-implementation of the Guide. Interviews with 2 and no more than 3 primary care clinicians per practice will be conducted during Field Testing to solicit feedback on their experiences with the Guide materials. Information collected will include: i) perceptions on patient safety in primary care practices; ii) perceptions of patient and family engagement in primary care practices; iii) feedback from the clinician perspective on the Guide materials and their general use; iv) feasibility of adopting the intervention materials in practice; v) feedback on the clinicians’ experiences of the Guide and its relation to patient safety.
Post-Implementation Focus Groups for Practice Staff Members (Appendix E). Information from practice staff members (e.g., practice administrators, medical assistants, schedulers, practice facilitators, other non-clinical staff, etc.) on their experiences with the Guide and its interventions will be solicited twice during the Field test – once at 3-months and again at 6-months post-implementation of the Guide. Focus groups with between 6-8 primary care practice staff will be conducted in each practice during Field Testing to solicit feedback on their experiences with the Guide materials. Information collected will include: i) perceptions on patient safety in primary care practices; ii) perceptions of patient and family engagement in primary care practices; iii) feedback from the practice staff perspective on the Guide materials and their general use; iv) feasibility of adopting the intervention materials in practice; v) feedback on the practice staff’s experiences of the Guide and its relation to patient safety.
Monthly Telephone Interviews with Practice Champions (Appendix F). This survey will be completed over the phone on a monthly basis with the practice champions from the twelve primary care practices engaged in the Field Testing of the Guide. Information collected will include: i) current progress towards implementation of the intervention(s); ii) movement towards target goals set in the prior meeting; iii) barriers to implementation; iv) facilitators of implementation; v) perceived impact on patient safety; vi) perceived impact on patient and family engagement; vii) plans for the coming weeks/months.
The Field Test evaluation of the Guide is being conducted by AHRQ through its contractor the MedStar Health Research Institute (MHRI) pursuant to (1) 42 U.S.C. 299b-7; (2) AHRQ’s authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness, and value of health care services, 42 U.S.C. 299a(a)(1); and (3) AHRQ’s authority to support the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policymakers, and educators, 42 U.S.C. 299(b)(2).
AHRQ will use the information collected through this Information Collection Request to assess the feasibility of adopting the Guide to Improving Patient Safety in Primary Care Settings by Engaging Patients and Families. A mixed-methods approach will be used to identify barriers and facilitators to uptake and sustainability, and to answer the question “How and in what contexts do the chosen interventions work or can they be amended to work”, rather than “Do they work?” Testing will occur at up to 12 primary care sites and feasibility of implementation will be assessed at the patient, provider, and practice levels. The information collected will be used to revise the Guide in order to promote widespread adoption of the Guide.
The specific purpose of each of the data collection instruments is described below:
Baseline Practice Assessment of Primary Care Practices (Appendix A). The baseline data collection instruments will be used to provide baseline descriptive characteristics of the practices engaged in the Field Test. They will be used to provide information, in aggregate, about the typical practice that engaged in the Field Test. The baseline assessment will also be used to establish a baseline level of experience in the Guide interventions, patient safety culture, and any competing priorities within the primary care practices. This is important for the evaluation component of the project as it will provide information on the type of practices where the Guide interventions were field tested and support generalizability and acceptability of the Guide revisions.
Post-Implementation Focus Groups for Patients and Families (Appendix B). The data collected is highly qualitative and will be used to assess the challenges of implementation and recommended revisions to the Guide materials to support adoption of the interventions and the Guide in practice.
Baseline Practice Readiness Assessment (Appendix C). This data collection will be used to assess each practice’s readiness to change or adopt the Guide interventions into practice. The assessment will provide information on an individual practices’ change readiness, a critical component influencing implementation effectiveness of patient safety practices. This assessment will help in identifying practices that may perceive greater challenges in implementation due to readiness constraints. The assessment will also provide insight on how a practice may self-assess readiness prior to embarking on a practice improvement initiative. This information will be used to craft a self-assessment to be included in the final revised Guide.
Post-Implementation Interviews of Primary Care Clinicians (Appendix D). The data collected is highly qualitative and will be used to assess the challenges of implementation and recommended revisions to the Guide materials to support adoption of the interventions and the Guide in practice.
Post-Implementation Focus Groups for Practice Staff Members (Appendix E). The data collected is highly qualitative and will be used to assess the challenges of implementation and recommended revisions to the Guide materials to support adoption of the interventions and the Guide in practice.
Monthly Telephone Interviews with Practice Champions (Appendix F). This data collection tool will be used to gather information from practice champions on a monthly basis regarding challenges to implementation. This information will be used to provide insight on the need for technical assistance in adopting the Guide into practice and the common challenges to implementation encountered by primary care practices. This information will be used to revise the Guide.
These evaluation instruments (Appendices A-F) are designed to capture primarily qualitative data (Appendix B-F) with some quantitative data (Appendix A; Participant Characteristics Table). No claim is made that the results from this evaluation will be generalizable in a statistical sense. However, every attempt will be made to recruit primary care practices that are representative of diverse geographic locations as well as diverse patient populations served including practices that serve low income patient populations. The goals of the evaluation are aimed at determining the challenges to implementation of the Guide in order to revise the Guide materials prior to dissemination.
All of the research described herein will rely on paper data collection instruments in the form of interview and focus group guides to be used by the facilitators and moderators for each activity. Interviews and focus groups will be audio recorded for the purpose of transcription and coding. There will otherwise be no automated, electronic, or other technological collection techniques or other forms of information technology used for the information collection.
The first phase of the parent study involved an extensive environmental scan (previously cleared under a Generic Clearance by AHRQ) to review the literature, including published, unpublished, and internet sources to identify existing interventions and resources pertinent to Guide development. The environmental scan revealed key gaps which are being addressed by the design of the Guide and subsequently evaluated by this data collection effort. To our knowledge, this does not involve a duplication of any existing efforts.
The information being collected under this effort will reflect the variety of settings in which the Guide will actually be used. This includes medium and large primary care practices. However, we do not anticipate that any of the practices volunteering to participate would be considered small businesses.
This is a onetime data collection
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on August 10, 2016, on page 52864 for 60 days (see Attachment G). No substantive comments were received.
AHRQ has convened an external Technical Expert Panel (TEP) to provide expertise and guidance to develop the plan and design for this full project (the Guide) and each phase including the Guide development and evaluation for which this data collection is designed. The Technical Expert Panel (TEP) is scheduled to meet a total of six times over the course of the 38-month project which started in September 2015, approximately twice in each year of the project. The first meeting was held on Friday, January 29th via Webex. This virtual meeting convened 28 participants including TEP members, AHRQ project staff and MedStar project staff for a 2-hour period. The overall purpose of this meeting was to set expectations of the TEP members, review work that had been conducted as well as the planned data collections. The second TEP meeting was convened in person in Washington, DC on March 18th. The meeting brought together 14 attendees and included a discussion and review of the completed phases of the project (e.g. Environmental scan) and deliberations on the interventions that will comprise the Guide. The TEP members also provided input on the interventions and the planned data collection effort set forth in this clearance. The TEP will convene twice during the 12-month field test as set forth in this clearance.
Patient and Family Member Participants
Patient and family participants in the described data collection effort will be offered a token of appreciation for their effort. The incentive is to thank the participant for the burden of the data collection. Focus groups for patients and families will be conducted within the practice community in the evening to enhance the opportunity for broad inclusion of patients. The focus group will not be conducted within the primary care practice in conjunction with a usual primary care visit, thus the patient and/or their family member will be required to assume an additional burden to participation of travel and time. A token of appreciation of $50 will be offered to patients and family members for a 90-minute focus group for patients and family members. Patient focus groups will be held during the day and/or evening.
Based on the contractor (MedStar’s) previous recruitment efforts for similar studies as well as the published literature, the requested stipend represents the minimum amount necessary to recruit, secure participation, and ensure adequate response rates in data collection efforts using similar approaches.12–15 Arthur, Smith, and Wright conducted a series of focus groups with women conducted within the community and outside of standard clinical practice to explore barriers to and experiences of care after open heart surgery, heart attack, or percutaneous intervention.16 Recruitment challenges resulted in offering a $50 per focus group incentive to each participant to support initial focus group participation and enhance longitudinal response rates. The proposed evaluation is similar to this study, seeking to recruit patients and family members along the spectrum of engagement and from socioeconomically diverse practice communities. Here, engagement in healthcare activities may range from disenfranchised or disengaged (including those with limited experience and exposure to the health system) to patients who are activated and empowered to advocate on behalf of their own healthcare and the healthcare of others.17 Research demonstrates that individuals with limited experience and exposure to the healthcare system, those with lower health consumption, patients from deprived and low income neighborhoods, the elderly and youth, and those from lower socioeconomic backgrounds are less likely to participate in research.14,18
Clinical and Non-Clinical Staff
Clinical and non-clinical staff (e.g. practice staff) will be offered a token of appreciation to participate in this data collection effort in an endeavor to enhance response rates and ensure longitudinal participation in the proposed evaluation.
Non-clinical staff from each of the 12 primary care practices will be asked to participate in one 90-minute focus group at 3-months and another at 6-months post-implementation. It is essential that the evaluation not disrupt clinical care and primary care practice operations, thus the focus groups will be held after normal practice operating hours and outside the practice setting (e.g. at a space within the community) representing an additional time and travel burden for non-clinical staff. Thus, an incentive of $50 per focus group participant is required to ensure adequate participation and response rates from the non-clinical staff participants.
Clinical staff from each of the 12 primary care practices will be asked to participate in one up to 60-minute interview at 3-months and another interview at 6-months post-implementation. To minimize disruption to the primary care clinicians’ clinical practice, interviews will be scheduled on non-clinical days or at times that are outside of standard primary care practice operating hours. To ensure adequate participation and response rates of the clinical staff as part of this evaluation, we are offering clinical staff an incentive of $50 per interview.
The data collection described here is similar in scope and approach to several completed and ongoing projects of the contractor. Here, incentives including offering refreshments and monetary tokens of appreciation have been important to ensure diverse representation of clinical and non-clinical staff in data collection efforts.6,19–24
The contractor also consulted with patient representatives from the target demographics (e.g. range of socioeconomic status, age, low and high users of healthcare) as well as primary care clinicians and non-clinical practice staff members to examine the necessity of incentives to ensure adequate response rates to the information collection. The minimum recommended incentive of $50 per focus group or interview was described as necessary to achieve an adequate response rate to the data collection for each of our target populations.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
In the baseline assessment, descriptive information about the practice is collected. The names of the contact person and medical director are collected only to allow future communication between MedStar and the practice and to facilitate arranging site visits, interviews, and focus groups. The person who completes the baseline assessment is not identified and is not necessarily the contact person or medical director. These names will be used for communication only and will not be made public.
For the interviews and focus groups, information that can directly identify the respondent, such as name and/or social security number will not be collected. The following participant characteristics will be collected for individuals volunteering to test the intervention materials. These data will be presented in aggregate and used to describe the sample of individuals providing feedback. No information that will otherwise allow for individual identification of participants will be collected.
Table 1 Participant Characteristics
Characteristic |
Collection Method |
Format* |
Participant Category |
Interview/Focus Group |
Provider Staff Patient Administrator Other |
Age |
Interview/Focus Group |
Age (years) |
Sex |
Interview/Focus Group |
Male Female |
Education Level |
Interview/Focus Group |
Elementary High School Diploma Some College Associate’s degree Bachelor’s degree Master’s degree Professional Degree Doctorate |
Race |
Interview/Focus Group |
White Black or African American American Indian or Alaska Native Asian Two or more races |
Ethnicity |
Interview/Focus Group |
Hispanic or Latino Not Hispanic or Latino |
Location |
Interview/Focus Group |
City, State |
Location |
Interview/Focus Group |
Urban Inner City Rural Suburban Other |
Priority Population |
Interview/Focus Group |
Yes No Unknown |
Self-reported Health Status |
Focus Group (Patients only) |
Excellent Very Good Good Fair Poor |
Chronic Disease |
Focus Group (Patients Only) |
Yes/No |
*Each characteristic must include an option for did not respond/did not provide an answer |
||
Participants will also receive the following confidentiality statements printed on any respondent materials:
“The confidentiality of your responses are protected by Sections 944(c) and 308(d) of the Public Health Service Act [42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)]. Information that could identify you will not be disclosed unless you have consented to that disclosure.”
Data collection and storage mechanisms will be as described. Information collected during the course of the evaluation will be maintained in a secure HIPAA-compliant data server. All data collection will be conducted using the contractor, MedStar’s REDCap™ research data capture database. REDCap™ is a mature, secure web application for building and managing online surveys and databases. While REDCap™ can be used to collect virtually any type of data, it is specifically geared to support data capture for research studies. The REDCap™ Consortium is composed of 1,711 active institutional partners in 96 countries who utilize and support REDCap™ in various ways. REDCap™ can be established to support data entry forms and to conduct web-enabled surveys. The patient and family engagement guide will also use REDCap™ project space to securely store any documents received from the intervention practices during the project. The MedStar Health Research Institute is a REDCap™ project collaborator site with a robust history of using this method for data collection.
Data collected will be primarily qualitative in nature (Appendices B-F). Responses to focus groups will be recorded and notes for each session transcribed. Transcription files will be uploaded to the REDCap™ file repository for security. Paper files will be retained in a locked file cabinet within Dr. Smith’s office at MedStar Health. A case record form will be created within REDCap™ to record output from the thematic reviews of the transcripts to facilitate reporting and feedback to participants and to the study team.
There are no questions of a sensitive nature
Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in this evaluation of the Guide during field testing. Two formative evaluations will be conducted during field testing in twelve primary care practices in at least two geographic regions of the United States. Evaluation efforts will include collection of baseline practice level data prior to Guide implementation and two separate rounds focus groups and interviews conducted 3-months and 6-months after Guide implementation. Baseline assessments will be conducted on paper via phone consultation between the Contractor and the local practice champion and will take between 30-60 minutes. Site visits and field observations will be conducted over two, two day periods. Patient focus groups will be conducted at the 3- and 6-month evaluation periods; each lasting between 60-90 minutes. Practice staff focus groups will be conducted during each of the site visits, conducted outside regular practice hours, and last between 60-90 minutes. Primary care clinicians will be interviewed will last approximately up to 60 minutes. We estimate that approximately 12 individuals will participate in the monthly telephone interviews over the 9-month implementation and evaluation period.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Appendix A: Baseline Practice Assessment |
12 |
1 |
1 |
12 |
Appendix B: Post-Implementation Focus Group for Patients and Family Members |
72 |
2 |
1.5 |
216 |
Appendix C: Interview Guide- Baseline Practice Readiness |
12 |
1 |
.75 |
9 |
Appendix D: Post-Implementation Interview Protocol- Providers |
24 |
2 |
.75 |
36 |
Appendix E: Post-Implementation Focus Group Protocol- Practice Staff |
72 |
2 |
1.5 |
216 |
Appendix F: Topic guide for Telephone Protocol- Guide Practice Champions |
12 |
6 |
.5 |
36 |
Total |
204 |
NA |
NA |
525 |
Exhibit 2 shows the estimated annualized cost burden based on the respondents’ time to participate in this project. The total cost burden is estimated to be $18,629.16.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Appendix A: Baseline Practice Assessment |
12 |
12 |
$37.40a |
448.80 |
Appendix B: Post-Implementation Focus Group for Patients and Family Members |
72 |
216 |
$23.23a |
5,017.68 |
Appendix C: Interview Guide- Baseline Practice Readiness |
12 |
9 |
$37.40a |
336.60 |
Appendix D: Post-Implementation Interview Protocol- Providers |
24 |
36 |
$94.48b |
3401.28 |
Appendix E: Post-Implementation Focus Group Protocol- Practice Staff |
72 |
216 |
$37.40a |
8078.40 |
Appendix F: Topic guide for Telephone Protocol- Guide Practice Champions |
12 |
36 |
$37.40a |
1346.40 |
Total |
204 |
525 |
|
18,629.16 |
* National Compensation Survey: Occupational wages in the United States May 2015, “U.S. Department of Labor, Bureau of Labor Statistics.”
a Based on the mean wages for Miscellaneous Healthcare Worker (Code 29-9090)
b Based on the mean wages for Internists, General (Code 29-1063)
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3a and Exhibit 3b shows the estimated annualized cost to the government for the contractor and government personnel for developing, maintaining, and managing the database and analyzing the data and producing reports for each year in which data are collected. The cost is estimated to be $388,316.16 annually.
Exhibit 3a. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Project Development |
$19,035.20 |
Data Collection Activities |
$158,199.67 |
Data Processing and Analysis |
$38,930.39 |
Publication of Results |
$6,011.73 |
Project Management |
$24,046.93 |
Overhead |
$142,092.24 |
Total |
$388,316.16 |
Exhibit 3b. Federal Government Personnel Cost
Activity |
Federal Personnel |
Hourly Rate |
Estimated Hours |
Cost |
Project oversight to include data collection oversight and review of results |
Project Officer GS14 |
68.05 |
25 |
$1,701.38 |
Health Scientist Administrator GS 13 |
50.21 |
25 |
$1,255.25 |
|
Total |
$2,956.63 |
|||
Annual salaries based on 2016 OPM Pay Schedule for Washington/DC area: http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2016/DCB.pdf
This is a new collection of information, thus no changes in hour burden is expected or reported here.
A final project report will be delivered to AHRQ on September 20, 2018. Current publication plans include:
Publication of the Environmental Scan results – Fall, 2016
Publication of the results of the field test – Summer 2018
Guide field testing will begin in late January 2017 upon submission of the final Guide materials to AHRQ on January 13, 2017 and continue through December 31, 2017. The Field testing, evaluation, and Guide Revision (Task 7) will be initiated and completed within one calendar year. Activities during this period will include:
Recruitment of Primary Care Practices – January-April 2017
Field Testing of the Guide – February-November, 2017
Evaluation of the Guide – February-December, 2017
Conduct Baseline Evaluation – February-April 2017
Conduct Interim Analysis – May-July, 2017
Conduct Final Evaluation – August-November 2017
Analyze and Process Data – November-December 2017
Evaluation Report – January 2018
Publication of the results of the field test will be developed along with the evaluation report and submitted for publication in early January 2018.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A: Baseline Practice Assessment
Attachment B: Implementation Focus Group for Patients and Family Members
Attachment C: Interview Guide-Baseline Practice Readiness
Attachment D: Post-Implementation Interview Protocol- Providers
Attachment E: Post-Implementation Focus Group Protocol- Practice Staff
Attachment F: Topic guide for Telephone Protocol- Guide Practice Champions
Attachment G: Federal Register Notice
1. Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. Committee on Quality of Health Care in America I of M, ed. 2000.
2. Berger Z, Flickinger TE, Pfoh E, Martinez KA, Dy SM. Promoting engagement by patients and families to reduce adverse events in acute care settings: a systematic review. BMJ Qual Saf. 2014;23(7):548-555. doi:10.1136/bmjqs-2012-001769.
3. Kushner C, Davis D. Improving safety: engaging with patients and families makes a difference! Healthc Q. 2014;17 Spec No:41-44. http://www.ncbi.nlm.nih.gov/pubmed/25344615. Accessed July 11, 2015.
4. Helmchen, Lorens; Richards, Michael R.; McDonald TB. How Does Routine Disclosure of Medical Error Affect Patients’ Propensity to Sue and Their Assessment of Provider Quality?: Evidence From Survey Data. Med Care. 2010;48(11):955-961. http://journals.lww.com/lww-medicalcare/Abstract/2010/11000/How_Does_Routine_Disclosure_of_Medical_Error.4.aspx.
5. Smith KM, Hatlie MJ, Mayer DB, McDonald TB. A 10-year journey engaging patients in patient safety education, research and improvement. In: International Society for Communication Science and Medicine. Montecatini Terme, Italy; 2015:SP57, p36. http://iscome.org/sites/default/files/iscome-2015/LibrettoProgramma.pdf.
6. McDonald TB, Helmchen LA, Smith KM, et al. Responding to patient safety incidents: the “seven pillars”. Qual Saf Health Care. 2010;19(6):e11. doi:10.1136/qshc.2008.031633.
7. Scholle S, Torda P, Peikes D, Han E, Genevro J. Engaging Patients and Families in the Medical Home.; 2010. http://pcmh.ahrq.gov/sites/default/files/attachments/Engaging Patients and Families in the Medical Home.pdf.
8. Schwappach DLB. Review: engaging patients as vigilant partners in safety: a systematic review. Med Care Res Rev. 2010;67(2):119-148. doi:10.1177/1077558709342254.
9. Entwistle VA, Mello MM, Brennan TA. Advising patients about patient safety: current initiatives risk shifting responsibility. Jt Comm J Qual Patient Saf. 2005;31(9):483-494. http://www.ncbi.nlm.nih.gov/pubmed/16255326. Accessed July 21, 2015.
10. Entwistle VA. Differing perspectives on patient involvement in patient safety. Qual Saf Health Care. 2007;16(2):82-83. doi:10.1136/qshc.2006.020362.
11. Lyons M. Should patients have a role in patient safety? A safety engineering view. Qual Saf Health Care. 2007;16(2):140-142. doi:10.1136/qshc.2006.018861.
12. Arthur HM, Wright DM, Smith KM. Women and Heart Disease: The Treatment May End but the Suffering Continues. Can J Nurs Res. 2001;33(3):17-29.
13. Smith K, Winegard K, Hicks AL, McCartney N. Two Years of Resistance Training in Older Men and Women: The Effects of Three Years of Detraining on the Retention of Dynamic Strength. Can J Appl Physiol. 2003;28(3):462-474. doi:10.1139/h03-034.
14. Koopmans B, Nielen MM, Schellevis FG, Korevaar JC. Non-participation in population-based disease prevention programs in general practice. BMC Public Health. 2012;12:1. doi:10.1186/1471-2458-12-856.
15. Ver Ploeg M, Moffitt RA, Citro CF. Studies of Welfare Populations: Data Collection and Research Issues. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs. Washington DC; 2002. https://www.nap.edu/read/10206.
16. Arthur H, Wright D, Smith K. Women and heart disease: the treatment may end but the suffering continues. Can J Nurs Res. 2001;33(3):17-29. http://europepmc.org/abstract/MED/11845619/reload=0. Accessed February 7, 2014.
17. Carman KL, Dardess P, Maurer M, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Heal Aff. 2013;32(2):223-231. doi:10.1377/hlthaff.2012.1133.
18. Bonevski B, Randell M, Paul C, et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. 2014. doi:10.1186/1471-2288-14-42.
19. Smith KM, Mazor KM, Gallagher TH. We Want to Know. MedStar Health. http://www.wewant2know.org/. Published 2014. Accessed January 1, 2016.
20. Quality A for HR and. Communication and Optimal Resolution (CANDOR) Toolkit. https://www.ahrq.gov/professionals/quality-patient-safety/patient-safety-resources/resources/candor/introduction.html.
21. Lambert BL, Centomani NM, Smith KM, et al. The “Seven Pillars” Response to Patient Safety Incidents: Effects on Medical Liability Processes and Outcomes. Health Serv Res. 2016:1-25. doi:10.1111/1475-6773.12548.
22. Mazor KM, Smith KM, Fisher KA, Gallagher TH. Speak Up! Addressing the Paradox Plaguing Patient-Centered Care. Ann Intern Med. February 2016:1. doi:10.7326/M15-2416.
23. Bell SK, White AA, Yi JC, Yi-Frazier JP, Gallagher TH. Transparency When Things Go Wrong: Physician Attitudes About Reporting Medical Errors to Patients, Peers, and Institutions. J Patient Saf. February 2015. doi:10.1097/PTS.0000000000000153.
24. Bell SK, Smulowitz PB, Woodward AC, et al. Disclosure, apology, and offer programs: stakeholders’ views of barriers to and strategies for broad implementation. Milbank Q. 2012;90(4):682-705. doi:10.1111/j.1468-0009.2012.00679.x.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |