SUPPORTING STATEMENT
Part A
Taking Efficiency Interventions in Health Services Delivery to Scale
Version: August 25, 2014
Agency for Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 5
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 6
5. Involvement of Small Entities 6
6. Consequences if Information Collected Less Frequently 7
7. Special Circumstances 7
8. Consultation outside the Agency 7
9. Payments/Gifts to Respondents 8
10. Assurance of Confidentiality 8
11. Questions of a Sensitive Nature 9
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 10
14. Estimates of Annualized Cost to the Government 10
15. Changes in Hour Burden 11
16. Time Schedule, Publication and Analysis Plans 11
17. Exemption for Display of Expiration Date 11
List of Attachments 11
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. Research that develops and presents scientific evidence regarding all aspects of health care; and
2. Synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. Initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care. The reauthorization of the Agency for Healthcare Research and Quality (AHRQ) in 1999 established the Agency as a leader in support of research designed to improve the quality of health care, reduce its costs, promote patient safety and reduce medical errors, and broaden access to effective services. More recently AHRQ has included a focus on workforce issues as part of its efforts in producing evidence that results in improving efficiencies in cost and quality of care. This is significant considering the multiple challenges that the U.S. primary care workforce is facing.
The primary care workforce is facing imminent clinician shortages and increased demand. With the implementation of the Affordable Care Act (ACA), Federally Qualified Health Centers (FQHCs) are expected to play a major role in addressing the large numbers of people who become eligible for health insurance as well as continue in their role as safety net providers. Thus, understanding new models of service delivery and improving efficiency within FQHCs is of national policy import. The proposed data collection supports
the goal of developing a more efficient FQHC service delivery model through studying outcomes associated with a “delegate model,” which is designed to improve provider and team efficiency, and the spread of this model throughout a large FQHC.
Recent models of practice transformation have documented the use of an Organized Team Model1 that distributes responsibility for patient care among an interdisciplinary team, therby allowing physicians to manage a larger panel size while practicing high quality care. The delegate model requires that all team members perform at the top of their skill level, and that tasks currently performed by clinicians are delegated to non-clinician team members in a safe and effective manner. Researchers at the University of California, San Francisco have estimated that delegation provides opportunities for panel size increases to be accommodated for through shifts in tasks to non-physician team members.2 More specifically, if portions of preventive and chronic care services are delegated to non-physicians, primary care practices can meet recommended quality and care guidelines while maintaining panel sizes with a limited primary care workforce.
AHRQ is working with John Snow, Inc. (JSI) and its partner, Penobscot Community Health Center (PCHC), to evaluate the effectiveness and spread of a delegate model in 5 of PCHC’s 15 primary care service sites. PCHC is an FQHC located in Bangor, Maine that serves northeastern Maine. Currently, PCHC’s primary care providers (PCPs, which include medical doctors, osteopaths, nurse practitioners, and physician assistants) each work with a Medical Assistant (MA). Under the delegate model, a pair of PCPs will be assigned an “administrative” MA to enhance their team. This position will enable shifting of responsibilities among the team, with the intent of relieving the PCPs of administrative tasks and incorporating new tasks that will enhance team efficiency. Examples of tasks that an administrative MA may take on include standardized prescription renewals, schedule management, in-box management, scribing, previsit planning with preappointment laboratory tests, and identification of patients for ancillary referrals (e.g. behavior health and case management).
This study has the following goals:
To evaluate the spread and effectiveness of the delegate model in five of PCHC’s primary care sites;
To evaluate the influence of the delegate model on provider satisfaction, team functioning, and patient satisfaction;
3) To assess the contextual factors influencing the above outcomes; and
4) To disseminate findings.
To achieve the goals of this project the following data collections will be implemented:
1) Team Survey (TS) – A TS will be disseminated to members of all primary care teams in all five participating sites to assess job satisfaction and team functioning in all participating sites at two points in time. (Attachment A)
2) Key Informant Interviews (KII) – KIIs will be conducted with staff in each of the participating sites during two rounds of site visits, with key informants to include the Medical Director, Practice Director, members of primary care teams implementing the delegate model or potential future implementers, and ancillary staff. A condensed version of the interview will be used for a conference call with each participating site’s Medical Director and Practice Director as an interim activity between the two site visits. (Attachment B)
Through this study, AHRQ will also collect Quality Improvement Data (QID) and hold Medical Assistant Discussions (MAD), which are described below. The QID is already being collected by PCHC, and so does not represent new burden. The MAD are unstructured discussions with a limited number of individuals. For these reasons, these two activities are not included in the burden estimate.
3) Quality Improvement Data (QID) – A subset of PCHC’s quality improvement data, to include delegate and non-delegate primary care providers’ clinical full-time equivalents (FTEs), panel size, scheduled appointments, number of 15 and 30 minute appointments, number of visits, wait time to 3rd next available appointments, patient no shows, cycle time, number of patients receiving pre-visit laboratory work, number of referrals to ancillary staff, three measures of patient satisfaction, and 12 clinical indicators (6 for family practice sites and 6 for the one pediatric practice) will be transferred to JSI on a monthly basis.
4) Medical Assistant Discussions (MAD) – During the first six months of implementation of the delegate model, phone calls with Medical Assistants of selected implementing teams will be held to gather their perspectives on the new model of care and their role and activities as they occur and potentially change over this first six month implementation period.
This study is being conducted by AHRQ through its contractor, JSI pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
The information yielded from this study is expected to inform a wide cross section of audiences and stakeholders about provider efficiency, practice redesign, team-based care, workforce strategies, and spread of an innovation. This study is not intended to make generalizations about the effectiveness of the delegate model of care, but rather to explore a promising new model and provide guidance on how similar models might be spread and evaluated.
Dissemination of the findings is one of the goals of this study. AHRQ with its contractor, JSI, will develop a Dissemination Plan in collaboration with PCHC to ensure broad distribution and awareness of the project’s findings with attention to specific audience groups, most notably primary care providers (FQHCs and non-FQHCs); managers and administrators of primary care clinics (FQHCs and non-FQHCs); and researchers and policymakers interested in workforce, costs of care, efficiency, productivity, and spread. This last audience includes Federal agencies such as the Health Resources and Services Administration (HRSA) and the Center for Medicare & Medicaid Services’ (CMS)’ Center for Medicare and Medicaid Innovation that develop programs and policies for safety net providers. Dissemination avenues will be geared to each audience and may include peer-reviewed journal articles, professional conferences, and more broadly, online in uploaded issue briefs, webinars, videos, and social media.
The Team Survey will be completed electronically via a secure on-line survey platform. Electronic surveys are convenient and alleviate administrative burden on the part of the practice sites.
Quality Improvement Data (QID) is already being captured by PCHC through its electronic medical records and practice management system. QID for the study will be transferred to JSI on a monthly basis through a secure File Transfer Protocol (FTP) site.
A comprehensive literature search in the area of team-based models of care and use of task delegation suggests promising implications for primary care transformation. There are some studies assessing provider satisfaction related to task delegation. While there are several studies that use modeling and estimation to determine the effect of task delegation on patient access and panel size, there are very few studies that use actual clinic data to assess these outcomes. Further literature review on the spread of innovations has been descriptive and qualitative in nature and without a systematic approach to assessing context related to spread outcomes. The proposed study will provide new evidence in scaling of interventions that improve efficiencies and access to care and new information on efficiency, access, team functioning, and team satisfaction related to task delegation. Additionally, through its focus on context, this study will inform the transfer of successful innovations into other settings. AHRQ staff inquired across the agency, as well as with other HHS agencies such as HRSA, the Office of the Assistant Secretary for Planning and Evaluation (ASPE), and CMS, to ensure there was no duplication of efforts across the department.
None of the participating health centers is considered a small entity.
Primary data collection via team surveys and key informant interviews will be conducted to provide quantitative and qualitative information about effectiveness, team satisfaction and team functioning, adoption of the innovation, and how context influences all. It will also be used to assist in the interpretation of quality improvement data.
The frequency of primary data collection (Team Survey and Key Informant Interviews,) is designed to limit burden on practice staff. The TS will be issued pre-delegate model and post-delegate model implementation to assess whether there has been an improvement. The KIIs will be conducted at three points: early in the spread of the delegate model, in the middle of the spread of the delegate model, and during the end of the spread of the delegate model to assess key informants’ perspectives over these time periods and to identify contextual factors that may be influencing the implementation of the model or the spread strategy. To collect the TS less frequently would mean an inability to assess change in satisfaction and team functioning. To collect KIIs less frequently would mean the loss of key stakeholders’ perspectives on the spread and implementation details of the delegate model, which would significantly limit the ability to explain the process of spread and the details of the delegate model.
There is one special circumstance relevant to this data collection: results may not be generalizable to the universe of the study.
The immediate universe for the study is all primary care providers at the five participating sites. However, providers are self-selecting to participate in the delegate model or may be encouraged by site and/or PCHC leadership to participate. In that participating providers are not randomly selected and may, in fact, differ from the universe on important characteristics, may limit the generalizability of findings. The broader universe of the study is all FQHC primary care providers. Given that the study is local to one FQHC in the Bangor, Maine area, study results may not be generalizable to FQHCs in other areas. The patient and provider population and the environment in which they interact may differ from other FQHCs in important ways that would make the intervention being studied more or less effective, or make it easier or harder to spread among providers. However, this exploratory study intends to capture many of those factors, so that other providers may determine whether this intervention would be appropriate for them to implement.
With this exception noted, this request is otherwise consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2).
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on page 19333, April 8, 2014 for 60 days (see Attachment C). No comments were received.
JSI and PCHC are partners in this effort and have worked jointly on the design of the study. Dr. Noah Nesin, Chief Quality Officer of PCHC is a study Co-Investigator and oversees quality improvement efforts at PCHC. The development of all data collection tools and protocols related to frequency and means of data collection efforts has been conducted in consultation with Dr. Nesin and key staff of PCHC. JSI, PCHC, and AHRQ staff presented the proposed study and data collection to an HHS audience representing ASPE, HRSA, The Office of the National Coordinator for Health Information Technology (ONC), CMS, and AHRQ in order to obtain their feedback. Feedback was very positive overall, and specific suggestions were incorporated into the study design.
No payments or gifts will be provided to respondents other than remuneration of contractors.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
The purpose of all data collections will be explained to respondents at the beginning of data collection and will note that participation is voluntary. Permission to conduct interviews will be secured before proceeding.
Respondents to the Team Survey and KIIs will be identifiable through name. No individual level patient data will be collected. All data will be governed by a jointly-developed JSI/PCHC Data Use and Sharing agreement that specifies which information will be shared with whom and the methods for secure data transfer. This agreement confirms that that the data recipient, JSI, will not disclose or use the data for any purpose other than the agreed upon uses for the project.
Data reporting will not include identifying information about providers or others respondents, although given the relatively small numbers of potential providers at each site, respondents may potentially be identifiable. For all data collection tools and interviews, the following language will be included:
The confidentiality of your responses are protected by Sections 944(c) and 308(d) of the Public Health Service Act [42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)]. Information that could identify you will not be disclosed unless you have consented to that disclosure.
11. Questions of a Sensitive Nature
The Team Survey includes questions of a sensitive nature in that it will gather appraisals of other individuals with whom respondents have close professional, and sometimes supervisory, relationships. The Team Survey is an essential component of the study in that improved team functioning and provider satisfaction are two hypothesized outcomes of adoption of the delegate model and essential for further uptake and spread of the model.
The data will be collected through an online survey with responses received directly by JSI. Analysis of the data will compare team satisfaction and functioning pre- and post-delegate model adoption and adopting teams to non-adopting teams. Reporting out will be anonymous but given the relatively small numbers of teams, there is the potential for identification among those with knowledge of the sites.
The purpose and use of the data, the voluntary nature of participation in the survey, and the potential for identification of individuals will be disclosed on the survey tool.
Exhibit 1 shows the estimated annualized burden for the respondents' time to participate in this research. Information will be collected through an internet-based team survey and in-person and telephone interviews.
Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this research. The total annual cost burden is estimated to be $23,271.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Team Survey
Total |
21 34
55 |
2 2
2 |
15/60 15/60
15/60 |
11 17
28 |
Key Informant Interviews (Site visits)
Total |
2 2 5 10
19 |
2 2 2 2
2 |
30/60 30/60 30/60 30/60
30/60 |
2 2 5 10
19 |
Key Informant Interviews (Phone calls)
Total |
3 3
6 |
1 1
1 |
1 1
1 |
3 3
6 |
Total |
80 |
na |
na |
53 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Team Survey
Total |
21 34
55 |
11 17
|
$59.83a $12.51b
na |
$13,821 $ 7,231
$21,052 |
Key Informant Interviews (Site Visit)
Total |
2 2 5 10
19 |
2 2 2 2
8 |
$88.43c $48.72d $59.83a $12.51b
na |
$ 354 $ 195 $ 598 $ 250
$1,397 |
Key Informant Interviews (Phone calls)
Total |
3 3
6 |
2 2
4 |
$88.43c $48.72d
na |
$530 $ 292
$ 822 |
Total |
80 |
na |
na |
$23,271 |
* National Compensation Survey: Occupational wages in the United States May 2013, “U.S. Department of Labor, Bureau of Labor Statistics.”
a Based on the average mean wages for three categories of primary care provider ($88.43 – Family/General Practitioner; $45.36 PAs; and $45.71 – NPs).
b Based on the mean wage of Medical Assistants.
c Based on the mean wages for Family/General Practitioner.
d Based on the mean wages for Medical and Health Services Managers.
There are no direct costs to respondents other than their time to participate in the study.
The estimated annual and total cost listed in Exhibit 3 represents the estimated full cost to the Federal Government of implementing this project. These costs primarily reflect the efforts of the contractors and subcontractors. However, the Project Management field also includes the oversight of the Contracting Officer’s Representative (COR) at AHRQ. The COR is a Social Science Analyst (GS-12, Step 1) and devotes approximately 10% FTE to this project. At an annual salary of $75,621, this adds $7,562 to the annualized estimate and $22,686 to the total estimate. The total cost is estimated to be $1,331,594. The tasks occur throughout the three-year project term (36 months); thus, the estimated annualized cost to the Federal Government is $443,864.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$111,023 |
$37,008 |
Data Collection Activities |
$271,727 |
$90,576 |
Data Processing and Analysis |
$ 77,636 |
$25,879 |
Publication of Results |
$ 90,354 |
$30,118 |
Project Management |
$185,809 |
$61,936 |
Overhead |
$595,045 |
$198,348 |
Total |
$1,331,594 |
$443,865 |
eAnnual salary based on 2014 OPM Pay Schedule for Washington/DC area: http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2014/DCB.pdf
The timeline for data collection, analysis, and publication is shown in Exhibit 4 below.
Exhibit 4: Timeline for data collection, analysis and publication
Task/Activity |
Timeline |
Submit 60 day Federal Register notice for public comment |
February, 2014 |
Submit OMB clearance package |
April 2014 |
Team Survey |
September 2014, June 2016 |
Key Informant Interviews (Site visits) |
September 2014, June 2016 |
Key Informant Interviews (Phone calls) |
September 2015 |
Quality Improvement Data |
Monthly starting September 2014-July 2015 |
Analysis (qualitative/quantitative) |
September 2015 – August 2016 |
Report annual findings and recommendations |
September 2014, 2015, 2016 |
Dissemination products (issue briefs, white paper, webinars, manuscript) |
December 2015-September 2016 |
AHRQ does not seek this exemption.
List of Attachments:
Attachment A -- Team Survey
Attachment B -- Key Informant Interviews
Attachment C – Federal Register Notice
1 Scherger, JE. It’s time to optimize primary care for a healthier population. Med Econ. 2010; 87(23):86-88.
2 Altschuler, J., Margolius, D., Bodenheimer, T & Grumbach, K. Estimating a Reasonable Patient Panel Size for Primary Care Physicians With Team-Based Task Delegation. Annals Fam Med. 2012 ;10(5):396-400.
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | erwin.brown |
| File Modified | 2014-08-25 |
| File Created | 2014-08-25 |