Supporting Statement

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see https://www.ahrq.gov/policymakers/hrqa99a.html), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health system practices, including the prevention of diseases and other health conditions. AHRQ shall promote healthcare quality improvement by conducting and supporting

1. Research that develops and presents scientific evidence regarding all aspects of healthcare;

2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. Initiatives to advance private and public efforts to improve healthcare quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of healthcare in inner-city areas, and in rural areas (including frontier areas); and (B) healthcare for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special healthcare needs, including individuals with disabilities and individuals who need chronic care or end-of-life healthcare.

Coordination of care for patients with comorbidities is a known problem. This is particularly important for those with chronic kidney disease (CKD) and other conditions as they require coordination across settings and providers. Care plans can help promote coordination. A care plan is a dynamic and personalized way to document an individual’s needed healthcare activity, goals, and objectives.^{1, 2} An electronic care (e‑care) plan application (app) allows patient-centered data to be available across care and research settings. AHRQ and the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) have partnered to develop and support an e‑care plan app for people with CKD and multiple chronic conditions (MCC). This effort is significant because CKD is common, costly, and consequential;³ people with CKD often also have MCC.^4-8 Therefore, care plans are crucial tools to address and coordinate health needs of people with MCC.⁹

Cognitive Medical Systems is developing two key deliverables in a separate effort: (1) a Substitutable Medical Applications, Reusable Technologies (SMART®) on Fast Healthcare Interoperability Resources (FHIR®) e‑care plan app and (2) an FHIR Implementation Guide (IG) that documents the agreed upon technical mechanisms for data exchange between the app and other health information technology (IT) systems. To understand how the app and IG work in practice, it is necessary to understand the extent to which the app and IG enable collecting and sharing standardized data across diverse health settings and health IT systems for people with CKD and MCC. RTI (along with our subcontractors) will design and implement a pilot test to assess the e‑care plan app’s usefulness for patients with CKD and at least one other chronic illness by determining whether the app facilitates standardized data collection and data sharing across clinical and community settings and systems. This pilot test’s outcomes will guide further enhancement of the e‑care plan app and its IG to enable physicians to care for patients with MCC better and to enable patient-centered outcomes research.

We anticipate that a total of six sites will participate in the pilot of the e‑care plan app. The pilot sites are representative of patient care settings across this continuum, including in ambulatory care, specialty care, hospitals, long-term and post-acute care, dialysis, and community partners. RTI International has intentionally identified a broad range of sites connected to our partner, Oregon Health and Science University (OHSU), that vary by population served, technical capabilities, and health IT systems used. These sites have extensive experience with care plans and have participated in observational and other research studies. We anticipate the following site mix, composed of 50% primary care organizations:

3 primary care (2 OHSU, 1 Northwest Primary Care)
1 nephrology (1 OHSU)
1 dialysis (Fresenius or DaVita)
1 long-term and post-acute care (Holladay Park or Mirabella)

The following data collection will support the evaluation of the e‑care plan app and IG, we will conduct the following data collection efforts:

1) Healthcare Staff Interviews: After rollout of the e‑care plan app, semi-structured interviews will be conducted with up to three healthcare staff members at each of the six sites. There will be one primary care provider, one specialty care provider, and one other clinical staff member interviewed at each site. During the interviews, healthcare staff will be asked to explore the following concepts:

Integration of e‑care plan app into workflow,
Usefulness of app to meet care coordination needs,
How they used the e‑care plan app,
Degree to which providers trusted the data in the e‑care plan app,
Overall opinions of the e‑care plan app,
Appropriateness of the data included in the e‑care plan app,
Accuracy of the data included in the e‑care plan app, and
Feasibility and ease of use by different members of the healthcare team.

2) Health IT Professional Interviews: To understand IT staff’s experience with the implementation, semi-structured interviews will be conducted with seven IT implementation staff (one from OHSU and one from each study site). IT implementation staff will be asked about

Perceptions of the utility of the IG when implementing the app,
Integration of the e‑care plan app into the health IT system,
Ability to pull data from the electronic health record into the app using common terminologies, and
Ability to transmit clinical data using the HL7 FHIR standard.

3) Patient Interviews: Up to nine patients will be randomly recruited across the six clinical sites to be interviewed. Concepts that will be discussed with patients include

Usefulness of the e‑care plan app,
Appropriateness of the data in the e‑care plan app,
Overall opinions of the e‑care plan app, and
How they used the e‑care plan app.

A.2 Purpose and Use of Information

The purpose of this data collection is to inform changes to the e‑care plan app and associated IG. We will seek information about functionality and usability of the IG and the e‑care plan app. RTI will review the data collected during the pilot to form a comprehensive report. The report will highlight usability enhancements, lessons learned, and recommendations for future app and IG development. This report will be instrumental to inform AHRQ’s objective of standardizing e‑care plans and making e‑care plan apps more usable and useful for all stakeholders.

A.3 Use of Improved IT

One of the goals of this effort is to identify and evaluate advanced techniques that will help AHRQ obtain the necessary amount of information with a minimum amount of burden through the use of e‑care plans to substitute for paper and automated processes whenever feasible.

A.4 Efforts to Identify Duplication

No similar data have been gathered by the research team or are available from other sources known to the research team.

A.5 Involvement of Small Entities

It is unlikely that any sites participating in this pilot test will be classified as small businesses.

A.6 Consequences if Information Collected Less Frequently

This is a one‑time information collection, and if these data are not collected then we will not be able to obtain user perspectives on the app and IG.

A.7 Special Circumstances

No special circumstances apply.

A.8 Federal Register Notice and Outside Consultations

A.8.a Federal Register Notice

A Federal Register notice is not required for this generic clearance.

A.8.b Outside Consultations

RTI’s partners are OHSU and Clinical Cloud Solutions, LLC. RTI will manage the project collaborations, including coordination with partner sites with the support and expertise of Dr. Dorr and others at OHSU. Dr. Dorr has previously collaborated with stakeholders (e.g., patients, providers, professional societies, IT staff, and electronic health record and content vendors) from various primary and specialty care practices and other settings, and will provide our team with a readily available pool of partner sites to pilot test the e‑care plan app and help identify participants.

A.9 Payments/Gifts to Respondents

Interview Participation: Healthcare staff, patients, and health IT professionals will not receive any payment or remuneration.

Participating Sites: Participating sites will not receive any payment or remuneration. Participating sites will have valuable care plan data for key patient populations integrated into their electronic health record systems for providers to use during clinic visits with patients.

A.10 Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). This law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.

Information that can directly identify the respondent like name or social security number will not be collected.

A.11 Questions of a Sensitive Nature

Interview tools have been designed without the need to ask questions of a sensitive nature. If any question is expressed to be of a sensitive nature during data collection, these questions will be modified to address the concern.

A.12 Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated burden hours throughout the pilot evaluation for the respondents’ time to participate in research activities that may be conducted under this clearance.

Exhibit 1. Estimated annualized burden hours

Form Name	Number of Respondents	Number of Responses per Respondent	Hours per Response	Total Burden Hours
Attachment A - Healthcare Staff Interview Guide (for providers & non-providers)	18 (100%)	1	1.0	18
Attachment B - Health Information Technology Professional Interview Guide	7 (100%)	1	1.0	7
Attachment C - Patient Interview Guide	9 (100%)	1	1.0	9
Total				34

Exhibit 2 shows the estimated cost burden throughout the pilot evaluation based on the respondents’ time to participate in these research activities. The total cost burden is estimated to be $3,331.01. Data from the 2018 National Compensation Survey was used for the “All Occupations” category since this was no longer available in the 2019 data.

Exhibit 2. Estimated Annualized Cost Burden

Form Name	Number of Respondents	Total Burden Hours	Average Hourly Wage Rate*	Total Cost Burden
Attachment A - Healthcare Staff Interview Guide (for providers & non-providers)	12	12	$97.81^a	$1,173.72
Attachment A - Healthcare Staff Interview Guide (for providers & non-providers)	6	6	$40.21^b	$241.26
Attachment B - Health Information Technology Professional Interview Guide	7	7	$44.43^c	$311.01
Attachment C - Patient Interview Guide	9	9	$24.98^d	$224.82
Total				$1,950.81

* Occupational Employment and Wages, May 2019, “U.S. Department of Labor, Bureau of Labor Statistics.”

^a Based on the mean wages for 29-1228 Physicians, All Other; and Ophthalmologists, Except Pediatric (29-1288)

^b Based on the mean wages for all Healthcare Practitioners and Technical Occupations (29-0000)

^c Based on the mean wages for all Computer Occupations (15-1299)

* National Compensation Survey: Occupational wages in the United States May 2018, “U.S. Department of Labor, Bureau of Labor Statistics.”

^d Based on the mean wages for All Occupations (00-0000)

A.13 Estimates of Annualized Respondent Capital and Maintenance Costs

There are no direct costs to respondents other than their time to participate in the implementation and evaluation.

A.14 Estimates of Total and Annualized Cost to the Government

Exhibit 3a. Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Soft Rollout & Go-Live Site Meetings	$10,132	$10,132
Site Interviews & Analysis	$16,606	$16,606
Report Writing	$80,874	$80,874
Total	$107,612	$107,612

Exhibit 3b. Federal Government Personnel Cost

Activity	Federal Personnel	Hourly Rate	Estimated Hours	Cost
Data Collection Oversight	Health Scientist Administrator GS 14	$65.88	50	$3,294
Review of Results	Health Scientist Administrator GS 14	$65.88	150	$9,882

Annual salaries are based on 2020 OPM Pay Schedule for Washington/DC area: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2020/DCB_h.pdf.

A.15 Changes in Hour Burden

This is a new collection of information.

A.16 Time Schedule, Publication, and Analysis Plans

The timeline of scheduled tasks is provided below. AHRQ will disseminate findings only when appropriate. This may include presentations at professional meetings, publications on the AHRQ website, or in professional journals that focus on app usability and pilot testing.

1. Execution of the Pilot Test

a. Conduct Soft Rollout of the E‑Care Plan App (November 2020)

b. Conduct Full Rollout of the E‑Care Plan App (December 2020–July 2021)

c. Evaluation of the E‑Care Plan App including data collection and analysis (March 2021–July 2021)

Interviews will occur first then will be analyzed through thematic analysis to identify, analyze, and interpret patterns within descriptive data. We will identify themes across roles and sites. In addition, we will review site characteristics to identify if there are relationships between site characteristics and themes from our qualitative data.

2. Reporting Results

a. Submit Final Report Outline (June 4, 2021)

b. Submit Draft Report (August 6, 2021)

c. Submit Final Report (September 10, 2021)

A.17 Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

Attachment A - Healthcare Staff Interview Guide (for providers & non-providers)

Attachment B - Health Information Technology Professional Interview Guide

Attachment C - Patient Interview Guide

References

1. International Organization for Standardization (ISO). Health informatics—System of concepts to support continuity of care. ISO 13940:2015. Geneva, Switzerland: International Organization for Standardization; 2015. https://www.iso.org/standard/58102.html.

2. Dykes PC, Samal L, Donahue M, et al. A patient-centered longitudinal care plan: vision versus reality. J Am Med Inform Assoc. 2014 Nov-Dec;21(6):1082-90. doi: 10.1136/amiajnl-2013-002454. PMID: 24996874.

3. Centers for Disease Control and Prevention (CDC). Chronic Kidney Disease: common—serious—costly [last updated 12/12/19]. Washington, DC: U.S. Department of Health & Human Services; n.d. https://www.cdc.gov/kidneydisease/prevention-risk/CKD-common-serious-costly.html. Accessed on December 27, 2019.

4. Fraser SD, Roderick PJ, May CR, et al. The burden of comorbidity in people with chronic kidney disease stage 3: a cohort study. BMC Nephrol. 2015 Dec 1;16:193. doi: 10.1186/s12882-015-0189-z. PMID: 26620131.

5. Lee WC, Lee YT, Li LC, et al. The number of comorbidities predicts renal outcomes in patients with stage 3(-)5 chronic kidney disease. J Clin Med. 2018 Nov 28;7(12). doi: 10.3390/jcm7120493. PMID: 30486496.

6. Fraser SD, Blakeman T. Chronic kidney disease: identification and management in primary care. Pragmat Obs Res. 2016;7:21-32. doi: 10.2147/POR.S97310. PMID: 27822135.

7. Centers for Disease Control and Prevention (CDC). Chronic Kidney Disease (CKD) surveillance system [last updated 11/5/19]. Washington, DC: Department of Health & Human Services; n.d. https://nccd.cdc.gov/ckd/default.aspx. Accessed on December 27, 2019.

8. Zelnick LR, Weiss NS, Kestenbaum BR, et al. Diabetes and CKD in the United States population, 2009-2014. Clin J Am Soc Nephrol. 2017 Dec 7;12(12):1984-90. doi: 10.2215/CJN.03700417. PMID: 29054846.

9. Lion KC, Mangione-Smith R, Britto MT. Individualized plans of care to improve outcomes among children and adults with chronic illness: a systematic review. Care Manag J. 2014;15(1):11-25. doi: 10.1891/1521-0987.15.1.11. PMID: 24761537.

10. Atlassian. Jira Software: bug tracking done right. Sydney, NSW (HQ): Atlassian; n.d. https://www.atlassian.com/software/jira/bug-tracking.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Ortiz, Alexa
File Modified	0000-00-00
File Created	2021-01-14