Evidence Based Telehealth Network Program Report

OMB # 0906-0043 Expiration Date: XX/XX/202X

RTRC Study Protocol
Protocol for Collecting Data on a Uniform Set of Data Elements
by EB TNP Grantees for a Data Pooling Study

The purpose of this document is to describe the research design and procedures for a study to
routinely collect data on a uniform set of data elements to be submitted by the EB TNP
grantees to RTRC for pooling, analysis, and dissemination. This document includes the
following major sections:
Contents
Overview, Purpose, Study Design, and Research Questions .......................................................... 1
In-person (Non-telehealth Comparison) Group Guidelines ........................................................... 3
Data Elements to be Included in Data Collection ........................................................................... 7
Timing of Collection of Data Elements ......................................................................................... 10
Guidance on Specific Data Elements ............................................................................................ 11
Data Collection Procedure ............................................................................................................ 12

Overview, Purpose, Study Design, and Research Questions
Overview
Rural areas in the U.S. suffer from a shortage of available healthcare services. Telehealth
services have helped to address local health care provider shortages. While initial evidence
supports the use of telehealth, additional well-designed studies are needed to elucidate the
best applications of telehealth services to increase access in rural settings. The Office for the
Advancement of Telehealth (OAT) funded the Evidence Based Telehealth Network Program (EB
TNP) grantees and charged the Rural Telehealth Research Center (RTRC) with identifying
approaches that can be used across the grantees to contribute to the evidence base for directto-consumer telehealth use in rural communities.
Background and Rationale
The shortage of available treatment services in rural areas and initial evidence supporting
telehealth approaches point to the need for additional, well-designed studies to build the
evidence base to support further adoption of and reimbursement for telehealth services to
increase access in rural settings. To address the shortage of available treatment services, OAT
has been offering grant funding to existing telehealth networks to further expand their services
to rural and underserved areas. Specific to this project, OAT released a Notice of Funding
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 1

OMB # 0906-0043 Expiration Date: XX/XX/202X

Opportunity (NOFO) (HRSA 21-082) for the EB TNP focused on direct-to-consumer telehealth. In
September 2021, OAT identified 11 grantees to receive 5 years of funding. The NOFO specified
that the grantees would submit data to RTRC on patients who receive direct-to-consumer
telehealth and a comparable group of patients who receive in-person services. The NOFO
specified that RTRC would identify data collection protocols and would serve as the data
coordinating center for this grant program.
A priority area for OAT is the clinical effectiveness of telehealth. This proposed project is
specifically designed to conduct comparative effectiveness analysis of direct-to-consumer care
using telehealth versus in-person services. Further, it will do so across 11 healthcare networks,
connecting to multiple rural settings, which will greatly increase the generalizability and
applicability of policy-relevant findings.
Direct-to-consumer telehealth was the fastest growing telehealth application prior to the
pandemic and was largely provided by a few commercial firms. Research on this delivery model
by these commercial firms has been particularly limited. To help mitigate concern about the
spread of COVID-19, a different model of direct-to-consumer telehealth became widespread
during the public health emergency. This newer model provides direct-to-consumer telehealth
through the patients’ medical home in established healthcare delivery systems. Given the
novelty of this expansion, research on its’ use and outcomes in large-scale prospective studies is
limited, and to date, few comparative effectiveness studies have been published. Thus, this
project is positioned to contribute substantially to the evidence base around telehealth by
pooling data across multiple health networks and comparing direct-to-consumer telehealth
with in-person care within those established systems.
As research on telehealth grows, efforts to examine the evidence base for telehealth
applications, such as conducting systematic reviews of this evidence, are hampered by the
disparate structures, processes, and outcome measures used in telehealth research. Policies to
reduce the barriers to telehealth and further its useful expansion will rely on sound evidence of
its effectiveness. It will be especially meaningful when multiple studies replicate positive
findings on a common set of measures. This data pooling project will help advance the field by
using a comparative effectiveness research design and pooling data across multiple grantees on
a defined set of measures to enhance external validity and generalizability.
Purpose
The ultimate objective of this project is to contribute to the evidence base for telehealth in
rural settings by pooling data collected across EB TNP grantees on the services they offer
through telehealth and in-person care related to primary care, urgent care, behavioral health,
maternal care, substance use disorder, and/or chronic care management services. Pooling data
will be possible by using a standardized set of data elements related to access/utilization,
cost/efficiency, and clinical outcomes. Enhancing the evidence base will be possible by
collecting an adequate amount of patient-specific data, statistically analyzing that data to
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 2

OMB # 0906-0043 Expiration Date: XX/XX/202X

compare the telehealth treatment group with the in-person treatment group, and publishing
peer-reviewed journal articles of the findings.
Patients in Data Collection Activities
Eleven organizations were awarded five-year grants beginning in September 2021 in response
to an OAT NOFO for the EB TNP. Each organization is responsible for recruiting and enrolling
patients for both telehealth and in-person data collection.
Study Design and Research Questions
Data are to be collected on all patients where direct-to-consumer telehealth services are used
as part of the EB TNP telehealth group and on a comparison group of patients who receive
comparable services in-person. Thus, the study design is a prospective, observational,
comparative effectiveness study of two active interventions.
The primary research questions that can be investigated focus on the comparative effectiveness
of patients who receive direct-to-consumer telehealth services versus a group of similar
patients who receive comparable in-person care. Some of these include:
1. How do patient characteristics (e.g., age, gender, race, ethnicity, health insurance type,
rurality) compare in direct-to-consumer telehealth services versus in-person care?
2. How do care processes (e.g., indicated by CPT/HCPCS codes, clinician providing care,
medication changes) compare in direct-to-consumer telehealth care versus in-person care?
3. How do clinical outcomes (e.g., measured by PHQ-9, GAD-7, SBP/DBP, HbA1c, smoking
status, weight) compare in direct-to-consumer telehealth care versus in-person care for
patients receiving relevant services?
4. What is the natural course of direct-to-consumer telehealth care compared to in-person
care (e.g., how does the frequency and timing of telehealth encounters compare to inperson encounters)?
5. How do maternal care and chronic care management differ when remote patient
monitoring is or is not incorporated into these services (e.g., how does the duration of
services and clinical outcomes compare)?
6. What is the frequency and pattern of crossovers from one treatment modality to another?
7. How do patient characteristics, care processes, outcomes, natural course of care, and
frequency and pattern vary by treatment site type?
In-person (Non-telehealth Comparison) Group Guidelines
This section is meant to clarify the purpose of the in-person (Non-telehealth comparison)
treatment group1 and offer guidelines for identifying a Non-telehealth comparison group. We
1

Throughout this document we refer to this group as the “in-person treatment group” except in this section when
quoting directly from the NOFO which used the term “non-telehealth comparison group”.
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 3

OMB # 0906-0043 Expiration Date: XX/XX/202X

begin with presenting the wording in the EB TNP NOFO (upon award, grantees are bound to the
guidelines presented in the NOFO), then provide guidelines to further clarify criteria for
identifying the Non-telehealth comparison group. The EB TNP NOFO definitions for identifying a
Telehealth group and a Non-telehealth comparison group are shown below:
Evidence Based Telehealth Network Program
Notice of Funding Opportunity: HRSA-21-082
Appendix: Common Definitions – Page 41
“DIRECT -TO -CONSUMER TELEHEALTH: Direct-to-Consumer (DTC) care is defined as patient
initiated telehealth care, typically from their home. In DTC telemedicine, no clinician is
physically present with the patient as he or she interacts with the distant telemedicine
clinician.”
“COMPARISON GROUP: Data are to be collected on all patients where telehealth services
are used as part of the award (Telehealth group) and on a 1-to-1 comparison sample of
patients who receive comparable services in-person (Non-telehealth comparison group).
Collecting data on Non-telehealth comparison groups is an important component of the
research design and will enable important research questions to be answered using a more
rigorous research approach. Ideally, award recipients will be able to identify treatment sites
that provide in-person services that are comparable to those delivered through telehealth,
and to patients who are similar to those receiving telehealth services. The type of services
should be roughly comparable in terms of delivering similar diagnostic and treatment
services to patients with similar acuity (for example, an urgent care clinic). Likewise, the
patient characteristics (e.g., rural location, age, gender, race, ethnicity, insurance coverage,
principal diagnosis, and presenting complaint) should be similar for the two groups –
Telehealth group and Non-telehealth comparison group. The most important matching
variables are comparable services and presenting complaint and/or principal diagnosis.
Following those, the patients should be matched, as a group, on the remaining patient
characteristics.”
Specific Operational Clarification for Groups
As a first point of clarification, our aim is to identify a group of patients who receive in-person
(Non-telehealth comparison group) services that are comparable to the group of patients who
receive direct-to-consumer telehealth (Telehealth group) services. We want to be able to
compare average characteristics of the two groups and find them similar.
• For example, when we average the ages of the Telehealth group patients and the Inperson comparison group patients, the group averages should be similar. It does NOT
mean if a grantee has a 60-year-old male in the Telehealth group, that they need to
treat a 60-year-old male in the In-person comparison group. Instead, grantees’ aim is to
identify two groups whose average characteristics are roughly the same.

Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 4

OMB # 0906-0043 Expiration Date: XX/XX/202X

Specific Operational Clarification for the Most Important Matching Variables
The definition above states, “The most important matching variables are comparable services
and presenting complaint and/or principal diagnosis …. The type of services should be roughly
comparable in terms of delivering similar diagnostic and treatment services to patients with
similar acuity.”
Data elements for both the Telehealth group and In-person comparison group will include the
type of service delivered as indicated by Current Procedural Terminology (CPT) or Healthcare
Common Procedure Coding System (HCPCS) codes for each encounter. Summing across
patients and encounters, CPT/HCPCS codes for both groups should match reasonably well to
meet the objective.
• For example, if a grantee is delivering behavioral health services and in the Telehealth
group 55% will receive individual therapy-60 minutes (CPT 90832) and 45% will receive
group therapy-60 minutes (CPT 90853), then the In-person comparison group should
receive these same types of services in approximately the same proportions.
• As another example, if the grantee is providing diabetes services and in the Telehealth
group 70% will receive follow-up office visit (CPT 99211) and 30% will receive diabetes
education (CPT 99202), then the In-person comparison group should receive these same
types of services in approximately the same proportions.
Another data element for both the Telehealth group and the In-person comparison group will
be diagnosis in the form of International Classification of Diseases, Tenth Revision (ICD-10)
codes. Operationally, ICD-10 codes for both groups should match reasonably well.
• For example, if a grantee is delivering behavioral health services and the Telehealth
group patients will have 60% depression (ICD-10 F32.1) and 40% anxiety (ICD-10 F41.1)
as their primary diagnosis, then the In-person comparison group should be patients with
the same diagnostic categories in approximately the same proportions.
• As another example, if the grantee is providing chronic care services and the Telehealth
group patients will have 50% hypertension (ICD-10 I10), 28% diabetes (ICD-10 E08.9),
and 22% congestive heart failure (ICD-10 I50.10) as their primary diagnosis, then the Inperson comparison group should be patients with the same diagnostic categories in
approximately the same proportions.
We realize an exact match for patients receiving Remote Patient Monitoring (RPM) services is
impossible as there is no matching code for in-person. However, we expect grantees to find an
In-person comparison group that matches with presenting complaint and/or principal
diagnosis.
Specific Operational Clarification for Other Matching Variables
The NOFO definition above states, “The most important matching variables are comparable
services and presenting complaint and/or principal diagnosis. Following those, the patients
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 5

OMB # 0906-0043 Expiration Date: XX/XX/202X

should be matched, as a group, on the remaining patient characteristics…. The patient
characteristics (e.g., age, gender, race, ethnicity, insurance coverage) should be similar for the
two groups – Telehealth group and Non-telehealth comparison group.”
Data elements for both the Telehealth group and In-person group to be considered for
comparison purposes will include the following patient characteristics: age, gender, race,
ethnicity, and insurance coverage. Summing across patients, both groups should be reasonably
similar on major categories.
• For example, if a grantee is delivering services to a pediatric population for the
Telehealth group, then the In-person comparison group should receive similar services
in a pediatric population.
• As another example, if the grantee is providing services to patients who are primarily
covered by Medicaid in the Telehealth group, then the In-person comparison group
should include Medicaid coverage in approximately the same proportion.
As stated above, specific individual-level patient demographics are less important for creating
comparable groups than CPT/HCPCS codes and ICD-10 codes, so priority should be given to
matching on CPT/HCPCS codes and ICD-10 codes. But major categories of patient demographics
should not vary widely between the Telehealth group and the In-person comparison group
(e.g., a pediatric Telehealth group should NOT be compared to an adult In-person group).
Identification of Separate Treatment Sites/Clinics/Providers
The NOFO included language that the Non-telehealth comparison group should be identified
from similar treatment sites, as follows: “Ideally, award recipients will be able to identify
treatment sites that provide in-person services that are comparable to those delivered through
telehealth, and to patients who are similar to those receiving telehealth services.” Again, the
objective is to identify an In-person comparison group that is receiving similar services. This
means that the In-person comparison group is NOT comprised of patients who are offered but
refuse Telehealth group participation. There are many reasons why patients who are offered
telehealth treatment might refuse, which will then create sources of data bias.
• Examples of separate treatment sites could be ambulatory clinics in separate towns that
offer similar services for similar types of patients, with one clinic doing so primarily
through telehealth and the other clinic doing so primarily in-person.
• Another example could be separate behavioral health clinicians in the same clinic who
offer similar therapeutic approaches for similar types of mental health conditions, but
by different modalities (telehealth vs in-person).
Geographic Requirements
The NOFO included specific language that the Telehealth group patients must be rural, as
shown below (NOFO, page 5):
“For the purposes of this award, the originating site is the location of the patient at the time
the service being furnished via a telecommunications system occurs. The originating site is
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 6

OMB # 0906-0043 Expiration Date: XX/XX/202X

required to be solely located in HRSA-defined rural areas in order to receive funds through
this award.”
We understand that some grantees are providing Telehealth services to rural areas where
comparable In-person services are not available. This presents a considerable challenge for
some grantees for identifying an In-person comparison group that is also rural. The wording in
the NOFO specifies that the Telehealth group must be rural. If identifying an In-person
comparison group that is also rural is not possible, then grantees may be able to substitute an
In-person comparison group that is not rural. If grantees plan to go that route, then they MUST
receive written approval to do so from their OAT Project Officer.
Data Elements to be Included in Data Collection
As stated above, RTRC has been charged by OAT with identifying approaches that can be used
across the EB TNP grantees to enhance the evidence base for direct-to-consumer telehealth use
in rural communities. A key component of this approach will involve pooling data across the EB
TNP grantees. To pool data, the specific data components must be identical. To that end, RTRC
identified a set of data elements appropriate for use in this study. The data elements will
include specific information that will be entered for each patient upon enrollment and for each
encounter during the study period. The goal in identifying data elements was to meet multiple
priorities: 1) data elements that are useful for answering important research questions that
address access/utilization, cost/efficiency, and clinical outcomes; 2) prioritized for usefulness in
contributing to the evidence base including that data are available from multiple grantees; 3)
alignment with commonly used clinical outcome measures to permit benchmarking against
published norms; 4) including demographics needed for describing the study sample and for
use as covariates in analyses; and 5) availability as data fields in electronic medical records used
by the grantees to reduce burden. RTRC identified candidate data elements and reviewed each
with the grantees. Based on grantee feedback about data collection feasibility, RTRC selected
the 27 data elements shown in Tables 1 - 3. Table 1 includes 13 data elements that will be
collected at the patient level once, Table 2 includes 7 data elements that will be collected at
each encounter over the 12-month follow-up period, and Table 3 includes 7 data elements
representing clinical outcomes that will be collected at least quarterly on patients receiving
relevant services. The data elements are described briefly below and more detail about each of
these data elements are included in the Data Element Dictionary.
Table 1. Data Elements to be Collected at the Patient Level Once
These patient-level data elements are collected once for each patient who receives any health
services as part of the EB TNP (for both the Telehealth and In-person groups). These data
elements will be collected at enrollment (see below for more explanation). Although these data
elements tend to be constant over time, it is possible that some may change during the 12month follow-up period. However, the data entered should reflect the response options that
are most accurate at the time of enrollment.

Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 7

OMB # 0906-0043 Expiration Date: XX/XX/202X

Data Element

1. Patient identification

2. Treatment site ID
3. EB TNP enrollment date
Treatment Site ID
4. Assigned Treatment
Group
5. Age at intake
6. Gender
7. Race
8. Ethnicity
9. Language that the patient
is best served in
10. Patient’s insurance status
11. EB TNP primary service
provided to patient ZIP
code
12. Patient’s residence ZIP
code
13. Patient travel miles to
planned place of health
services

Description of Data Element

An ID assigned to each patient that is automatically converted to a
non-linkable ID when data are submitted to protect the patients’
confidentiality
An ID assigned to each treatment site
The date when patient enrolled in EB TNP
Indicates whether the patient is in the telehealth group or the inperson treatment group
The patient's age at EB TNP enrollment date
The patient's gender
The patient’s racial group
The patient’s ethnic group
Indicates whether or not the patient is best served in English
The primary type of insurance that the patient has at time of EB TNP
study enrollment
Indicates the principal service to be provided to the patient through
the EB TNP
5-digit ZIP code for the location where the patient resides at time of
study enrollment
Miles from the patient’s residence to where the patient plans to
receive health services as part of the EB TNP

Table 2. Data Elements that Are Collected at Each Scheduled Encounter
These encounter-level data elements are to be collected for each patient who receives services
through the EB TNP at each encounter for both the Telehealth and In-person groups. All
encounters that are scheduled to be delivered as part of the EB TNP should be entered, with
the exception of remote patient monitoring data transmissions, which will only be entered once
per month (see below for more information on this exception). The duration of data collection
is 12 months. Grantees may, and indeed are likely to, provide services beyond the first 12
months after enrollment. But for the purposes of this data pooling effort, data will only be
transmitted to RTRC representing the first 12 months of treatment.
Remote patient monitoring presents a special situation for data collection. In remote patient
monitoring, multiple services and CPT codes may be billed to Centers for Medicare & Medicaid
Services (CMS) monthly. These include patient data that are being monitored (CPT 99457 and
99458) and interpreted (CPT 99091) by clinicians and billed each 30 days. In addition, there are
other activities that involve initial set-up and patient education (CPT 99453) and supply of
device (CPT 99454) that occur once or periodically. Each of these activities, matching one of
these cited CPT codes, should be treated like an encounter for data collection purposes.

Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 8

OMB # 0906-0043 Expiration Date: XX/XX/202X

Data Element

Description of Data Element

14. Scheduled encounter
date
15. Encounter modality

The date when an encounter was scheduled

16. Encounter status of the
scheduled encounter
17. Treatment service type

Whether the scheduled session was completed, or reason if it was not
completed
CPT or HCPCS code(s) for each encounter

18. Clinician type

Type of clinician seen for services during this encounter

19. Patient’s diagnosis (ICD10)

The International Classification of Diseases, Tenth Revision (ICD-10)
code(s) associated with the diagnosis established to be chiefly
responsible for the services during this encounter
Prescribed medications that are newly prescribed, modified, renewed,
or discontinued during encounter using one form of medication drug
code (either NDDF or RxNorm or NDC)

20. Prescribed medications

The modality intended for the encounter

Table 3. Data Elements that Are Collected on Patients Receiving Related Services
These clinical outcome data elements are to be administered repeatedly during treatment to
patients as appropriate to the services the patient is receiving. Generally, repeatedly means as
often as clinically relevant, but quarterly at a minimum, during the 12-month follow-up period.
Skip logic will be used to include these data elements that are relevant to the service the
patient is receiving as indicated in the patient-level section of the data collection tool. It is
assumed that these data elements will be collected at the time of an encounter. If any of these
data elements are available at a time other than an encounter, treat them like they were an
encounter and fill in all known data elements.
Remote patient monitoring presents a special situation for data collection. Remote patient
monitoring often generates daily data values that are transmitted from the patient to the
monitoring site. We do NOT want to collect a large number of such readings so we are limiting
the data elements shown below to once per month since that coincides with the billing
approach used by CMS.
Data Element

Description of Data Element

21. PHQ-9 depression
symptoms score
22. GAD-7 generalized
anxiety symptoms score
23. Smoking status

Use the Patient Health Questionnaire – 9 (PHQ-9) to assess depression
symptoms
Use the Generalized Anxiety Disorder Scale – 7 (GAD-7) to assess
anxiety symptoms
The patient’s smoking status

24. Vaping status

The patient’s vaping status

25. Blood pressure

The patient’s systolic and diastolic blood pressure

26. HbA1c

The patient’s hemoglobin A1c value

Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 9

OMB # 0906-0043 Expiration Date: XX/XX/202X

27. Height/weight/BMI

The patient’s height, weight, and/or BMI value

Timing of Collection of Data Elements
The data elements are separated into three sections depending on the timing of data collection.
The first section are data elements that are collected once, the second section are data
elements that are collected at each scheduled encounter for the first 12 months of follow-up,
and the third section are data elements that are collected at least quarterly for the first 12
months of follow-up, but only on those patients receiving pertinent services.
As shown in Table 1, a set of data elements including demographic data are to be collected
when the patient is enrolled in the EB TNP treatment activities. Enrollment should occur
directly before the initiation of treatment. Enrollment will include collection of “baseline” data
that are stable over time and not expected to change during the 12-month follow-up period.
Enrollment would likely be when the patient is informed about the study, agrees, and consents
to participate. See ‘Identification of Enrollment Date’ section below.
As shown in Table 2, a set of data elements involving treatment processes are to be gathered
for each scheduled encounter during the 12-month follow-up period. We understand that
treatment may not extend for the full 12-month follow-up period. In fact, it may only last a few
weeks or months. Nevertheless, data should be collected for the full 12-month follow-up
period. We also understand that there will be patients who are lost to follow-up (i.e., ceased to
seek services for any cause) during the 12-month follow-up period. The research design follows
“intent to treat” principles and thus it is important that available data are transmitted for every
patient who enters treatment as part of the project. If a patient is lost to follow-up because
they stop using the services or move to a different, unaffiliated provider or clinic, they are still
considered part of the research project and the data already collected should not be
eliminated. Enter all available data on every patient that is enrolled in the study even if they are
lost to follow-up and data become unavailable at some point in the 12-month follow-up period.
As shown in Table 3, a set of data elements involving clinical outcome measures will be
collected periodically. Specific clinical outcome measures will only be pertinent for patients
receiving related services. For example, the HbA1c measure assesses glycemic control and is
only pertinent to patients with diabetes or related conditions including gestational diabetes.
Pertinent measures are to be collected according to clinical guidance and reported whenever
these data are available. For example, HbA1c is recommended to be measured twice yearly in
well-controlled diabetics and four times yearly in poorly controlled individuals. But all clinical
outcome measures must be administered and reported at least quarterly during the 12-month
follow-up period. The clinical outcome assessment instruments are some of the most important
data elements, and we ask grantees to make efforts to collect these measures when pertinent
to the services they are delivering.

Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 10

OMB # 0906-0043 Expiration Date: XX/XX/202X

Guidance on Specific Data Elements
Patient Identification
This data element is necessary so that grantees and their treatment sites can identify patients
who participate in the EB TNP data collection. It will never be uploaded or saved in the RTRC
study database. It is only for grantees and their treatment site’s own reference. This serves as a
tracking mechanism for data management activities. The patient ID must remain consistent
over the duration of the project. When data are transmitted to RTRC, an anonymous (nonlinkable) case ID will be automatically assigned to the record and the patient ID used by the
grantee/treatment site will never be viewable by RTRC. This patient ID should be entered for
Data element number: Patient – 1. It will be entered for that data element but will be replaced
by a non-linkable case ID when data transfer to RTRC occurs.
Identification of Treatment Site
A treatment site must be identified for each patient. This site is where the clinician providing
the EB TNP service is affiliated. It is the clinician affiliation site and not necessarily where the
patient receives any particular treatment/service. The site will be the clinic where the patient is
affiliated for ID purposes. This serves as a tracking mechanism for data management activities.
The name of the site should remain consistent for the duration of the study. This treatment site
should be entered for Data element number: Patient – 2.
Identification of Enrollment Date
This study requires that grantees identify a beginning point for each patient. This beginning
point is identified as the enrollment date. The enrollment date should coincide with when
baseline data are collected. Baseline data must be collected shortly before EB TNP treatment
begins. This date should be entered for Data element number: Patient – 3.
Ideally for the purposes of research, all patients would be receiving treatment for the first time,
but we know that is not a realistic expectation and would drastically reduce the number of
patients in the study. Thus, we expect that some patients will have received treatment
previously and may be in ongoing treatment that is continued during the study, especially for
patients receiving chronic care management services. Nevertheless, grantees must identify a
date when patients are enrolled in this study and begin the 12-month data collection period.
Patients may only participate in data collection for a single 12-month follow-up period. Over
the course of the 5-year grant funding, patients may receive EB TNP services more than once.
For example, a patient may receive behavioral health services in Year 2 and again in Year 4. Or a
patient may receive one EB TNP service (e.g., primary care) and then later begin to receive a
different EB TNP service (e.g., maternal care) if a grantee offers multiple services. However,
patients may only contribute data once to this project and only for one primary service.
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 11

OMB # 0906-0043 Expiration Date: XX/XX/202X

Identification of Telehealth Versus In-person Treatment
For each patient, grantees identify whether the patient will be assigned to the Telehealth
treatment group or the In-person treatment group at the beginning of data collection. This
means that the patients in the Telehealth group will primarily receive services via telehealth
and the patients in the In-person group will primarily receive in-person services. This
assignment is based on intent for the primary mode of treatment. It does not mean that the
patients in each group will exclusively receive services that way. There will be crossovers where
patients assigned to one group will occasionally receive services akin to the other group. One
data element will identify which group the patient was assigned to at the time of enrollment. A
separate data element (described later) will identify the treatment modality (e.g., telehealth or
in-person) for each encounter. This assignment group should be entered for Data element
number: Patient – 4.
Identification of Primary Service Provided to Patient
The EB TNP NOFO identified six services as the focus of treatment activities, and the grantees
indicated which services they plan to deliver. At the time of enrollment, a single, primary
service must be identified for each patient. If grantees are offering multiple services, it is
possible the patients will receive more than one service through the EB TNP, however only one
service must be chosen as the principal service for data collection purposes. Some grantees
plan to offer remote patient monitoring as part of chronic care management or maternal care
services, while other grantees will offer these services without remote patient monitoring.
Thus, select the response option for chronic care management and maternal care services that
indicates whether or not remote patient monitoring will be part of this service. This data
element will be used in skip logic to open clinical outcome data elements that are pertinent to
each service and hide clinical outcome data elements that are not relevant. This primary service
should be entered for Data element number: Patient – 11.
Identification of Encounter Modality
This data element captures for each encounter, whether it was in-person or via telehealth, and
if by telehealth, which modality (i.e., video, phone, or remote patient monitoring). Patients in
the Telehealth Group may, at times, be seen in an in-person encounter. Likewise, patients in
the In-person Group may, at times, be seen in a telehealth encounter. The modality for each
encounter should be entered for Data element number: Encounter – 2.
Data Collection Procedure
Data transfer and use agreements (DTUAs) will be established between RTRC and each of the
EB TNP grantees. The University of Iowa Institutional Review Board (IRB) will approve the
protocol for transmission of data to RTRC by the grantees. Grantees are responsible for
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 12

OMB # 0906-0043 Expiration Date: XX/XX/202X

securing IRB approval from their own institution for data collection and transmission to RTRC,
which may involve establishing DTUAs with their participating sites. OAT will submit the data
elements and data dictionary to the Office of Management and Budget (OMB) for clearance.
RTRC will create a Direct-to-consumer Telehealth Evidence Collection (D-TEC) Tool which will be
used for data collection and a D-TEC User Manual that will provide guidance on its use.
Grantees will be responsible for capturing data relevant to the data elements, either through
working with coders at their participating sites or centrally reviewing and coding patient
records. Grantees are expected to input the data in REDCap® using a link provided by RTRC as
requested by RTRC on a periodic basis. Secure data transmission processes will be employed.
Data Transmission Schedule
RTRC will establish a schedule for periodic data transmission with grantees. The D-TEC Tool will
be designed so that encounter-level data for each patient will be entered on an on-going basis.
This will aid in spreading the workload for data entry across the grant period. The D-TEC Tool
will be designed so that encounter-level data can be updated at any point during the 12-month
data collection period for each patient. This means that if updated or formerly missing
information becomes available, it can be entered and old data can be written over, replacing
the previous data with corrected or updated data. However, for patient-level data that are to
be collected at the time of patient enrollment, the D-TEC Tool will be designed to “lock” the
data once it is submitted.
Data Management Procedure
Data monitoring and management activities will include: 1) overseeing the progress of the data
collection process; 2) quality control measures to identify trends and areas for improvement; 3)
identifying root cause of problem; and 4) taking steps to correct processes and reduce or
eliminate problems. The aim of the data monitoring and management function is to verify data
validity (e.g. responses are within valid value ranges), accuracy (e.g. responses are clinically
meaningful), completeness (e.g. low percent of missing data), consistency (e.g. data extraction
practices are consistent within and across organizations), and timeliness (e.g. data are inputted
in a timely fashion). Thus, after each data submission period, RTRC will process the submitted
data from each grantee and will create “issue reports” for grantees to review and address.
Analysis and Dissemination
The purpose of this data collection effort is not to evaluate any individual grantee’s efforts, but
rather to pool data across grantees to provide sufficient data for statistical analysis aimed at
addressing important research questions. Analyses will only be presented in aggregate form.
Individual grantee, treatment site, and patient data will be kept confidential and will not be
identified in manuscripts. The goal will be to contribute to the evidence base by publishing
multiple peer-reviewed journal articles.
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 13

OMB # 0906-0043 Expiration Date: XX/XX/202X

Statistical analyses will involve multiple steps. Differences in characteristics of patients who
used telehealth versus in-person treatment will be described. Regression models (logistic for
binary, linear for continuous variables, Poisson or negative binomial models for count data) will
be used to test for differences between the telehealth and the in-person treatment groups.
Models will be run for any telehealth use and also by frequency of telehealth utilization to
explore dosing effects. Multivariable regression models will be used to estimate effect sizes for
covariates of interest. Covariates of interest will include patient characteristics (e.g., age, sex,
race, ethnicity, diagnosis, insurance) and provider/service type. Data transformations will be
made when necessary to satisfy any underlying statistical assumptions. In addition to point
estimates of comparative effectiveness, associated confidence intervals will be produced.

We offer this document as the protocol governing RTRC collection of data using the D-TEC Tool.
If you have additional questions, please contact Kim at RTRC ([email protected]).

Public Burden Statement: The purpose of the Evidence-Based Telehealth Network Program is to fund
evidence-based projects that utilize telehealth technologies through telehealth networks to improve
access to, and the quality of, health care services. This program will work to help HRSA assess the
effectiveness of evidence-based practices with the use of telehealth for patients, providers, and payers.
An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this
project is 0906-0043 and it is valid until XX/XX/202X. This information collection is required to obtain or
retain a benefit [section 330I of the Public Health Service Act (42 U.S.C. 254c-14), as amended]. Public
reporting burden for this collection of information is estimated to average 36 hours per response,
including the time for reviewing instructions, searching existing data sources, and completing and
reviewing the collection of information. Send comments regarding this burden estimate or any other
aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports
Clearance Officer, 5600 Fishers Lane, Room 14N136B, Rockville, Maryland, 20857 or
[email protected].
Rural Telehealth Research Center - Study Protocol for EB TNP Data Collection – 2023-10-20

Page 14