Evaluating a Structured Reporting Template to Increase Transparency and Reduce Review Time for Healthcare Database Studies

Annals of Internal Medicine

RESEARCH AND REPORTING METHODS

Graphical Depiction of Longitudinal Study Designs in Health
Care Databases
Sebastian Schneeweiss, MD, ScD; Jeremy A. Rassen, ScD; Jeffrey S. Brown, PhD; Kenneth J. Rothman, DrPH;
Laura Happe, PharmD, MPH; Peter Arlett, MD; Gerald Dal Pan, MD, MHS; Wim Goettsch, PhD; William Murk, PhD; and
Shirley V. Wang, PhD

Pharmacoepidemiologic and pharmacoeconomic analysis of
health care databases has become a vital source of evidence to
support health care decision making and efficient management
of health care organizations. However, decision makers often
consider studies done in nonrandomized health care databases
more difficult to review than randomized trials because many
design choices need to be considered. This is perceived as an
important barrier to decision making about the effectiveness and
safety of medical products. Design flaws in longitudinal database
studies are avoidable but can be unintentionally obscured in the
convoluted prose of methods sections, which often lack specificity. We propose a simple framework of graphical representation

that visualizes study design implementations in a comprehensive, unambiguous, and intuitive way; contains a level of detail
that enables reproduction of key study design variables; and
uses standardized structure and terminology to simplify review
and communication to a broad audience of decision makers.
Visualization of design details will make database studies more
reproducible, quicker to review, and easier to communicate to a
broad audience of decision makers.

T

make it easier for reviewers and decision makers to distinguish the useful from the flawed or irrelevant (13). Graphical study design representations were recommended by
the most recent guidance for reporting on database studies from the REporting of studies Conducted using Observational Routinely collected health Data statement for
pharmacoepidemiology (RECORD-PE) (14), as well as recently published consensus papers by 2 leading professional societies (15, 16).
We propose a simple framework of graphical representations that will clarify critical design choices in
database analyses of the effectiveness and safety of
medical products. A recent consensus statement laid
out a set of parameters that define decisions in database study implementation, which, if reported, would
increase reproducibility of studies (16). Building on
these parameters, we sought to develop a visualization
framework that describes study design implementation
in a comprehensive, unambiguous, and intuitive way;
contains a level of detail that enables reproduction of key
study design variables; and uses standardized structure
and terminology to simplify review and communication to
a broad audience of decision makers. Our multistakeholder group comprised international leaders with more
than 75 years of combined experience in academia, regulatory decisions, health technology assessment, journal
leadership, payer decision making, and analyses of distributed health care data networks. The example figures
and templates are covered by a Creative Commons license.
The PowerPoint figures are free to download and adapt,
with appropriate attribution, from www.repeatinitiative
.org/projects.html.

he pharmacoepidemiologic and pharmacoeconomic
analysis of databases containing administrative claims
and electronic health records has become a routine
source of evidence to support regulatory (1) and reimbursement (2) decisions, as well as efficient management
of health care organizations. When decision makers understand the study design and analytic choices of a nonrandomized database study and recognize those choices
as valid, they have confidence in their decisions based on
the study's evidence about the comparative effectiveness
and safety of medical products (3, 4). Generally, they consider nonexperimental database studies more difficult to
review than randomized trials and see the increased complexity, greater variability in design and analysis options,
and lack of consistency in presentation of design choices
as key barriers to using database evidence for high-stakes
decisions.
Unfortunately, some poorly designed studies have
led to negative generalizations about the entire field of
health care database research rather than a refined
view that distinguishes robust evidence from less reliable evidence (5). Confounding from treatment selection based on outcome risk is well known to cause bias
(6). Time-related study design flaws can also introduce
large biases, including immortal time bias (7), reverse
causation (8, 9), adjustment for causal intermediates,
unobservable time bias (10), and depletion of susceptibles (11, 12). The methods sections of study reports
should describe the study design and analytic choices
clearly enough to allow the reader to judge the validity
of findings. However, convoluted prose often makes it
difficult for most readers to understand what methods
were implemented or identify avoidable design flaws.
Design diagrams provide key information that needs
to be considered when evidence is interpreted from
pharmacoepidemiologic and pharmacoeconomic studies
done with health care databases. Improving transparency
in how these studies are designed and implemented will
Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Ann Intern Med. doi:10.7326/M18-3079
For author affiliations, see end of text.
This article was published at Annals.org on 12 March 2019.

Annals.org

TERMINOLOGY
The terminology we suggest for temporal anchors
is frequently used in descriptions of database studies and
in textbooks (17), as well as in the recently published conAnnals of Internal Medicine © 2019 American College of Physicians 1

RESEARCH AND REPORTING METHODS

Graphical Depiction of Study Designs

Table. Temporal Anchors
Term
Base anchor (defined in calendar time; describes source data)
Data extraction date
Source data range
Study period

First-order anchors (defined in patient event time; specifies
study entry or index date)
Cohort entry date
Outcome event date*
Second-order anchors (defined in patient event time, relative
to first-order anchor)
Washout window for exposure

Washout window for outcome
Exclusion assessment window
Covariate assessment window

Exposure assessment window

Follow-up window

Definition
The date when the data were extracted from the dynamic transactional database.
The calendar time range covered by a data source that is available to create the
study population.
The calendar time boundaries for data used to create the analyzed study data
set, including exposures, inclusion and exclusion criteria, covariates, outcome,
and follow-up.

The date when patients enter the study population.
The date of an outcome event occurrence.

An interval used to define incident exposure. If there is no record of the exposure
(and/or comparator) of interest within this interval, the next exposure is
considered a “new” initiation; otherwise, it is considered prevalent exposure.
An interval used to define incident outcomes. If there is no record of outcomes
within this interval, the next outcome is considered incident.
An interval during which patient exclusion criteria are assessed.
An interval during which patient covariates are assessed. The covariate assessment
window should precede the exposure assessment window in order to avoid
adjusting for causal intermediates. It is sometimes called baseline period.
The window during which exposure status is assessed. The exposure status is
defined at the end of the exposure assessment window.† The exposure
assessment window should precede the follow-up window to avoid reverse
causation.
The interval during which occurrence of the outcome of interest in the study
population will be included in the analysis. The follow-up window may involve
stockpiling algorithms, grace periods, exposure extension, and/or censoring
related to exposure discontinuation.

* Can be a first-order anchor in some study designs (e.g., case-crossover and case– control).
† This is relevant in sampling designs when the occurrence of the exposure is not a first-order anchor defining cohort entry.

sensus statement (15, 16). We define 3 categories of temporal anchors (Table): base anchors, first-order anchors,
and second-order anchors. Base anchors are defined in
calendar time and describe the source database—that is,
the longitudinal streams of administrative or clinical health
care data from which an analyzable study data set is derived. First-order anchors are defined in patient event
time rather than calendar time and specify the study entry
or index date. Second-order anchors are also measured
in patient event time and are defined relative to the firstorder anchor. We provide more detail on each temporal
anchor in the following section.

STUDY DESIGN IMPLEMENTATION IN HEALTH
CARE DATABASES
The Nature of Health Care Databases Relevant to
Effectiveness Research
Health care databases are derived from transactional databases that record clinical and administrative
information for delivering and administering health
care. As encounters occur and services are provided,
records are generated and tallied. Each addition to the
database comes with a service date stamp and is attributed to the patient via a unique patient identification
number, thus generating longitudinal patient records
of increasing duration. There is substantial literature
describing the details of data integration, cleaning, and
normalization (18 –20).
2 Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

For each patient, all encounters with the health
care system that are reimbursable by health insurance
(or are captured by the provider's electronic health record system) can be sorted by the service date in calendar time (Figure 1). Each encounter is associated
with information on medical services, diagnoses, procedures, and similar events, plus information on payments (in claims data) or charges (in electronic health
record data). The rules and algorithms that stem from a
specific study implementation will then be applied to
each patient's longitudinal data stream. The study implementation is usually oriented around an eventbased timeline anchored to a key event, in contrast to
the calendar time arrangement of the raw data (Figure
1) (21).
Dates and Time Windows
Certain principles guide the design and implementation of studies in health care data streams. One of the
most important is temporality. Unlike in primary data collection, many measurements in health care databases—for
example, patients' baseline characteristics—are measured
by reviewing information recorded during multiple health
care encounters over time. In primary data collection, a
study participant's health state is usually established when
the patient is thoroughly interviewed or examined at a
study visit. Health care databases have no defined interview date with the investigator team; rather, studies rely
on the occurrence of routine visits and other health care
Annals.org

RESEARCH AND REPORTING METHODS

Graphical Depiction of Study Designs

Figure 1. From transactional data to study implementation.
Hospital Stay
1 May 2016

Dx

V Rx

July 1

Lab

September 1

Rx

Rx

V Dx

November 1

January 1

Study rules are applied to each participant and
arranged in event time anchored at cohort entry.

Washout Window E,O
Covariate Assessment Window
Follow-up Window

Time
Cohort Entry Date

Individual patient data are documented as encounters from various sources and are arranged in calendar time. This work is licensed under CC BY,
and the original versions can be found at www.repeatinitiative.org/projects.html. Dx = diagnosis; E = exposure; Lab = laboratory test; O = outcome;
Rx = drug dispensing; V = visit.

encounters to collect information that was recorded during provision of care. Thus, information that may be conceptualized as characterizing a point in time, such as
baseline patient characteristics before the start of exposure, is actually recorded during a time window through a
series of encounters.
Anchors in Calendar Time
For a database study to be reproducible, temporal
anchors must be defined to specify the underlying
longitudinal data used to create a study population
(Table). The data extraction date is particularly important to record when working with recent data that are
still fluid. The dynamic data flow in a health care database is stabilized by extracting and physically or virtually setting aside requested data for research purposes.
However, some administrative records may be corrected or amended retroactively for up to 6 months or
longer (22). If the underlying database has data that are
dynamically updated over time, a study using the most
recently available data extracted today will probably
not be exactly replicated using data covering the same
period but extracted a year later.
The source data range reflects the calendar date
boundaries beyond which encounter information is not
captured for patients. Investigators must be clear about
the lag between the most recent update to the data
source and the calendar time boundaries for data included in their study (study period). For example, investigators may access a data source where the tables containing up-to-date information on patient health care
contacts are extracted on 1 January 2019 (data extraction date). The source data range included in those tables covers 1 January 2003 to 31 December 2018. The
investigators, however, choose a study period that focuses on time after market entry of a drug and does not
use the most recent 6 months, a period during which
the data may be more fluid. The data extraction date
and source data range do not need to be included in
visualization of study design, but reporting them and
Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

archiving extracted longitudinal data will make study
implementation reproducible (16).
Anchors in Patient Event Time
When an effectiveness or safety study is implemented in a longitudinal database, the time scale shifts
from calendar time to patient event time. Specific algorithms define events in the patient timeline. As in randomized controlled trials, where the randomization
date is the anchor date, the cohort entry date (CED,
also called the index date) is the primary anchor in a
nonrandomized database study (Table).
The CED is the date when patients enter the analytic
study population. For some study designs, study entry
can be defined by an event date (as described under
Nested Case–Control Study and in Self-Controlled Study
Design Visualization in the Appendix, available at Annals
.org). The CED is considered a first-order anchor because
most other anchors and parameters used in study implementation will be defined relative to it. The CED is defined by an inclusion rule, along with multiple exclusion
criteria that are sequentially applied. Clarity in the definitions and sequence of these criteria is essential. For example, whether exclusions are applied before or after selection of the CED should be clear. If the wrong patients are
excluded or if the study entry date is shifted, results may
not be reproducible (16).
Secondary temporal anchors are defined relative to
the first-order anchor, the CED. As in temporal ordering in a randomized trial (23), we wish to assess all patient characteristics before the start of exposure to
avoid adjusting for causal intermediates. The exclusion
assessment window and the covariate assessment window are often defined to begin a set number of days
before the CED and end the day before or the day of
the CED (Table) (24). These windows are sometimes
identical, but in some studies, separate windows may
be specified for subsets of exclusion criteria or confounders. For example, history of cancer might be measured over all available time before the CED, whereas
Annals of Internal Medicine

3

RESEARCH AND REPORTING METHODS
recent myocardial infarction might be measured within
30 days before the CED. Research has suggested that
use of a flexible window for covariate assessment, starting from the beginning of the available data stream
and continuing until the day of CED, is preferable to
use of a fixed window (25). The effect on confounding
adjustment may vary by setting (26).
For studies where exposure is not a first-order anchor, it can be defined in an exposure assessment window. This window itself is defined relative to the CED.
For example, a cohort study looking at risk for cardiovascular outcomes in patients after percutaneous coronary intervention or acute coronary syndrome defined
the CED as the date of hospital discharge (27). Patients
were further required to receive clopidogrel for the first
time within 7 days after the CED. The exposure of interest was proton-pump inhibitor use, which was assessed
during the 21 days before and 7 days after the CED. To
avoid immortal time bias, outcomes should not be
counted as exposed outcomes until after the exposure
definition has been met (28).
In many applications, we want to make sure that
the outcome of interest has not yet occurred at the time
of study entry. To study newly occurring events, investigators can require an outcome washout window. Similarly, new use of a drug or other treatment can be defined by requiring an exposure washout window of
defined duration (Table).
The analytic follow-up window, during which the
study population is at risk for developing the outcome

Graphical Depiction of Study Designs

of interest, begins after study entry. It may begin on the
CED or after an assumed induction window before
which there is no biologically plausible effect of exposure on outcome. The maximum analytic window for
follow-up is defined by 1 or more censoring criteria. For
analyses that focus on follow-up time while patients are
exposed to a treatment, the analytic follow-up time may
incorporate stockpiling algorithms, grace windows for
drug exposures, hypothesized induction windows before the effect of exposure begins, or hypothesized duration of biological risk beyond the end of observed
exposure (16, 29).
The outcome event date is the date of outcome
occurrence during analytic follow-up. For some study
designs, such as case-crossover (where assessment
windows are anchored on the outcome), the outcome
event date is a first-order anchor equal to the CED. In
the nested case– control design, secondary temporal
anchors may be defined relative to the CED for the
underlying source cohort as well as the outcome event
date.

GRAPHICAL REPRESENTATION OF DESIGN
IMPLEMENTATION
Because of the complexity of the timeline and the
interrelated nature of the factors described in this article, researchers often find it helpful to illustrate their
study design implementation on the longitudinal health
care record of an imaginary patient. However, the de-

Figure 2. Exposure-based cohort entry where the cohort entry date is selected before application of exclusion criteria.
Cohort Entry Date
(First prescription of ACEI or ARB)
Day 0
Exclusion Assessment Window
(Intermittent medical and drug coverage)*
Days [–183, –1]
Washout Window (exposure, outcome)
(No ACEI, ARB, angioedema)
Days [–183, –1]
Exclusion Assessment Window
(Age ≤18 y, initiate both ACEI and ARB)
Days [0, 0]
Covariate Assessment Window
(Age, sex)
Days [0, 0]
Covariate Assessment Window
(Baseline conditions)†
Days [–183, –1]
Follow-up Window
Days [0, Censor]‡

Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html. ACEI = angiotensin-converting
enzyme inhibitor; ARB = angiotensin-receptor blocker.
* Up to 45-d gaps in medical or pharmacy enrollment were allowed.
† Baseline conditions included allergic reactions, diabetes, heart failure, ischemic heart disease, and use of nonsteroidal anti-inflammatory drugs.
‡ Earliest of outcome of interest (angioedema), switching or withdrawing study drugs, death, disenrollment, 365 d of follow-up, or end of study period.
4 Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Annals.org

RESEARCH AND REPORTING METHODS

Graphical Depiction of Study Designs

Figure 3. Exposure-based cohort entry where the cohort entry date is selected after application of exclusion criteria.
Cohort Entry Date
(First prescription of ACEI or ARB in a treatment episode*)
Day 0
Exclusion Assessment Window
(Intermittent medical and drug coverage)†
Days [–183, –1]
Washout Window (exposure, outcome)
(No ACEI, ARB, angioedema)
Days [–183, –1]
Exclusion Assessment Window
(Age ≤18 y, initiate both ACEI and ARB)
Days [0, 0]
Keep first qualifying new initiator episode
observed within study period for each patient
Covariate Assessment Window
(Age, sex)
Days [0, 0]
Covariate Assessment Window
(Baseline conditions)‡
Days [–183, –1]
Follow-up Window
Days [0, Censor]§
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html. ACEI = angiotensin-converting
enzyme inhibitor; ARB = angiotensin-receptor blocker.
* Treatment episodes were defined by date of dispensing and days' supply with a stockpiling algorithm if a new dispensing occurred before the end
of days' supply. Gaps of <30 d between end of days' supply and next dispensing were bridged. 30 d was added to the last dispensing days' supply
in an exposure episode.
† Up to 45-d gaps in medical or pharmacy enrollment were allowed.
‡ Baseline conditions included allergic reactions, diabetes, heart failure, ischemic heart disease, and use of nonsteroidal anti-inflammatory drugs.
§ Earliest of outcome of interest (angioedema), switching or withdrawing study drugs, death, disenrollment, 365 d of follow-up, or end of study period.

sign elements represented in a diagram and the level
of detail provided in published reports vary widely (30 –
34). We propose a framework for visualizing the design
of nonrandomized database study implementation that
uses standardized structure and terminology and focuses on summarizing details of first- and second-order
temporal anchors (Table). These design diagrams include bracketed numbers representing time intervals
anchored on the CED (day 0). Following conventional
mathematical notation, we indicate open intervals (which
do not include the end points) with parentheses and
closed intervals (which do include the end points) with
square brackets. First-order time anchors are represented
as columns indicating a date on the patient timeline,
whereas second-order anchors (time windows) are represented as separate boxes. Boxes are placed in different
rows so that overlap can be easily distinguished. The
steps to create the analytic cohort from data tables in the
longitudinal source are laid out sequentially from top to
bottom in the design diagram. Attrition tables could be
incorporated into these diagrams, with patient counts inserted in the relevant rows for exclusion criteria. We used
standardized structure and terminology to provide examples of graphical representation for several designs that
can be used in nonrandomized database studies, includAnnals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

ing cohort designs; designs that sample from cohorts
(case– control, case– cohort, and 2-stage sampling); and
self-controlled study designs.
Cohort Study
The cohort study design is widely used in research in
large health care databases and encompasses a range of
designs in which a group of patients enters the study population on the CED. Baseline characteristics or covariates
are usually (but not always) defined before and outcome
events after the CED. When covariate assessment windows are after the CED (for example, time-varying covariates), these should occur before the relevant exposure
assessment window to avoid adjustment for causal intermediates. Numerous variations of the cohort study design
could be implemented. These decisions can greatly affect
results. For example, study entry could be based on initiation of an exposure of interest, occurrence of a health
event, calendar time, or a combination thereof (28). Patients could be allowed to enter only 1 time or every time
they meet entry criteria.
Example 1: Exposure-based cohort entry. A cohort
study investigated whether angiotensin-converting
enzyme inhibitors (ACEIs) differ from angiotensinreceptor blockers (ARBs) with respect to risk for angioAnnals of Internal Medicine

5

RESEARCH AND REPORTING METHODS

Graphical Depiction of Study Designs

edema (35). The inclusion criterion was initiation of use
of a study drug (ACEI or ARB) after 183 or more days
without dispensings of either of the drug groups being
compared (Figure 2). In this study, as in most, patients
were allowed to enter the study population only 1 time.
The CED was the date of first prescription for ACEI or
ARB. Exclusion criteria were then applied. Patients were
excluded if they were younger than 18 years or started
receiving both ACEI and ARB on the CED. Patients
were also excluded if they had intermittent medical and
drug coverage (defined as gaps in coverage >45 days)
in the 183 days before, but not including, the CED. The
covariate assessment window and washout for incident
exposure and outcome were also the 183 days before,
but not including, the CED. Follow-up began on the
CED and continued until the outcome of interest (angioedema), switching or withdrawal of study drugs,
death, disenrollment, 365 days of follow-up, or the end
of the study period, whichever came first.
In contrast to this study, which defined the CED
before applying exclusion criteria, a similar study investigating ACEI versus ARB on risk for angioedema identified every new episode of new treatment initiation
within the study period and then picked only the first

new initiation episode for each patient after exclusion
criteria were applied (Figure 3) (36).
Example 2: Exposure-based cohort entry restricted
to adherent users. A cohort study investigated whether
statins differed from glaucoma agents with respect to
mortality risk among patients who adhered to statin or
glaucoma therapy and were not at high risk for death
(37). The CED was defined by initiation of study drug
use after at least 12 months continuously enrolled without
any dispensing of study drugs. Nonadherent patients
were excluded, where nonadherence was defined as
fewer than 3 dispensings for statin or glaucoma therapy
within 180 days (Figure 4). The study specified an exclusion assessment window of 12 months before the CED to
exclude patients with evidence of dementia or cancer and
those without evidence of at least 1 risk factor for a major
vascular event (angina; intermittent claudication; hypertension; diabetes; history of stroke, transient ischemic attack, myocardial infarction, arterial surgery, or amputation
for vascular disease; or smoking), as well as a 6-month
exclusion assessment window to exclude patients with
cardiovascular-related hospitalizations. Patients were excluded if they were younger than 65 years on the CED or
started receiving both statins and glaucoma agents on the

Figure 4. Exposure-based cohort entry restricted to adherent users.
Cohort Entry Date
(Initiation of statin or glaucoma Rx)
Day 0
Washout Window
(No exposure to statin, glaucoma Rx)
Days [–365, –1]
Exclusion Assessment Window
(Intermittent medical and drug coverage,
baseline conditions)*
Days [–365, –1]
Exposure Assessment
Window
Days [0, 180]
Exclusion Assessment Window
(CVD conditions)
Days [–183, –1]
Exclusion Assessment Window
(Age ≤65 y, initiate statin and glaucoma Rx)
Days [0, 0]
Covariate Assessment Window
(Baseline conditions)†
Days [–365, –1]
Covariate Assessment Window
(Age, sex)
Days [0, 0]
Follow-up Window
Days [3rd refill, Censor]‡
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html. CVD = cardiovascular disease;
Rx = prescription.
* Excluded if there is evidence of dementia or cancer or no evidence of ≥1 of the following conditions: angina, intermittent claudication, hypertension, diabetes, history of stroke, transient ischemic attack, myocardial infarction, arterial surgery, amputation for vascular disease, or smoking.
† Full list and code algorithms are in the Appendix (available at Annals.org).
‡ Censored at earliest of outcome, death, disenrollment, or end of study period.
6 Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Annals.org

RESEARCH AND REPORTING METHODS

Graphical Depiction of Study Designs

Figure 5. Visualizing a nested case– control design with risk-set sampling.
Cohort Entry Date
(Initiation of any antidiabetic therapy)
Day 0
Washout Window
(No oral antidiabetic agents)
Days [–∞, –1]
Exclusion Assessment Window
(History of bladder cancer)
Days [–∞, –1]
Exclusion Assessment Window
(>1 y medical history)
Days [–365, –1]
Exclusion Assessment Window
(Age <40 y, prescribed insulin)
Days [0, 0]
Covariate Assessment Window
(Baseline conditions)
Days [–∞, –1]
Follow-up Window
Days [0, Censor]*
ED†
Exclusion Assessment Window
(<1 y between cohort entry
and ED)
Days [0, 365]
Exposure Assessment Window
(Ever pioglitazone or rosiglitazone)
[0, ED – 365]
Covariate Assessment Window
(Age, sex)
Days [ED, ED]
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html. ED = event date.
* Censored at first of incident bladder cancer, death, disenrollment, or end of study period.
† Control patients were risk-set matched on year of cohort entry, duration of follow-up (from cohort entry), age, and sex.

CED. Confounders for eligible patients were captured in a
12-month covariate assessment window before the CED.
Follow-up began on the date of the third refill and continued until outcome, death, disenrollment, or end of the
study period.
Nested Case–Control Study
A nested case– control study samples the analytic
study population from a fully enumerated source cohort (17). In database studies, the source cohort for a
case– control study can be fully enumerated, making
nested case– control studies feasible. The CED is the
date of entry to the source cohort. Exclusions are applied to the source cohort before or on the CED, and
the follow-up window begins on or after the CED. Case
patients are identified on the basis of occurrence of the
outcome-defining event during the cohort follow-up
window. Exposure is assessed in 1 or more windows
that fall between the CED and the outcome event date.
When risk-set sampling of control patients is used, a
fixed number of members of the source cohort who are
Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

at risk for the outcome on the date of a given case
patient's event are sampled as potential control patients. With such individual matching, a control patient's person-time is anchored by the outcome event
date for the case patient to which he or she is matched.
Example 3: A nested case– control study with riskset sampling. A nested case– control study compared
pioglitazone versus other oral antidiabetic agents on
risk for bladder cancer (38). The CED for the cohort was
first initiation of antidiabetic agent use, defined with a
washout window that included all available data before
the CED (Figure 5). Patients were required to be enrolled in the primary care database with at least 1 year
of medical history before the CED. They were excluded
if the first antidiabetic drug prescribed was insulin, they
were younger than 40 years on the CED, or they had a
history of bladder cancer ever recorded before the
CED. The covariate assessment period included all
available data before the CED. Follow-up started on the
CED and continued until censoring at the first of inciAnnals of Internal Medicine

7

RESEARCH AND REPORTING METHODS
dent bladder cancer, death, disenrollment, or end of
the study period. The diagnosis date of incident bladder cancer was the outcome date. Only case patients
with at least 1 year of follow-up between the CED and the
event date of the case-defining outcome were included.
At the time of each such date, matched control patients
were sampled from the cohort of persons who initiated
antidiabetic therapy (matched on year of cohort entry, duration of follow-up, age, and sex) and were still at risk for
incident bladder cancer. The exposure assessment window began at the CED and continued until 1 year before
the outcome event date for case patients and their
matched control patients in the nested sample.
The Appendix gives additional examples of calendar
time– based cohort entry and self-controlled designs.

DISCUSSION
In this article, we focus on use of graphical representation to clearly communicate design decisions made
when generating evidence from administrative and clinical data that were collected as part of routine care, not for
research purposes. We provide examples of graphical
representation for different study designs using standardized structure and terminology. The figures in this article
are freely available for download from drugepi.org, and
users can adapt them as needed. We look forward to user
experiences and suggestions for improvement.
Visualization of study design is a powerful communication tool that provides a clear and concise summary
of study implementation details. It can help consumers
of pharmacoepidemiologic and pharmacoeconomic
evidence assess how that evidence was generated, but
it does not remove the need for examination of study
strengths and limitations, including measurement issues, which should be discussed in reports. Recent
publications of database studies have provided informative graphs that show key aspects of longitudinal
study designs, varying from high-level conceptual descriptions to information-packed diagrams (31, 33, 34).
These diagrams depict critical temporal aspects with
clarity. Once the basic temporal aspects of a study are
understood, it is easier to comprehend the longer
prose typically used to describe study design in detail.
We believe that a widely used framework with a common structure and terminology for graphical representation of database study designs would promote clearer understanding of database research. This framework would
encourage researchers and reviewers to think systematically about time-related aspects in the context of typical
study designs when designing studies or preparing manuscripts. It would also help readers understand critical
temporal aspects of a longitudinal database study. Ultimately, these factors support the confidence of decision
makers in evidence generated from nonrandomized database studies.
From Brigham and Women's Hospital and Harvard Medical
School, Boston, Massachusetts (S.S., S.V.W.); Aetion, New
York, New York (J.R., W.M.); Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts
(J.B.); RTI Health Solutions, Durham, North Carolina (K.J.R.);
8 Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Graphical Depiction of Study Designs
Journal of Managed Care and Specialty Pharmacy, Alexandria,
Virginia (L.H.); European Medicines Agency, London, United
Kingdom (P.A.); U.S. Food and Drug Administration, Silver
Spring, Maryland (G.D.); and The National Health Care Institute,
Diemen, and Utrecht University, Utrecht, the Netherlands (W.G.).
Disclaimer: The views expressed in this article are those of the

authors and may not be understood or quoted as being made
on behalf of or reflecting the position of the European Medicines Agency or the U.S. Food and Drug Administration.
Financial Support: Writing of this manuscript was supported

by internal funds from the Division of Pharmacoepidemiology
and Pharmacoeconomics of the Department of Medicine at
Brigham and Women's Hospital and Harvard Medical School.
Disclosures: Dr. Schneeweiss reports grants from the U.S.
Food and Drug Administration, the Patient-Centered Outcomes Research Institute, and the National Institutes of Health
during the conduct of the study and personal fees from
WHISCON, equity in Aetion, and being principal investigator
of research contracts to Brigham and Women's Hospital from
Bayer, Vertex, Boehringer Ingelheim, and the Arnold Foundation outside the submitted work. In addition, Dr. Schneeweiss
has a patent for a database system for analysis of longitudinal
data sets, with no royalties paid. Dr. Rassen reports that he is
an employee of and has an ownership stake in Aetion outside
the submitted work. Dr. Murk reports that he is an employee
of and holds stock options in Aetion outside the submitted
work. Dr. Wang reports being principal investigator on research contracts to Brigham and Women's Hospital from
Boehringer Ingelheim, Novartis, and Johnson & Johnson, and
the Arnold Foundation outside the submitted work. She is
also a consultant to Aetion for unrelated work. Dr. Arlett is a
full-time employee of the European Medicines Agency. Dr.
Dal Pan is a full-time employee of the Food and Drug Administration. Authors not named here have disclosed no conflicts
of interest. Disclosures can also be viewed at www.acponline
.org/authors/icmje/ConflictOfInterestForms.do?msNum=M18
-3079.
Corresponding Author: Shirley V. Wang, PhD, 1620 Tremont

Street, Suite 303, Boston, MA 02120; e-mail, swang1@bwh
.harvard.edu.
Current author addresses and author contributions are available at Annals.org.

References
1. Ball R, Robb M, Anderson SA, Dal Pan G. The FDA's sentinel
initiative—a comprehensive approach to medical product surveillance. Clin Pharmacol Ther. 2016;99:265-8. [PMID: 26667601] doi:
10.1002/cpt.320
2. Makady A, Ham RT, de Boer A, Hillege H, Klungel O, Goettsch W;
GetReal Workpackage 1. Policies for use of real-world data in health
technology assessment (HTA): a comparative study of six HTA agencies. Value Health. 2017;20:520-32. [PMID: 28407993] doi:10.1016
/j.jval.2016.12.003
3. Hampson G, Towse A, Dreitlein B, Henshall C, Pearson SD. Real
world evidence for coverage decisions: opportunities and challenges. A report from the 2017 ICER membership policy summit.
Boston: Institute for Clinical and Economic Review; 2018. Accessed
at https://icer-review.org/wp-content/uploads/2018/03/ICER-Real
-World-Evidence-White-Paper-03282018.pdf on 14 January 2019.
4. Malone DC, Brown M, Hurwitz JT, Peters L, Graff JS. Real-world
evidence: useful in the real world of US payer decision making?
Annals.org

Graphical Depiction of Study Designs
How? When? And what studies? Value Health. 2018;21:326-33.
[PMID: 29566840] doi:10.1016/j.jval.2017.08.3013
5. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Agreement of treatment effects for mortality from routinely collected data
and subsequent randomized trials: meta-epidemiological survey.
BMJ. 2016;352:i493. [PMID: 26858277] doi:10.1136/bmj.i493
6. Maclure M, Schneeweiss S. Causation of bias: the episcope.
Epidemiology. 2001;12:114-22. [PMID: 11138805]
7. Suissa S. Immortal time bias in observational studies of
drug effects. Pharmacoepidemiol Drug Saf. 2007;16:241-9. [PMID:
17252614]
8. Gruber S, Chakravarty A, Heckbert SR, Levenson M, Martin D,
Nelson JC, et al. Design and analysis choices for safety surveillance
evaluations need to be tuned to the specifics of the hypothesized
drug-outcome association. Pharmacoepidemiol Drug Saf. 2016;25:
973-81. [PMID: 27418432] doi:10.1002/pds.4065
9. Jena AB, Goldman D, Weaver L, Karaca-Mandic P. Opioid prescribing by multiple providers in Medicare: retrospective observational study of insurance claims. BMJ. 2014;348:g1393. [PMID:
24553363] doi:10.1136/bmj.g1393
10. Suissa S. Immeasurable time bias in observational studies of
drug effects on mortality. Am J Epidemiol. 2008;168:329-35. [PMID:
18515793] doi:10.1093/aje/kwn135
11. Moride Y, Abenhaim L, Yola M, Lucein A. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47:731-7. [PMID: 7722586]
12. Herna´n MA, Alonso A, Logan R, Grodstein F, Michels KB,
Willett WC, et al. Observational studies analyzed like randomized
experiments: an application to postmenopausal hormone therapy
and coronary heart disease. Epidemiology. 2008;19:766-79. [PMID:
18854702] doi:10.1097/EDE.0b013e3181875e61
13. Goodman SN, Schneeweiss S, Baiocchi M. Using design thinking
to differentiate useful from misleading evidence in observational research [Editorial]. JAMA. 2017;317:705-7. [PMID: 28241335] doi:10
.1001/jama.2016.19970
14. Langan SM, Schmidt SA, Wing K, Ehrenstein V, Nicholls SG, Filion KB, et al. The reporting of studies conducted using observational
routinely collected health data statement for pharmacoepidemiology (RECORD-PE). BMJ. 2018;363:k3532. [PMID: 30429167] doi:10
.1136/bmj.k3532
15. Berger ML, Sox H, Willke RJ, Brixner DL, Eichler HG, Goettsch W,
et al. Good practices for real-world data studies of treatment and/or
comparative effectiveness: recommendations from the joint ISPORISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf. 2017;26:1033-9. [PMID:
28913966] doi:10.1002/pds.4297
16. Wang SV, Schneeweiss S, Berger ML, Brown J, de Vries F, Douglas I, et al; joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making. Reporting to improve reproducibility and facilitate validity assessment for healthcare database
studies V1.0. Pharmacoepidemiol Drug Saf. 2017;26:1018-32.
[PMID: 28913963] doi:10.1002/pds.4295
17. Rothman KJ, Greenland S, Lash TL. Modern Epidemiology. 3rd
ed. Philadelphia: Lippincott Williams & Wilkins; 2008.
18. Gini R, Schuemie M, Brown J, Ryan P, Vacchi E, Coppola M, et al.
Data extraction and management in networks of observational
health care databases for scientific research: a comparison of EUADR, OMOP, Mini-Sentinel and MATRICE strategies. EGEMS (Wash
DC). 2016;4:1189. [PMID: 27014709] doi:10.13063/2327-9214.1189
19. Curtis LH, Weiner MG, Boudreau DM, Cooper WO, Daniel GW,
Nair VP, et al. Design considerations, architecture, and use of the
Mini-Sentinel distributed data system. Pharmacoepidemiol Drug Saf.
2012;21 Suppl 1:23-31. [PMID: 22262590] doi:10.1002/pds.2336
20. Matcho A, Ryan P, Fife D, Reich C. Fidelity assessment of a Clinical Practice Research Datalink conversion to the OMOP common
data model. Drug Saf. 2014;37:945-59. [PMID: 25187016] doi:10
.1007/s40264-014-0214-3
21. Schneeweiss S. Automated data-adaptive analytics for electronic
healthcare data to study causal treatment effects. Clin Epidemiol.
2018;10:771-88. [PMID: 30013400] doi:10.2147/CLEP.S166545
Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

RESEARCH AND REPORTING METHODS
22. Hartzema AG, Racoosin JA, MaCurdy TE, Gibbs JM, Kelman JA.
Utilizing Medicare claims data for real-time drug safety evaluations: is
it feasible? Pharmacoepidemiol Drug Saf. 2011;20:684-8. [PMID:
21847800]
23. Herna´n MA, Herna´ndez-Dı´az S. Beyond the intention-to-treat in
comparative effectiveness research. Clin Trials. 2012;9:48-55. [PMID:
21948059] doi:10.1177/1740774511420743
24. Schneeweiss S. A basic study design for expedited safety signal
evaluation based on electronic healthcare data. Pharmacoepidemiol
Drug Saf. 2010;19:858-68. [PMID: 20681003] doi:10.1002/pds.1926
25. Brunelli SM, Gagne JJ, Huybrechts KF, Wang SV, Patrick AR,
Rothman KJ, et al. Estimation using all available covariate information versus a fixed look-back window for dichotomous covariates.
Pharmacoepidemiol Drug Saf. 2013;22:542-50. [PMID: 23526818]
doi:10.1002/pds.3434
26. Nakasian SS, Rassen JA, Franklin JM. Effects of expanding the
look-back period to all available data in the assessment of covariates.
Pharmacoepidemiol Drug Saf. 2017;26:890-9. [PMID: 28397352]
doi:10.1002/pds.4210
27. Rassen JA, Choudhry NK, Avorn J, Schneeweiss S. Cardiovascular outcomes and mortality in patients using clopidogrel with
proton pump inhibitors after percutaneous coronary intervention or
acute coronary syndrome. Circulation. 2009;120:2322-9. [PMID:
19933932] doi:10.1161/CIRCULATIONAHA.109.873497
28. Suissa S. Immortal time bias in pharmaco-epidemiology. Am J
Epidemiol. 2008;167:492-9. [PMID: 18056625]
29. Rothman KJ. Induction and latent periods. Am J Epidemiol.
1981;114:253-9. [PMID: 7304560]
30. Layton JB, Kshirsagar AV, Simpson RJ Jr, Pate V, Jonsson Funk
M, Stu¨rmer T, et al. Effect of statin use on acute kidney injury risk
following coronary artery bypass grafting. Am J Cardiol. 2013;111:
823-8. [PMID: 23273532] doi:10.1016/j.amjcard.2012.11.047
31. Kim SC, Solomon DH, Rogers JR, Gale S, Klearman M, Sarsour K,
et al. Cardiovascular safety of tocilizumab versus tumor necrosis factor inhibitors in patients with rheumatoid arthritis: a multi-database
cohort study. Arthritis Rheumatol. 2017;69:1154-64. [PMID: 28245350]
doi:10.1002/art.40084
32. Bykov K, Schneeweiss S, Glynn RJ, Mittleman MA, Bates DW,
Gagne JJ. Updating the evidence of the interaction between clopidogrel and CYP2C19-inhibiting selective serotonin reuptake inhibitors: a cohort study and meta-analysis. Drug Saf. 2017;40:923-32.
[PMID: 28623527] doi:10.1007/s40264-017-0556-8
33. Brookhart MA. Counterpoint: the treatment decision design.
Am J Epidemiol. 2015;182:840-5. [PMID: 26507307] doi:10.1093
/aje/kwv214
34. Douglas IJ, Langham J, Bhaskaran K, Brauer R, Smeeth L. Orlistat
and the risk of acute liver injury: self controlled case series study in
UK Clinical Practice Research Datalink. BMJ. 2013;346:f1936. [PMID:
23585064] doi:10.1136/bmj.f1936
35. Toh S, Reichman ME, Houstoun M, Ross Southworth M, Ding X,
Hernandez AF, et al. Comparative risk for angioedema associated
with the use of drugs that target the renin-angiotensin-aldosterone
system. Arch Intern Med. 2012;172:1582-9. [PMID: 23147456] doi:
10.1001/2013.jamainternmed.34
36. Gagne JJ, Wang SV, Fox M, Lash T, Eddings W, Dublin S, et al.
Mini-Sentinel methods: analytical methods to assess robustness of
drug safety monitoring results. 2015. Accessed at www.sentinelinitiative
.org/sites/default/files/Methods/Mini-Sentinel_Methods_Analytical
-Methods-to-Assess-Robustness-of-Drug-Safety-Monitoring-Results
_0.pdf on 14 January 2019.
37. Schneeweiss S, Patrick AR, Stu¨rmer T, Brookhart MA, Avorn J,
Maclure M, et al. Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results. Med Care. 2007;45:S131-42. [PMID: 17909372]
38. Azoulay L, Yin H, Filion KB, Assayag J, Majdan A, Pollak MN,
et al. The use of pioglitazone and the risk of bladder cancer in people with type 2 diabetes: nested case-control study. BMJ. 2012;344:
e3645. [PMID: 22653981] doi:10.1136/bmj.e3645
Annals of Internal Medicine

9

Current Author Addresses: Drs. Schneeweiss and Wang: 1620
Tremont Street, Suite 303, Boston, MA 02120.
Drs. Rassen and Murk: 1441 Broadway 20th Floor, New York,
NY 10018.
Dr. Brown: 401 Park Drive, Suite 401 East, Boston, MA 02215.
Dr. Rothman: 307 Waverley Oaks Road, Suite 101, Waltham,
MA 02452-8413.
Dr. Happe: JMCP, 675 North Washington Street, Suite 220,
Alexandria, VA 22314.
Dr. Arlett: Department of Pharmacovigilance and Epidemiology, European Medicines Agency, 30 Churchill Place, London
E14 5EU, United Kingdom.
Dr. Dal Pan: 10903 New Hampshire Avenue, Silver Spring, MD
20993.
Dr. Goettsch: Zorginstituut Nederland, Health Care Insurance
Board Health Care TA Program, Postbus 320, NL-1110 AH
Diemen, the Netherlands.
Author Contributions: Conception and design: S. Schneeweiss,

S.V. Wang, J.A. Rassen.
Analysis and interpretation of the data: S. Schneeweiss, S.V.
Wang.
Drafting of the article: S. Schneeweiss, J.A. Rassen, J.S. Brown,
K.J. Rothman, L. Happe, P. Arlett, G. Dal Pan, W. Goettsch, W.
Murk, S.V. Wang.
Critical revision of the article for important intellectual content: S. Schneeweiss, J.A. Rassen, J.S. Brown, K.J. Rothman, L.
Happe, P. Arlett, G. Dal Pan, W. Goettsch, W. Murk, S.V.
Wang.
Final approval of the article: S. Schneeweiss, J.A. Rassen, J.S.
Brown, K.J. Rothman, L. Happe, P. Arlett, G. Dal Pan, W.
Goettsch, W. Murk, S.V. Wang.

APPENDIX: ADDITIONAL DESIGN
VISUALIZATION EXAMPLES
Calendar Time–Based Cohort Entry Example
A simple cohort study investigated how well a combined comorbidity score predicted 1-year mortality
(39). A uniform calendar time– based CED was assigned
(1 January 2005) regardless of whether patients had a
health care encounter on that date. Patients were required to be aged 65 years or older and have no record of death in the year before the CED. They were
further required to have a recorded pharmacy dispensing between 365 and 485 days before the CED. These
requirements were designed to restrict the population
to older adults who were likely to have been enrolled in
the pharmacy benefits program during the study period. Risk factors in 2 versions of the comorbidity score
were defined within the 365 days before, but not including, the CED. Additional baseline covariates were
measured over the same time frame. Patients were followed for the outcome for 1 year after the CED (Appendix Figure 1).
Self-Controlled Study Design Visualization
Self-controlled designs have many variants (41). All
use within-person comparisons (self-matching) to control time-invariant confounders. Effect estimates are
Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

generated from an analytic population comprising patients that have the outcome of interest as well as intermittent exposure over the sampled person-time. The
CED depends on the variant of self-controlled design.
For example, the primary anchor in the case-crossover
design is the case-defining outcome event (42). Assessment of exposure occurs in 2 or more windows before
the outcome event date. The exposure assessment windows are often separated by a washout window to address carryover effects of exposure. In contrast, a selfcontrolled risk interval design is anchored on exposure,
and the CED is the date of initiating the exposure of
interest (43). Follow-up for outcomes occurs in 2 or
more intervals of person-time after the exposure-based
CED. The CED for the self-controlled case series is defined by the minimum age or calendar time boundary
where a patient contributes follow-up for the outcome
(44). Other variants of self-controlled designs may include follow-up for outcomes before and after an
exposure-based CED or exposure assessment window
before and after an outcome event– based CED (such
as self-controlled case series and prescription symmetry) (44, 45).

An Outcome-Indexed, Self-controlled Design
(Case-Crossover)

A case-crossover study evaluated whether acute respiratory infection transiently increased risk for first
myocardial infarction (46) (Appendix Figure 2). The
outcome event date was the first acute myocardial infarction ever recorded in the patient's longitudinal history within the source data range. Patients were excluded if they were older than 75 years on the date of
myocardial infarction or if they did not have medical
history within the data source for at least 3 years before
the outcome event date. Exposure to acute respiratory
infection was assessed in the 11 days before, but not
including, the myocardial infarction date, as well as in
an 11-day window 1 year earlier. Case patients with
discordant exposure to acute respiratory infection in
the 2 exposure assessment windows contributed to the
within-person matched analysis. Exposure status was
not evaluated during the washout window between the
2 exposure assessment windows.

An Exposure-Indexed, Self-controlled Design
(Self-controlled Risk Interval)

A self-controlled risk interval study using data from
an integrated electronic health record system evaluated the degree to which measles, mumps, rubella
(MMR) vaccines increased risk for febrile seizures in
children aged 11 to 23 months (47). (Appendix Figure
3). Children were allowed to contribute to the analytic
data set every time they met inclusion and exclusion
Annals of Internal Medicine

criteria. The CED was defined as the date of MMR administration. Eligible CEDs could not have any vaccinations in the immunization schedule or diagnoses of febrile seizures recorded in the 56 days prior. Only
incident outcomes were included in the analysis. Incident outcomes were defined by the first inpatient or
emergency department code for seizure after 56 days
without any codes for seizure. The analytic follow-up
windows where children were considered to be at risk

for seizure were the 7 to 10 days after MMR vaccination
(hypothesized exposure risk window) and 14 to 56 days
after MMR vaccination (reference window). Days 1 to 6
were considered induction time before the biological effect of vaccination on seizure plausibly begins, and days
11 to 13 were used as washout for potential carryover
exposure effects. The analysis conditioned on the individual to make within-person analyses and accounted for differential person-time in follow-up windows.

Appendix Figure 1. Time-based cohort entry.
Cohort Entry Date
(1 January 2005)
Day 0
Exclusion Assessment Window
(Age ≤65 y)
Days [0, 0]
Exclusion Assessment Window
(No pharmacy claim)
Days [–485, –365]
Exclusion Assessment Window
(Died)
Days [–365, –1]
Covariate Assessment Window
(Predictors of mortality, comorbidity)*
Days [–365, –1)
Follow-up Window
Days [0, 365)
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html.
* Predictors of mortality included 17 conditions included in Romano's adaptation of the Charlson Index (40) and 30 conditions included in the
Elixhauser score (details in Appendix). Other comorbid conditions measured included hospitalization, use of any prescription drug, receipt of any
diagnosis, any physician visit, any time in a nursing home, number of hospital days, number of distinct prescription drugs used, number of
diagnoses, and number of physician visits.

Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Annals.org

Appendix Figure 2. An outcome-indexed, self-controlled design (case-crossover).
Event Date
(Myocardial infarction)
Day 0
Washout Window
(Outcome: myocardial infarction)
Days [–∞, –1]
Exclusion Assessment Window
(>3 y of medical history in data source)
Days [–1095, –1]
Exclusion Assessment Window
(Age ≤75 y)
Days [0, 0]
Exposure Assessment Window
(Acute respiratory infection)
Days [–11, –1]
Washout Window
(Exposure effects)
Days [–364, –12]
Exposure Assessment Window
(Acute respiratory infection)
Days [–376, –365]
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html.

Annals.org

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

Annals of Internal Medicine

Appendix Figure 3. An exposure-indexed, self-controlled design (self-controlled risk interval).
Cohort Entry Date
(Vaccination)
Day 0
Washout Window
(Exposure)
Days [–56, –1]
ED
(Seizure)
Washout Window
(Outcome)
Days [ED – 56, ED – 1]
Exclusion Assessment Window
(If vaccine and seizure do not
occur within 56 d)
Days [1, 56]
Exclusion Assessment Window
(Age <11 mo or >23 mo)
Days [0, 0]
Follow-up Window
(Hypothesized exposure risk window)
Days [7, 10]
Washout Window
(Exposure effects)
Days [11, 13]
Follow-up Window
(Reference window)
Days [14, 56]
Time

This work is licensed under CC BY, and the original versions can be found at www.repeatinitiative.org/projects.html. ED = event date.

Web-Only References
39. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64:749-59. [PMID:
21208778] doi:10.1016/j.jclinepi.2010.10.004
40. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity
index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46:1075-9. [PMID: 8410092]
41. Maclure M, Fireman B, Nelson JC, Hua W, Shoaibi A,
Paredes A, et al. When should case-only designs be used for
safety monitoring of medical products? Pharmacoepidemiol
Drug Saf. 2012;21 Suppl 1:50-61. [PMID: 22262593] doi:
10.1002/pds.2330
42. Maclure M. The case-crossover design: a method for studying
transient effects on the risk of acute events. Am J Epidemiol. 1991;
133:144-53. [PMID: 1985444]

Annals of Internal Medicine

Downloaded from https://annals.org by Harvard Medical School user on 03/12/2019

43. Baker MA, Lieu TA, Li L, Hua W, Qiang Y, Kawai AT, et al. A
vaccine study design selection framework for the postlicensure rapid
immunization safety monitoring program. Am J Epidemiol. 2015;
181:608-18. [PMID: 25769306] doi:10.1093/aje/kwu322
44. Whitaker HJ, Farrington CP, Spiessens B, Musonda P. Tutorial in
biostatistics: the self-controlled case series method. Stat Med. 2006;
25:1768-97. [PMID: 16220518]
45. Hallas J. Evidence of depression provoked by cardiovascular
medication: a prescription sequence symmetry analysis. Epidemiology. 1996;7:478-84. [PMID: 8862977]
46. Meier CR, Jick SS, Derby LE, Vasilakis C, Jick H. Acute
respiratory-tract infections and risk of first-time acute myocardial infarction. Lancet. 1998;351:1467-71. [PMID: 9605802]
47. Wang SV, Abdurrob A, Spoendlin J, Lewis E, Newcomer SR, Fireman B, et al. Methods for addressing “innocent bystanders” when evaluating safety of concomitant vaccines. Pharmacoepidemiol Drug Saf.
2018;27:405-12. [PMID: 29441647] doi:10.1002/pds.4399

Annals.org