PRDA - K:Progress report

Published: June 21, 2013
Progress Report
Grant ID: PHPB

Robert Wood Johnson Foundation PracticeBased Research Networks in Public Health
A Progress Report on a program to improve the performance
and capacity of public health agencies and systems
INTRODUCTION
Public health practice-based research networks (PBRNs) are practice-driven
partnerships between public health practitioners and researchers who collaboratively
identify, design, and carry out research studies on the organization, financing, and
delivery of public health services. The goal of these networks, each located within a
single state, is to improve the performance and capacity of local and state public health
agencies and systems.
The RWJF national program Practice-Based Research Networks in Public Health has
established 12 public health PBRNs. These networks, and 16 other networks established
on their own, receive technical assistance, dissemination support, and opportunities to
apply for research grants from the National Coordinating Center at the University of
Kentucky.
The RWJF Board of Trustees has approved three authorizations totaling up to $9.325
million for the networks since October 2007. Funding runs through November 2015.
See the Appendix for a list of people interviewed for this report.
WHAT ARE PUBLIC HEALTH PBRNS?
Public health PBRNs are “groups of practitioners and researchers who determine research
agendas jointly, conduct research collaboratively, and use the results of research to
inform the practice of state and local health departments,” says Robert Pestronk, MPH,
executive director of the National Association of County and City Health Officials
(NACCHO), which represents local health departments across the United States.
Paul K. Halverson, DrPH, a member of the public health PBRN national advisory
committee, has worn both a practice and a research hat—as director of the Arkansas
Department of Health and Professor of Public Health Policy and Management at the
University of Arkansas Medical School—positions he left since the interview for this

report to become the founding dean of the University of Indiana’s Richard M. Fairbanks
School of Public Health. Halverson describes the networks as settings in which academic
researchers and public health practitioners “together explore both the topics that will
make a difference in practice as well as look for ways to expand the body of knowledge
that describes contemporary public health practice.”
The networks were first established as part
of RWJF’s public health PBRN program.
Naima Wong, PhD, MPH, RWJF program
officer, notes the importance of the network
structure. “People have said ‘When you’ve
seen one public health department, you’ve
seen one public health department.’ So it is
important that the issues addressed go
beyond one public health department—and
networks make that happen.”
Where did they come from?

“We are trying not to
repeat the same errors
that medical research
went through before they
learned and started
partnering with patients
and the medical delivery
system.”—Glen P.
Mays, PhD, MPH,
program director

Public health PBRNs are building on the
PBRNs made up of clinicians and
researchers that began studying ways to
improve medical care in the 1980s.1
“Medicine has learned you can’t do medical research in a vacuum,” says Glen P. Mays,
PhD, MPH, director of the National Coordinating Center for the Practice-Based
Research Networks in Public Health at the University of Kentucky. “We are trying not to
repeat the same errors that medical research went through before they learned and started
partnering with patients and the medical delivery system.” Mays is the F. Douglas
Scutchfield Endowed Professor in Health Services and Systems Research at the
University of Kentucky.
Why study public health agencies and systems?

More than 75 percent of current health care costs are due to diseases that are largely
preventable, according to the Centers for Disease Control and Prevention (CDC).2 Yet,
1

In August 2002, RWJF funded a national program, Prescription for Health: Promoting Healthy Behaviors
in Primary Care Networks. See Program Results Report at www.rwjf.org/en/research-publications/findrwjf-research/2011/01/prescription-for-health-.html and a report on the diffusion of the model at
www.rwjf.org/en/research-publications/find-rwjf-research/2012/11/diffusion-of-a-model-for-addressingbehavioral-health-issues-in-.html for more information.
2
Centers for Disease Control and Prevention. Chronic Illness: The Power to Prevent, the Call to Control.
Atlanta: Centers for Disease Control and Prevention, 2009. Available at
www.cdc.gov/chronicdisease/resources/publications/aag/chronic.htm.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

2

there is very little evidence about how best to organize, finance, and deliver public health
services to prevent these diseases and reduce their costs.
Problems with evidence take several forms. When evidence-based strategies are
available, they are rarely feasible (due to logistics, cost, and politics) for the problems
that public health professionals routinely act on, such as preventing HIV. Also,
communities vary widely in their use of
available evidence-based strategies, such as
engaging the community in assessment
(identifying health and resource needs,
“We knew we had to
concerns, values, and community assets)
produce evidence that
and decision-making.
would drive
Evidence of what works in public health is
difficult to establish but especially
important because public health services are
delivered through multiple governmental
and private agencies characterized by wide
variation in resources and relationships.

improvements in the
public health systems
that would in turn,
improve public
health,”—Glen Mays,
program director

“We have recognized for a long time that
there is a thin evidence base to support the
everyday decisions made in public health
policy and practice. We knew we had to produce evidence that would drive
improvements in the public health systems that would in turn, improve public health,”
says Mays. This research has to be done in the real world, using knowledge that is
already available in the communities.
Where do networks fit within other RWJF efforts to build the public
health evidence base?

Public health PBRNs are part of the emerging field of public health services and systems
research, which “examines the organization, financing, and delivery of public health
services within communities, and the impact of these services on public health,”
according to an article by Mays and colleagues.3
RWJF’s commitment to developing this field includes the practice-based networks
program and the Public Health Services and Systems Research Program,4 Wong says. “I
3

Mays GP, Halverson PK, and Scutchfield FD. "Making Public Health Improvement Real: The Vital Role
of Systems Research." Journal of Public Health Management and Practice, 10(3): 183–185, 2004.
4
Other RWJF-supported activities include $12.5 million for a national program, Creating Resources for
Data Gathering and Study to Strengthen the Evidence Base, Performance, and Impact of Federal, State,
and Local Public Health and for grants to three public health associations (National Association of County

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

3

see the field building we are doing in Public Health Services and Systems Research
program as the overarching frame. The networks are a vehicle within which the field
building goes on.”
The Public Health Services and Systems Research Program has developed a new
research agenda to guide the future of the field. The University of Kentucky is also the
National Coordinating Center for this program, under the direction of F. Douglas
Scutchfield, MD. Scutchfield and Mays and the two programs join forces to conduct
research, support other researchers, and disseminate findings to practitioners and policymakers.
See the Public Health Services and Systems
Research Program website for more
information on this program.
HOW DO THE PUBLIC HEALTH
PBRNS WORK?
Program Management and
Guidance

RWJF established the National
Coordinating Center at the University of
Kentucky to oversee the creation and
operation of the public health PBRNs, and
tapped Mays to run it. Anna Goodman
Hoover, MA, PhD, is the deputy director.

“We have tried to not
have the networks based
in and controlled by the
academics. We wanted
the local health
department to be the
lead,”—Glen Mays,
program director

Mays in turn established a national advisory committee of public health researchers and
practitioners to help center staff select the networks, review ideas for studies, and
disseminate findings. The committee “provides a sounding board to try out new ideas,
give feedback on research approaches, and engage on a number of issues that are really
important,” according to committee member Halverson.
The Funded Networks

Through two rounds of competition and in consultation with RWJF staff, the national
advisory committee, and other experts, the National Coordinating Center selected
applicants in 12 states for funding. The first group of networks—Colorado, Kentucky,

and City Health Officials, Association of State and Territorial Health Officials, and the National
Association of Local Boards of Health). See Program Results Report at www.rwjf.org/en/researchpublications/find-rwjf-research/2011/05/strengthening-the-performance-and-impact-of-public-healthdepart.html.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

4

Massachusetts, North Carolina, and Washington—began operations in December 2008.
The second group—Connecticut, Florida, Minnesota, Nebraska, New York, Ohio, and
Wisconsin—began in December 2009.
Networks received up to $90,000 for two years, plus in-depth technical assistance to
establish an infrastructure, select a lead agency, recruit partners, identify technical
assistance needs, and develop and conduct
a preliminary small-scale research project.
They can also apply to the coordinating
center for competitive research grants
“I was impressed that
(described below).
there was enough
To increase the likelihood that research
studies would meet the needs of public
health practitioners, in selecting the
networks, preference was given to
applicants from public health agencies or
nonprofit organizations. “We have tried to
not have the networks based in and
controlled by the academics. We wanted
the local health department to be the lead,”
says Mays.

enthusiasm that even
without the seed money,
others would join,”—
Carolyn Leep, MS,
MPH, senior director of
research and evaluation
at NACCHO

Only one network, Ohio, is housed in an academic center, at Case Western Reserve
University. The program director, Scott Frank, MD, MS, knows that the involvement of
Ohio’s health departments is essential. “We have 125 local health departments in our
network, and we believe we have had participation from at least 110 of them,” he says.
The Affiliate Networks

When word got out about the networks, interest was so high that Mays created a parallel
affiliate program for networks that did not receive RWJF funds. Affiliates meet the same
criteria as funded networks and have access to technical assistance, networking
opportunities, and funds for research grants from the National Coordinating Center.
By April 2013, 16 affiliates had joined the program: Alabama, Arkansas, California,
Georgia, Illinois, Iowa, Kansas, Maryland, Missouri, New Hampshire, New Jersey,
Pennsylvania, South Carolina, Tennessee, Texas, and Vermont.
“I was impressed that there was enough enthusiasm that even without the seed money,
others would join,” said Carolyn Leep, MS, MPH, senior director of research and
evaluation at NACCHO.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

5

Why Technical Assistance Is So Important

The public health PBRNs are young entities involving an array of partners trying to
collaborate in new ways. Mays and his colleagues created a system of technical
assistance, training, and peer-learning opportunities to help the networks develop their
infrastructures, build relationships, and execute their research projects. “The networks
involve partnerships,” he says, “so they
need help in decision-making and
governance, collaboration, and engaging
the right talent pool from the academic
“The research faculty
side.”
have to be concerned not
The National Coordinating Center helped
the 12 RWJF-funded networks complete a
needs assessment and visited each one to
discuss its plans for network development
and research initiation. All of the networks
continue to have access to other types of
assistance, including web-based meetings,
webinars, conference calls, center
publications, electronic resources, and an
annual meeting.

just about what is
interesting—but also
what can be funded.”—
National Advisory
Committee member Paul
K. Halverson, DrPH

The web-based meetings, focused on research-in-progress, are especially useful for
Minnesota’s network. “They start with a review of emerging research from a network.
That is a timely and insightful way to know what other networks are doing,” says Kim
Gearin, PhD, senior research scientist at the Minnesota Department of Health and a
leader of Minnesota’s network.
Skill-building webinars address scientific and operational approaches to public health
PBRNs. Recordings of webinars and web-based meetings are available on the center’s
website.
During special-topic networking conference calls, members discuss shared research
interests and common methodological issues. An electronic newsletter profiles activities,
resources, and funding opportunities. The National Coordinating Center also posts
funding opportunities, guidance documents, reviews, and reports on its website and
manages electronic discussion forums. The networks meet in person each April in
Kentucky.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

6

WHAT TYPE OF RESEARCH ARE THE NETWORKS DOING?
The multifaceted research grants program in which individual networks apply for grants
to study topics of importance to members is managed by the National Coordinating
Center. In addition, groups of networks can collaborate on larger-scale data collection
and analyses across diverse geographical areas.
These grants “enable practice-based research to occur,” says national advisory committee
member Halverson. “The reality is that research faculty have to be concerned not just
about what is interesting—but also what can be funded.” These grants also help the
networks build their research capacity and position themselves to pursue other funding.
The main pots of research money available from the center are described below.
Quick Strike Research Fund

This allocation of money is for rapid-response, time sensitive research projects on
emerging issues that are generally completed within three to six months. The first Quick
Strike Research projects—studies conducted by the Kentucky and North Carolina
networks of local public health responses to the 2009 H1N1 influenza outbreak—were
completed between August and October that year. The National Coordinating Center has
funded 21 Quick Strike Research studies, as of October 2012.
Research Implementation Awards

These larger-scale research projects examined the implementation of evidence-based
practices in public health. The first research implementation grants began in 2010 and by
February 2011, the center had funded 10 projects. Studies included:
●

The influence of public health agency size, performance standards, and
regionalization on the use of evidence-based practices for food safety and infectious
disease control, conducted by the Massachusetts network

●

The identification and testing of measures of quality in delivering HIV/AIDS and
sexually transmitted diseases services, conducted by the New York network

Research Capacity and Expansion Series (RACE)

These grants allowed networks to expand existing research studies in order to examine
disparities in the delivery of public health services, incorporate methodological advances,
and/or enhance the diversity of research teams by mentoring investigators from underrepresented backgrounds, such as racial and ethnic minorities, people from low-income
communities or first-generation college graduates.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

7

As of October 2011, the National Coordinating Center had made eight RACE awards
starting in 2011. Examples include:
●

A study of the extent to which local health departments take action to reduce health
inequities and the characteristics of local public health systems that facilitate and
impede such action, conducted by the Minnesota network

●

A project to refine an index that
measures social and economic
determinants of health and to examine
how public health officials use the index
to reduce disparities, conducted by the
Connecticut network

Multi-Network Practice and
Outcome Variation Study (MPROVE)

Minnesota’s PBRN
takes a wide view of its
mission: “We are not
wedded to a single
content area, We are
focused on the
underlying system and
capacity in the
system.”—Kim Gearin,
PhD, research associate

To advance the reach of the networks by
creating research opportunities that involve
several of them in a common study and to
begin to develop evidence about the public
health system in general, the National
Coordinating Center started MPROVE in
2012. “This is the first attempt to get a
number of networks to standardize data
collection efforts across states. This will probably be worth the blood, sweat, and tears it
will take to get it done,” observes Wong.
In the first MPROVE project, six networks—Colorado, Florida, Minnesota, New Jersey,
Tennessee, and Washington—are collaborating to study variation in the delivery of three
core local public health services: communicable disease control, chronic disease
prevention, and environmental health protection. They are collecting data on a common
set of service delivery measures and will pool the data into a common registry and link
them with other sources to support both across-network and within-network analyses.
Snapshots of Two Networks
The Minnesota Public Health Research to Action Network

Minnesota’s public health PBRN is housed in the state Department of Health and guided
by a steering committee made up of representatives of the agency, academia, and local
public health departments, as well as local elected health officials. It takes a wide view of
its mission: “We are not wedded to a single content area,” says Gearin. “We are focused
on the underlying system and capacity in the system.”

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

8

Gearin and Beth Gyllstrom, PhD, another senior research scientist in the Minnesota
Department of Health and a network leader, see benefits in housing the network at the
state health agency, which manages the local health department reporting system. “I just
have to walk upstairs to get data,” says Gyllstrom.
The inaugural research project, conducted as part of the grant to establish Minnesota’s
public health PBRN, examined the local health department demographics, roles,
authority, governance, and structure.5 Findings were reported in the Journal of Health
Practice and Management6 and focused on six key authorities for local health directors.
With a research implementation award from the National Coordinating Center,7
Minnesota’s network has also developed a quality improvement maturity score,8 which
led to a follow-up study funded by a grant from RWJF’s Public Health Services and
Systems Research program to examine whether local health department characteristics are
related to achieving evidence-based policy changes.9 Another study addressed current
Minnesota health department efforts to address health disparities, social determinants of
health, and health inequities10 (RACE award). Minnesota is also one of the six networks
participating in the MPROVE study. As of March 2013, Minnesota had published eight
reports from its studies on its website.
“I think the network is starting to hit its stride. We have a brand, have applied findings to
benefit our public health system, and are gaining visibility—all of which helps us to
sustain the network,” says Gearin.
Read more about Minnesota’s network on its website.
The Ohio Research Association for Public Health Improvement

Ohio’s public health PBRN “is the research voice for local public health departments in
Ohio,” according to program director Frank. Although it is based at Case Western
Reserve University, Frank says, “The key is strong local health department leadership—
5

ID# 67018
Miner Gearin KJ, Thrash AM, Frauendienst R, Myhre J, Gyllstrom ME, Riley WJ and Schroeder J.
“Measuring the Authority of Local Public Health Directors in the Context of Organizational Structure: An
Exploratory, Multimodal Approach.” Journal of Public Health Management and Practice, 18(6): 545–550,
2012. Available at www.rwjf.org/en/research-publications/find-rwjf-research/2012/11/the-journal-ofpublic-health-management---practice-focuses-on-pu/measuring-the-authority-of-local-public-healthdirectors-in-the-.html.
7
ID# 68674
8
Gearin KJ, Gyllstrom ME, Joly BM, Frauendienst RS, Myhre J and Riley W. “Monitoring QI maturity of
public health organizations and systems in Minnesota: Promising early findings and suggested next steps.”
Frontiers in Public Health Services and Systems Research, 2(3), Article 3, 2013. Available at
http://uknowledge.uky.edu/frontiersinphssr/vol2/iss3.
9
ID# 69683
10
ID# 69495
6

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

9

and always going to the partnership with decision-making. The Ohio Department of
Health is an active partner.”
Established with RWJF funding, the Ohio network also received four Quick Strike
Research grants, all driven by local health departments. These studies:
●

Created a model for estimating costs for a standard package of core local public
health services11

●

Examined the financial effects of
consolidating local health departments12

●

Analyzed the agreement between
position descriptions and practice
standards for public health nurses13

●

Examined the causes and consequences
of local variation in public health
enforcement of the state's smoke-free
workplace act14

“This is more than just
‘if there is money, they
will come.’ It’s clear
that practitioners think
this is important, not
just a nice thing to
do.”—National
Advisory Committee
member Paul Halverson

Through a Research Implementation Award
and a Research Acceleration and Expansion
Award15 the Ohio network is analyzing the
role of local health departments in
preventing food-borne illness outbreaks. Trained student observers watched and recorded
more than 500 restaurant inspections conducted by local health departments. “We have a
really close look at what happens during the inspections. That should let us comment on
the value of the inspections and validate what the local health departments are doing to
protect the health of the public,” says Frank.
Read more about Ohio’s network on its website.
HOW ARE THE NETWORKS PROGRESSING?
The public health PBRNs are young but early signs suggest they are making progress.

11

One of two studies funded under ID# 69619
One of two studies funded under ID# 69619
13
ID# 66151
14
ID# 66151
15
ID#s 68673 and 69497
12

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

10

Networks Formed, Engaged Practitioners and Researchers, and
Launched Projects

By April 2013, 28 networks were participating in the RWJF Practice-Based Research
Networks in Public Health program. All the networks were within a single state, and most
were large—three-quarters of them covered the entire state. “This is more than just ‘if
there is money, they will come,” says
advisory committee member Halverson. “It
is clear that practitioners think this is
important, not just a nice thing to do.”
Some 926 local health departments, 20 state
health agencies, and 35 academic units have
been involved in RWJF public health
PBRN-funded research projects to date.
More than 50 studies were underway or
completed, including at least 15 that had
funding from outside of the RWJF national
program.

“We are giving our
federal agency staff
information about what
public health looks like
on the ground. It is
hard for them to see on
the ground, from where
they sit.”—Program
Director Glen Mays

Program Director Mays conducted a social
network analysis16 of the first five public
health PBRNs, which he reports, “reveals
broad engagement of both practitioners and
researchers in scientific inquiry, with practitioners in the periphery of these networks
reporting particularly large benefits from research participation.”17
“The networks are stimulating both young and more senior researchers,” adds
NACCHO’s Pestronk.
An Ohio Project Attracts Other Networks

Ohio’s network started an unfunded study using an online survey of Ohio health
departments to examine the future of teaching in public health given the economic
downturn. Frank announced the survey and invited other PBRNs to use the instrument
during a National Coordinating Center conference call. Two other networks, North
Carolina and Wisconsin, joined the study, which is still underway as of March 2013.
Findings will be available in aggregate and for each network individually. With an
Social network analysis “measures relationships between individuals and groups by mapping these
relationships and assessing their patterns. The resulting map provides a unique picture of how network
participants are communicating and behaving,” according to a January 2011 overview by RWJF.
17
Mays GP and Hogg RA. “Expanding delivery system research in public health settings: lessons from
practice-based research networks.” Journal of Public Health Management and Practice, 18(6): 485–498,
2012. PMID:23023272.
16

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

11

increase in enrollment in academic public health programs coinciding with a decrease in
health department resources, it is unclear where the essential, practical, community-based
training will take place in the future. This work informs that decision-making process.
Policy-Makers and Funders are Interested in Network Research

Federal agencies including the White House Office of Management and Budget, the
Congressional Budget Office, the Department of Health and Human Services’ Assistant
Secretary of Health and the Assistant Secretary for Preparedness and Response, and the
director of the CDC’s National Center for Injury Prevention and Control, have turned to
Mays and network leaders for briefings about their work.
Mays and Ohio’s Frank met with officials at the Food and Drug Administration (FDA), a
meeting facilitated by Katherine Papa, MPH, program director at AcademyHealth.18 Papa
works with Mays to link the networks and federal policy-makers.
The FDA was interested in learning about Ohio’s study of food-borne illness outbreaks.
“It was really affirming to share our findings with the deputy director of the FDA,” says
Frank.
“We are giving our federal agency staff information about what public health looks like
on the ground. It is hard for them to see on the ground, from where they sit,” Mays adds.
RWJF’s Wong sees “increasing demand for the work coming out of the networks” from
key policy-makers. “This is not application of evidence within the networks, but beyond
them,” she says.
Other funders are starting to take note. For example, in 2009, the CDC’s Preparedness
and Emergency Response Research Center and the Pandemic Influenza Planning and
Preparedness Program provided additional funding for the H1N1 research started with the
Quick Strike funding.
States Have Started to Change Practice

Network research has prompted action in some states. One hot topic is whether some
local health departments should consolidate into larger, more regional entities.
“Massachusetts is consolidating and regionalizing its public health,” says Mays. “Ohio is
also making some decisions about consolidating some rural public health departments.”
North Carolina and Kentucky’s Quick Strike studies of variations in local health
department responses to the 2009 H1N1 influenza outbreak19 enabled them to improve
18

AcademyHealth is a Washington-based organization that supports development and use of rigorous,
relevant, and timely evidence to improve health.
19
Funded under grant ID# 64676

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

12

their responses during the outbreak through better communication between medical
providers, public health officials, and others.
Ohio’s Quick Strike analysis of enforcement of the state’s Smoke Free Workplace Act,
described earlier, contributed to the decision “to retain funding for enforcement, when
people were looking to take the program away,” says Frank.
Minnesota's Public Health PBRN Studies Inform a State Program and
Help Secure Federal Funds

Minnesota is using findings from its network research, described earlier, in many
different ways. For example, staff from the Department of Health used findings from the
research on quality improvement measures to enhance the statewide public health
performance measurement system.
In a successful grant proposal for nearly $2 million submitted in 2010 as part of CDC’s
Strengthening Public Health Infrastructure for Improved Health Outcomes program,
Minnesota cited findings from its initial network research, and emphasized the potential
to enhance infrastructure development activities with independent practice-based research
conducted by the Minnesota PBRN.20 With this grant, Minnesota is promoting increased
use of performance management, quality improvement, and national public health
standards to promote a culture of quality in the
state and local health departments.
Networks Are Spreading Their
Findings

The findings from many studies have been
published in peer-reviewed journals. One
article, “Evidence Links Increases in Public
Health Spending to Declines in Preventable
Deaths,21” published in Health Affairs appeared
as third on the list of RWJF’s Most Influential
Research Articles of 2011, and as one of
AcademyHealth’s five Most Outstanding

“The value of the
PBRNs is that these
researchers can talk
about the research in
terms of policy.”—
Katherine Papa, MPH,
program director at
AcademyHealth

Strengthening Public Health Infrastructure for Improved Health Outcomes program is part of CDC’s
National Public Health Improvement Initiative, which supports health departments to make fundamental
changes and enhancements in their organizations and implement practices that improve the delivery and
impact of public health services.
21
Mays GP and Smith SA. "Evidence Links Increases in Public Health Spending to Declines in Preventable
Deaths," Health Affairs, 30(8): 1585–1593, August 2011. Available at www.rwjf.org/en/researchpublications/find-rwjf-research/2011/04/april-issue-of-health-affairs-focuses-on-patient-safety-andheal/evidence-links-increases-in-public-health-spending-to-declines-i.html.
20

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

13

Articles in the field.
Mays and his colleagues developed and edited two special journal issues published in
2012 on research conducted by the networks: one of the Journal of Health Management
and Practice,22 and one of the American Journal of Preventive Medicine.23
To accelerate the movement of public
health research into practice and policy,
in 2012, Mays launched Frontiers in
Public Health Services and Systems
Research, an online peer-reviewed
journal that features descriptions of
preliminary findings from empirical
studies or quality improvement projects.
Abstracts of articles also appear in a
special section of the American Journal
of Preventive Medicine.
Getting the Messages to
Practitioners and Policy-Makers

“Mays and the networks
have given attention to
communications channels
and multiple ways of
communicating—they get
information to a variety of
forums to get people in
public health to hear what
we have discovered.”—
Advisory Committee
member Paul Halverson

Getting research findings into the hands
of public health practitioners and policymakers—who do not generally read
peer-reviewed journal articles—is
essential to improving public health practice. “Mays and the networks have given
attention to communications channels and multiple ways of communicating—they get
information to a variety of forums to get people in public health to hear what we have
discovered,” says Halverson.
Mays’ connections with NACCHO provides an example of how network research gets
into the hands of public health officials—in this case representatives of 2,700 local health
departments—more broadly. NACCHO’s leaders encourage network researchers to
attend and present at the annual meeting. The Winter 2012 edition of NACCHO’s
quarterly newsletter was devoted to public health PRBNs, and network directors also can
submit summaries of network research for other NACCHO publications.
22

Mays GP and Scutchfield FD (eds.). "Advances in Public Health Services and Systems Research."
Journal of Public Health Management and Practice, 18:6, November–December 2012. Available at
www.rwjf.org/en/research-publications/find-rwjf-research/2012/11/the-journal-of-public-healthmanagement---practice-focuses-on-pu.html.
23
Scutchfield FED, Howard AF, Perez DJ, Monroe JA and Mays GP (eds.). "An Agenda for Public Health
Services and Systems Research." American Journal of Preventive Medicine, 4(5:Suppl 1), 2012. Available
at www.rwjf.org/en/research-publications/find-rwjf-research/2012/05/a-national-research-agenda-forpublic-health-services-and-system.html. Click Browse Contents to see all of the articles.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

14

Mays’ connections with Papa and AcademyHealth show how network research gets
shared with health researchers and policy-makers at the federal level. Papa taps into her
network of “Washington policy-makers and policy influencers” to find potential
champions for the networks’ research and to connect networks with policy-makers. “The
value of the PBRNs is that these researchers can talk about the research in terms of
policy,” she says.
AcademyHealth’s public health systems
research group is another way to share
public health PBRN findings with health
researchers and policy-makers. With
more than 2,700 members, it is the
largest of 16 interest groups of service
providers and researchers.
WHAT CHALLENGES IS THE
PROGRAM FACING?
Too Many Priorities, Too Little
Time

“The elephant in the room
is that there needs to be a
serious commitment made
at a federal level to
funding public health
systems and services
research.”—Advisory
Committee member Paul
Halverson

A significant challenge for the networks
and the National Coordinating Center is
moving research findings into mainstream public health practice and policy. “I think the
biggest enemy that people in public health have is time,” says Halverson. “We have to
change the pattern so practitioners understand the need to engage in what is going on
with the networks and practice research.”
Time is certainly a challenge in Minnesota. “There is never enough time for us and for
our partners. It is always a challenge to balance our shared research agenda with the dayto-day demands of running a local health department,” says Gearin.
Mays shares Halverson’s concern. “An evolving challenge involves the growing set of
competing priorities faced by the networks and the relatively limited capacity of networks
to address multiple priorities within existing resources.” The National Coordinating
Center’s technical assistance efforts include helping networks establish priorities and
sequence their activities.
NACCHO’s Pestronk also identifies obstacles in translating research in ways that address
the multiple priorities of public health agencies. “What those who make policy want isn’t
necessarily the science that research produces. The challenge is to produce both science
and stories written in a way that is useful to those who aren’t scientists and who tend to
practice or base policy on what they hear from their neighbors and friends.”

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

15

Getting Researchers to See the Value

Another challenge is “engaging the right researchers—people who can bring their
expertise and who are willing to share control, authority, and money with public health
practitioners, and not just call the shots themselves,” says Mays. “I do a lot of travel to
the sites and do a lot of work with academics about how doing this work can advance
their academic careers.”
Mays finds this an ongoing challenge
partly because, although researchers
rely on grant funds, the funds from the
National Coordinating Center are
relatively small, and they don’t have the
prestige of National Institutes of Health
grants.
Halverson agrees. “I see a real shortage
of qualified research faculty,” he said.
Sustaining the Networks

“We are building an
enormous level of trust and
engagement with the
federal health enterprise….
We are trying to get the
federal agencies to fund
networks, so first we show
them how networks help,
and then we ask them for
money.”—Glen Mays,
program director

Funding for this emerging field is
another ongoing challenge. “There is
more need than there is money,” says
Halverson, who calls for increased
federal involvement in funding this research. “The elephant in the room is that there
needs to be a serious commitment made at a federal level to funding public health
systems and services research.”
Sustaining the networks will take time in addition to money. “We know that despite even
big infusions of funding, it takes from 15 to 20 years to move from the clinical bench to
clinical practice,” says NACCHO’s Pestronk. “That is not a reason not to do this work,
but it is a challenge.”
Sustaining the networks administratively is also a challenge, says Mays. The networks
have grown so fast that the National Coordinating Center is having trouble keeping up
with their needs. “We want to help them, but there are just a couple of us here behind the
curtain,” says Mays.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

16

WHAT DOES THE FUTURE HOLD?
Proposed Activities

Going forward, the National Coordinating Center is focusing on cultivating new research
partners and sources of support, and building research capacity. Two areas of focus are
two priorities in the Affordable Care Act: comparative effectiveness research24 and
delivery system research that quantifies the health and economic value of different public
health strategies. Mays is hopeful that federal agencies will fund some of this research.
The center plans to award new public health delivery and cost studies grants to generate
evidence about the effectiveness and costs of core public health services and delivery
system strategies. It has also designed new grants for public health measurement and
comparison studies to develop new methods for measuring the reach, quality, and cost of
core public health services. Also, the Quick Strike, Research Acceleration and
Expansion, and MPROVE research grants will continue.
Technical assistance is being expanded through an updated website, an electronic
discussion group using social networking platforms, and PBRN Charrettes—focused,
facilitated guidance sessions to address a research or operational challenge.
Mays is also evaluating the overall public health PBRN model in facilitating research
translation, adoption, and implementation—including repeating the social network
analysis survey annually starting in 2013. To determine what would have happened
without the public health PBRNs, he is working with NACCHO to gather data to allow
comparisons between health departments that are and are not part of public health
PBRNs. The initial round of data from 2010 shows that local public health agencies
participating in PBRNs were two to three times more likely than non-participating
agencies to engage in research implementation and translation activities.25
Finally, he is collaborating with investigators from the University of Kentucky’s
Dissemination and Implementation Sciences Consortium26 to determine the best way to
communicate findings from network research.

The Institute of Medicine defines comparative effectiveness research as “the generation and synthesis of
evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and
monitor a clinical condition or to improve the delivery of care.”
25
Mays GP, Hogg RA, Castellanos-Cruz DM, Hoover AG, and Fowler LC. “Engaging Public Health
Settings in Research Implementation and Translation Activities: Evidence from Practice-Based Research
Networks.” American Journal of Preventive Medicine; 2013; in press.
26
The consortium conducts evidence-based research that accelerates the uptake of research findings into
actionable practice.
24

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

17

The Long View

These activities feed into Mays’ long-term vision for the public health PBRNs. “I think
we are building an enormous level of trust and engagement with the federal health
enterprise, that I am hopeful will pay off. We are trying to get the federal agencies to
fund networks, so first we show them how networks help, and then we ask them for
money.”
Prepared by: Mary Nakashian
Reviewed by: Lori De Milto and Molly McKaughan
Program Officer: Naima Wong
Program Area: Public Health
Grant ID#: PHPB
Program Director: Glen P. Mays, PhD, (859) 218-2029; [email protected]

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

18

APPENDIX
People Interviewed for This Report
Scott Frank, MD, MS
Director, Ohio Practice-Based Research
Network/Public Health
Case Western Reserve University
Cleveland, Ohio
Lori Tremmel Freeman, MBA
Associate Executive Director
National Association of County and City
Health Officials
Washington, D.C.
Kim Gearin, PhD
Senior Research Scientist
Minnesota Department of Health
St. Paul, Minn.
Beth Gyllstrom, PhD
Senior Research Scientist
Minnesota Department of Health
St. Paul, Minn.
Paul K. Halverson, DrPH, FACHE
Director
Arkansas Department of Public Health
Little Rock, Ark.
Carolyn Leep, MS, MPH
Senior Director of Research & Evaluation
National Association of County and City
Health Officials
Washington, D.C.

Glen P. Mays, PHD
Professor of Health Services and Systems
Research
National Program Director: Public Health
Practice-Based Research
College of Public Health, University of
Kentucky
Lexington, Ky.
Katherine Froebe Papa, MPH
Program Director
AcademyHealth
Washington, D.C.
Robert Pestronk, MPH
Executive Director
National Association of County and City
Health Officials
Washington, D.C.
Timothy W. VanWave, DrPH, MPH
Associate Director for Science
Office of State, Tribal, Local, and Territorial
Support
Centers for Disease Control and Prevention
Atlanta, Ga.
Naima T. Wong, PhD
Program Officer
Robert Wood Johnson Foundation
Princeton, N.J.

RWJF Progress Report—Robert Wood Johnson Foundation Practice-Based Research Networks in Public Health

19