HRSA memo 8-13-09

12 August 2009
TO:

Karen Matsuoka

FROM:

Amanda Cash

SUBJECT:

Patient Navigator Demonstration Program (PNDP) – Response to Comments

OMB Comment: For example, it seems like the grantees are basically being allowed to evaluate
themselves and set their own goals and assess the progress they’ve made. The usual checks that
OMB would make (e.g. sampling, non-response bias, assurances of confidentiality, etc.) are
essentially being bypassed because we have nothing to evaluate.
HRSA Response: Because of the nature of this demonstration program, each site has determined
its own baseline and follow-up measures. The abstracts provided outline what each site will be
collecting in terms of disease/condition-specific items at baseline and for the life of the program
to accomplish their particular goals and objectives.
OMB Comment: We’re also concerned about the use of the information to be gained from this
demo, given the exploratory nature of this study (i.e. we are generally not comfortable letting
exploratory studies be the foundation for future policy decisions; rather, we usually expect that
exploratory studies will be followed up by further rounds of more vigorous confirmatory studies
before policy decisions are made (e.g. about the effectiveness of these navigator projects, the
level of funding they should be provided, etc.).
HRSA Response: The program does not intend to use the results of this pilot for any policy
decisions. The program will share the report to Congress with OMB to ensure the language is
appropriate and the limitations of the study are appropriately addressed, including the fact that
there are no control groups.
OMB Comment: These are our 2 main concerns. Here’s how I’d propose moving forward in
light of these concerns.
1. Utility and rigor of evaluations –

We need more information about each of the 6 grantees and the evaluation plan they
have provided to HRSA (i.e. what is the goal of each grantee’s PN program, a
description of their PN program, and how do they plan to evaluate how effective their
program is). To the extent that they are assessing their programs on the criteria we
would normally assess studies on in Part A and B of supporting statements (e.g.
sampling, non-response bias, etc.), we would need to know how the grantees plan to
evaluate their studies on those criteria. In other words, if they had to complete a
supporting statement Part A and B for their individual sites, what would they say?

We would like more information on the benchmarks being used as part of this study.
Please provide the studies on which the benchmarks were determined. Given that the
grantees are allowed to determine their own goals and outcomes for the rest of the

1

evaluation, it seems a bit incongruous that the study would institute a firm benchmark
against which to assess all the sites.
HRSA Response: The abstracts for each grantee are attached with this memo for your review.
As discussed on the phone, the grantees are creating their own “benchmarks” for these studies
since the patient navigators have not been a commonly used method for improving health care
delivery. The benchmarks each site is using are actually baseline measures of current practice. If
OMB would like further information, HRSA will provide it.





2. Use of the information/study results –
We appreciate that HRSA has clarified the exploratory and descriptive nature of this
study. However, this seems to be contradicted by statements like “the evaluation ...
should provide guidance for development of future patient navigator programs.” This
seems to suggest that the next step is policy development rather than further study of
patient navigator programs. We’d like this type of language to be struck from all of the
documents (e.g. supporting statement) and we’d like language inserted that acknowledges
that the next step is further study, e.g. through confirmatory studies utilizing more
experimental study designs.
We’d like HRSA to provide a list of all the limitations of this study, and we’d also like
HRSA to certify that all of these study limitations will be acknowledged in any public
disclosure of these study results (e.g. in journal articles and in the Report to Congress).
Also, we’d like a briefing on the study results and we would like to see a copy of the
Report to Congress before it is submitted to Congress.

HRSA Response: HRSA will amend the language in the supporting statement to reflect OMB’s
language. HRSA will work with its contractor to provide a list of limitations of the study and
also acknowledge those limitations in public disclosure of the study results. HRSA will also brief
OMB on the study results and provide a copy of the Report to Congress before it is submitted to
Congress.

2