SupStmt_B_RevBurden_ClientOp2012_08_28_2012_toHRSA - REVISED

Supporting Statement B

Evaluation of the Patient Navigator Outreach and Chronic Disease Prevention Demonstration Program

OMB Control No. 0915-0346 REVISED

2. Collection of Information Employing Statistical Methods

This is a request for Office of Management and Budget (OMB) approval to conduct an additional data collection for the Evaluation of the Patient Navigator Outreach and Chronic Disease Prevention Demonstration Program (PNDP), previously approved under OMB number 0915-0346.

PNDP is a quality improvement initiative focused on improving access to interventions that prevent and treat chronic diseases and cancer. Client opinion data will be used at a local and a HRSA program level to examine whether clients perceptions of navigator efforts were helpful to clients in domains related to duties of the navigator specified by Congress.

All data collection methods and analyses build upon previous experience with FY 2008 Patient Navigator Outreach and Chronic Disease Prevention Demonstration Program (OMB number 200903-0915-003).

1. Respondent Universe and Sampling Methods

As part of ongoing quality improvement activities, client opinion data will be collected from the first 100 consecutive patients seen in the navigation program after a target date (possibly July 1) at each of the nine PNDP sites currently collecting data. Previously approved data collection involves the completion of a health status questionnaire at the end of the grant program, and we will administer the Client Opinion Survey at the same time for 100 clients. We anticipate that 900 clients will be asked to provide opinion data.

This sampling strategy was chosen because 100 consecutive clients with navigators near the end of grant funding will represent a reasonable sample of client opinions. Some clients will have achieved many or most goals of the navigator program, and others will still be in progress with respect to these goals. Thus the sampling strategy is designed to obtain feedback about clients’ perceptions of the program services regardless of how well the client has progressed toward program goals. In order to make program improvements, HRSA and grantees need to hear from clients who may be less pleased with their progress.

While in theory a random sample may appear to be more appealing, in practice it is difficult to implement consistently. Using the recommended sampling strategy, sites will be able to integrate the Client Opinion Survey procedures with other navigator procedures for all clients seen over a fixed period of time. In contrast, training navigators to implement data collection for some randomly selected clients rather than others will be more difficult to implement, resulting in a lower response rate and possible bias.

2. Procedures for the Collection of Information

Starting July 1 or some other target date immediately after OMB approval, the Client Opinion Survey will be administered by PNDP staff who have not provided navigation services to the client. Clients who have been navigated only for cancer will be asked to complete the Client Opinion Survey – Navigation to Cancer Services; clients who have been navigated for a range of chronic disorders excluding or including cancer will receive the Client Opinion Survey – Navigation to Chronic Disease Prevention and Early Intervention.

The surveys are quite similar. However, navigation for cancer occurs primarily in secondary care settings, with a more focused intervention than navigation for early identification and treatment of chronic disease. The latter is typically implemented in primary care, and involves navigation to a broad range of interventions and services. In order to ensure that clients consider only questions that are relevant to their experience, two different versions of the survey are available.

The Client Opinion Survey may be administered by the staff member by telephone or in-person; where possible, the survey will be self-administered. This may not be possible in all cases because of limitations related to literacy. It is in a short, easy-to-complete format and is expected to take about seven minutes to complete. It will be administered in conjunction with another survey that has been approved by OMB (OMB Control #0915-0346 Exp. date 1/31/2015). Up to four follow-up calls will be conducted to ensure completion of the survey.

Results may initially be collected on the paper form in Appendix A, and then entered into the PNDP electronic database along with other evaluation data previously approved by OMB. (OMB Control #0915-0346 Exp. date 1/31/2015). Alternatively, data may be entered directly into a form in the database, if the navigator has access to a computer during the client visit. Grantees consider the PNDP database easy to work with and user-friendly.

At the end of the data analysis, the data will be aggregated with other evaluation data and transferred into a standard statistical package for analysis (SPSS, STATA). Open-ended responses will be examined and coded.

Primary analyses will be descriptive in nature, and will answer questions about client perceptions of services received, whether the client perceived that navigation related to specific duties identified by Congress was helpful, and whether the experience in general was positive. Sample size is adequate to meet these basic objectives.

In terms of exploratory analyses to identify factors related to low satisfaction with navigation, the sample size needed to detect a statistically significant difference between five percent of one group and ten percent of another group is 435 cases, given p=.05 and power=.80. We expect to have at least 400 cases in groups with the following conditions: diabetes, cardiovascular disease, asthma, cancer, and cancer risk. In our expected sample of over 4,200 cases, we estimate at least 400 cases will have characteristics of interest such as multiple comorbid conditions or low mental health scores, or be navigated by nurses. Thus, sample size is sufficient to conduct rudimentary exploratory analyses comparing proportions.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Every attempt will be made to collect information from the respondent at the last face-to-face encounter. In addition, up to four follow up calls will be made to complete the survey by telephone. Response rates for the satisfaction survey will be maximized by means of the participants’ interest in supporting effective navigator program services. Possible bias due to nonresponse will be explored by comparing the characteristics of those who respond to the survey with those who did not respond.

4. Tests of Procedures or Methods to be Undertaken

The Client Opinion Form survey used CAHPS® surveys as starting points for this modified instrument. The Consumer Assessment of Healthcare Providers and Systems (CAHPS) program is a multi-year initiative of the Agency for Healthcare Research and Quality (AHRQ) to support and promote the assessment of consumers' experiences with health care. In addition to the Agency for Healthcare Research and Quality, several other Federal agencies contribute to the CAHPS program. The Centers for Medicare & Medicaid Services (CMS) has been a major partner in the CAHPS program since 1996. The Client Opinion Form questions and response item sets are modified from the CAHPS suite of questionnaires, primarily the CAHPS® Clinician & Group Surveys with Patient-Centered Medical Home (PCMH) Items, the CAHPS® Item Set for Addressing Health Literacy, and the CAHPS® Cultural Competence Item Set.

Since 1995, the Agency for Healthcare Research and Quality (AHRQ) has been creating a family of client experience surveys under the CAHPS^® trademark.^[1] Because each survey is the result of a systematic and thorough research and development process, including opportunities for public input, the CAHPS brand has come to signify a high level of scientific rigor, reliability, and credibility. All CAHPS surveys go through a similar development process that involves multiple steps. This process is designed to gather and apply input from relevant stakeholders and to ensure the reliability and usefulness of survey results. The development of reliable and valid survey items and procedures stems from

• Cognitive interviews with clients

• Revisions to reflect findings from cognitive testing

• Field testing and psychometric analysis of field test results

• Revisions to reflect findings from field tests

Information gained from the CAHPS experience has allowed the contractor to have confidence in the quality of the data collection instrument.

In addition, the draft instrument was reviewed by patient navigators from a range of health disparities communities, and their input was incorporated into the final instrument.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Individuals involved in data collection design and analysis:

Government Project Officer:

HRSA/BHPr
Alexis Bakos, PhD, MPH, RN
Deputy Director Division of Nursing
301-443-5688
[email protected]

NOVA Research Company, Evaluation Contractor
Paul A. Young, M.B.A., M.P.H.
Executive Vice President/Senior Program Director
240-483-4190

Caroline McLeod, PhD
Senior Evaluation Researcher
240-483-4191
[email protected]

Carmen-Anita Signes, BS
Data Manager
301-986-1891
[email protected]

Debra Stark, MBA
Research Associate
240-752-7337
[email protected]

Individuals responsible for collecting data:

Clinica Sierra Vista
Bill Phelps
PNDP Project Director
661-635-3050 x2156
[email protected]

New River Health Association
Tammy Campbell-Cline
PNDP Project Director
304-469-2906 x1364
[email protected]

Project Concern International
Maria Reyes
PNDP Project Director
619-791-2610 x305
[email protected]

The Queen’s Medical Center
Debbie Ishihara-Wong
PNDP Project Director
808-537-7574
[email protected]

Ravenswood Family Health Center
Belinda Hernandez
PNDP Project Director
650-330-7449
[email protected]

Texas Tech University
Christina Esperat
PNDP Project Director
806-743-2736
[email protected]

University of Utah
Randall Rupper
PNDP Project Director
801-587-3410
[email protected]

Vista Community Clinic
Connie Krisman
PNDP Project Director
760-631-5000 x1262
[email protected]

William F. Ryan Community Health Center
Nancy Andino
PNDP Project Director
212-316-8367
[email protected]

REFERENCES

Crofton C, Darby C, Farquhar M, Clancy CM. The CAHPS Hospital Survey: development, testing, and use. Jt Comm J Qual Patient Saf. 2005 Nov;31(11):655-9, 601.

1 Originally, CAHPS referred to AHRQ’s Consumer Assessment of Health Plans Study, but since 2005, the acronym stands for Consumer Assessment of Healthcare Providers and Systems.

5