Letter to Dr. Schwab

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

April 18, 2016

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Center for Questionnaire Design and Evaluation Research (CQDER) (OMB No. 0920-0222, exp. 07/31/2018) plans to conduct an evaluation of scorecard items on the forthcoming 2016 CDC Healthcare Systems ScoreCard (HSSC) Assessment Tool for Primary Care Practices. The purpose of the HSSC will be to assess the extent to which health systems have implemented evidence-based policies, practices, and systems to guide the delivery of care for adult patients with high blood pressure, high cholesterol, obesity, diabetes, cancer, and who use tobacco.

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. As you know, the purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: 2016 CDC Healthcare Systems ScoreCard (HSSC) Assessment Tool for Primary Care Practices

The HSSC is developed by CDCs Division for Heart Disease and Stroke Prevention (Applied Research and Evaluation Branch) in collaboration with a workgroup from CDC National Center for Chronic Disease Prevention and Health Promotion and external partners and stakeholders, such as state health department representatives and the National Association of Chronic Disease Directors. (The HSSC was previously tested in a sample of 9 respondents.) Using a scorecard approach, this quality improvement assessment tool will help CDC-funded state and local public health programs to better understand the implementation of evidence-based policies and protocols in health systems and to assess the current state of primary care management for adult patients with high blood pressure, high cholesterol, obesity, diabetes, cancer, and who use tobacco. It will also assist health systems to identify possible gaps and prioritize strategies with the highest impact to better manage chronic disease conditions. The HSSC consists of approximately 70 weighted binary questions about evidence-based health system policies that will be completed by ambulatory facilities to assess their own policies. Domains of inquiry include policies that support use of multi-disciplinary care teams, clinical decision supports, clinical guidelines, electronic functions for managing patient health and follow-up, and patient self-management.

The HSSC items we are evaluating are enclosed in Attachment 1. The Applied Research and Evaluation Branch (AREB) in the Division for Heart Disease and Stroke Prevention (DHDSP) decided that a thorough evaluation of the scorecard is necessary to improve the validity of the tool, reduce response variability, and streamline and improve the functionality of the tool. Improvements informed by cognitive testing will strengthen the scientific integrity of the HSSC and users in the public health and health care fields will deem the HSSC as a credible tool. At CDC, public health and health care system collaboration, and quality improvement are priority strategies for the prevention and management of chronic disease. Additionally, CDC aims to provide validated tools and resources to its funded partners in a timely manner as part of its technical assistance. The HSSC helps to achieve all these goals.

This cognitive interviewing project will evaluate the current form of the scorecard items, and any suggested modifications that emerge from the analysis of the cognitive interviews may be added to the forthcoming 2016 HSSC. This evaluation project has been scheduled to meet DHDSP AREB production deadlines for 2016. This evaluation will focus on healthcare administrators in ambulatory care facilities. While the evaluation will focus on different types of healthcare administrators of small to medium size health systems located in diverse settings, they will all receive the same scorecard instrument. The testing procedure conforms to the cognitive interviewing techniques that have been described in CQDER’s generic OMB clearance package (No. 0920-0222, exp. 07/31/2018).

As many as twenty 60-minute cognitive interviews may be conducted with healthcare administrators who work primarily in primary care practices. Throughout the project, the CQDER will brief DHDSP AREB on the progress and findings of the interviews, and small changes to the scorecard instrument may be incorporated based on these discussions. As the HSSC primary collection mode is self-report mail-back, the cognitive interviews will be primarily conducted using retrospective probing. The respondents will be presented with the scorecard and will be observed by the cognitive interviewer during the administration in order to note any clear patterns of interpretation and usability issues. Following the completion of the scorecard, the cognitive interviewer will retrospectively probe the respondent following the normal form outlined in our generic clearance.

Healthcare administrators in the sample area will be recruited from a list developed by the CQDER based on internet searches, site visits, word-of-mouth, and by contacting participants from past CQDER projects. An advanced letter (see Attachment 2) will be sent to organizations from this frame, indicating that a CQDER staff member will contact them within a week. Individuals will be told that their participation in the study is entirely voluntary. There will be no coercion. The 5 minute telephone screener to be used for verifying eligibility of the sampled healthcare administrator is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. Though our goal is to conduct 60-minute full-length interviews, if during recruitment individuals repeatedly express willingness for a shorter interview, we may conduct shorter interviews in lieu of no interview at all. It is anticipated that as many as 72 healthcare administrators may need to be screened in order to recruit 20 respondents.

Cognitive interviews will be conducted by CQDER staff members in a private room in the healthcare administrator’s office or a mutually agreeable location. Interviews will be audio recorded. These recordings will allow researchers to insure the quality of their interview notes.

After respondents have been briefed on the purpose of the study and the procedures that CQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5), which has been modified for this particular establishment-based project. This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

After the interview, respondents will be given the thank-you letter signed by Charles J. Rothwell, Director, National Center for Health Statistics (Attachment 6) which has been modified for this particular establishment-based project. This document is contained in our umbrella package. The respondent will also receive a copy of the informed consent form, and the cash incentive for participation. Then the respondent will be asked to read the Special Consent for Expanded Use of Audio Recordings (Attachment 7). There will be no coercion and the respondent will be told that they can call and reverse the decision at any time if they change their mind. If respondents do sign the special consent form they will be given a copy of that as well.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

We propose giving respondents a $100 incentive. This amount has been increased over and above the normal cognitive interview $40 incentive level for a number of reasons. First, the recruitment of healthcare administrators is necessary for the success of the cognitive study. Second, given funding and time constraints cognitive interviews will be limited to the Washington DC/Baltimore metropolitan area. Lastly, we will be asking the respondents to participate in the cognitive interview during their working hours, placing an extra burden on these respondents. Our proposed incentive level will be invaluable to obtaining a high response rate and reducing the number of cancelations from this busy, specialized population.

In total, for this project, the maximum respondent burden will be 26 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener	72	1	5/60	6
Questionnaire	20	1	60/60	20

Attachments (7)

cc:

V. Buie

T. Richardson

DHHS RCO

4 | Page