Justification Memo

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

January 31, 2008

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Questionnaire Design Research Laboratory (QDRL) plans to conduct research to test and evaluate health surveillance maps for the Division of Adult and Community Health for the National Center for Chronic Disease Prevention and Health Promotion under (NCCDPHP)/CDC (OMB No. 0920-0222, exp. 02/28/10). We propose to advertise for volunteer participants starting March 10th, and to start testing as soon as possible after that.

Background Information about Cognitive Testing

Cognitive interviewing was developed as a method for evaluating survey questions, but has been successfully used to evaluate any materials that communicate information to users.

Because the interviews generate narrative responses rather than statistics, results are analyzed using qualitative methodologies. This type of in-depth analysis reveals problems, and can help to improve the overall quality of the materials.

Proposed project: Evaluation of Maps for Presentation of Health Data

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. The purpose of cognitive testing is not to obtain survey data, but rather to obtain information about the processes people use to when reading health surveillance maps as well as to identify any potential problems in the formatting of the maps.

The Behavioral Risk Factors Surveillance System was first implemented in 1984 to collect information on risk behaviors and preventive health behaviors that affect the health of the U.S. population. BRFSS data is often displayed using maps. However, little is known about how different map formats (choropleth versus isopleth maps) affect the ability to extract information from the maps. The purpose of this test is to investigate the effectiveness of presenting BRFSS data in two different formats, isopleth versus choropleth maps. We will assess various cognitive tasks map users engage in when attempting to retrieve data from the maps. A questionnaire will be used as a tool for assessing these tasks. The testing instrument is shown in Attachment 1. Cognitive testing will primarily focus on map comprehension and map user’s preference of map layout.

This project will conform to the usual QDRL procedures for cognitive testing. We propose to recruit 30 public health professionals and epidemiologists who have at least a bachelor’s degree. Our sample will consist of epidemiologists and other public health professionals who would be typical users of health surveillance maps. A combination of advertisements, flyers, advance letters, and an NCHS email announcement (if needed) will be used to recruit participants (see Attachment 2). Interviews will be conducted in the QDRL, or in a private room of a University, or government agency.

With the consent of the participants, the interviews will be recorded on audio or videotape. Participants will be informed of taping procedures (including observation if applicable) in the process of reviewing the consent forms, and the equipment will be turned on once it is clear that the procedures are understood and agreed upon. Then the interviewer will conduct individual cognitive interviews with as many as 30 participants for 60 minutes each. At the end of the interviews, participants will be paid and provided with copies of all papers they signed.

We propose paying participants $75. Our approved base incentive is $40. We have found in past experience that we need to offer expert professional subjects more than the $40 base incentive. For example, in 1995 we offered a $75 incentive to nurses, farmers, truck drivers, auto mechanics, and dry cleaners to participate in a study on chemical exposures in the workplace. In 2000, we offered a $75 incentive to medical providers to participate in a study on childhood immunization. When we have not significantly increased the incentive for expert professional subjects we have experienced low participation, and have failed to meet our project goals. For example, in 2000 we offered a $50 incentive to epidemiologists and public health professionals to evaluate the cognitive aspects of disseminating statistical data in a publication. The $50 incentive proved to be insufficient in attracting a large and diverse pool of expert subjects. Due to the low response rate, the number of actual interviews conducted was well below our target goal, making the results difficult to analyze. We hope that the increased incentive will be sufficient to entice these busy highly paid professionals to take time out of their schedules to participate in our study. NCHS employees participating in the study will not receive an incentive.

In total, for this project, the maximum respondent burden will be 30 hours of interviewing in addition to travel time. An updated burden table for this project is shown below:

Projects	Number of Participants	Number of Responses/ Participant	Average hours per response	Response burden
QDRL Interviews
2. Other questionnaire testing	30	1	1	30

Attachments (2)

cc:

M. Moien

M. Daneshvar

S. Perryman

3