Justification

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

April 15, 2009

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Questionnaire Design Research Laboratory (QDRL) plans to conduct research to evaluate the quality of the collection of States’ revised Birth Certificate Data using focus groups and cognitive interviews for the Division of Vital Statistics (DVS) under (OMB No. 0920-0222, exp. 02/28/10). We propose to start testing as soon as we receive clearance.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. The purpose of cognitive testing is not to obtain survey data, but rather to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions.

Data collection procedures for cognitive interviewing are quite different from survey interviewing. While survey interviewers strictly adhere to scripted questionnaires, cognitive interviewers use survey questions as starting points to begin a more detailed discussion of questions themselves: how participants interpret key concepts, their ability to recall the requested information, and the appropriateness of response categories. Because the interviews generate narrative responses rather than statistics, results are analyzed using qualitative methodologies. This type of in-depth analysis reveals problems in particular survey questions and, as a result, can help to improve the overall quality of surveys.

Although we rely on focus groups less commonly than individual cognitive interviews, we have found that group discussions can generate useful information about experiences and perceptions that are particularly useful in the early stages of questionnaire development.

Proposed project: Testing and evaluation of the collection of data based on the U.S. Standard Certificate of Live Birth in Washington State

This project will conform to the usual QDRL procedures for cognitive testing of a questionnaire.

Periodically, all vital statistics forms (birth, death, and fetal death) and the entire process involved to complete them are analyzed for possible revision. The most recent analysis culminated in the 2003 Revision, with a new national standard certificate of live birth, recommended sources of data and related worksheets to be used to complete the certificate. It also brought about new electronic methods for completing data collection at the place of birth which is then transferred to the State vital statistic office. As of 2008, twenty-eight states had implemented this revision.

Extensive quality control procedures are in place for birth certificate data at both the state and national level. In Washington State, for example birth certificate data are reviewed and corrected through edits provided by the Electronic Birth Certificate (EBC) program and the program which uploads the data at the Center for Health Statistics (CHS).  Once the data have been uploaded, CHS does quarterly and year end checks comparing the data to previous years’ data and also looking for consistency between items.  CHS provides feedback to hospitals through an online system where they can check the completeness for their hospital for selected items.  In addition, CHS has performed detailed analysis comparing birth certificate data with hospital discharge data for selected items.       

At the national level, all electronic state files received by NCHS are automatically checked for completeness, individual item code validity, and unacceptable inconsistencies between data items (i.e., gestational age inconsistent with birth weight). NCHS staff reviews the files on an ongoing basis to detect problems in overall quality such as inadequate reporting for certain items, failure to follow NCHS coding rules, and systems and software errors. NCHS also analyzes year-to-year and area-to-area variations in the data. NCHS investigates all differences that are judged to have consequences for quality and completeness. In the review process, statistical tests are used to call initial attention to differences for possible follow-up. As necessary, registration areas are informed of differences encountered and asked to verify the counts or to determine the nature of the differences. Missing records and other problems detected by NCHS are resolved, and corrections are transmitted to NCHS.

NCHS also regularly compares levels for the medical and health items to be addressed in this study, reported by the revised reporting area (i.e., states which have implemented the 2003 revised birth certificate) with other data sources to evaluate data reliability, validity and completeness.

Since use of the new certificate includes new data items and changes in how these data are collected, it is crucial to ascertain how these new items and collection procedures are working. Therefore, the Division of Vital Statistics, NCHS, has requested our assistance in a qualitative study evaluating how birth certificate clerks collect a subset of data items included on the 2003 Revised U.S. Standard Certificate of Live Birth (and recommended to be captured using the facility worksheets). DVS is interested in how the birth certificate clerks collect specific data items, i.e., what sources containing specific data items are available to them to complete the worksheets (medical and/or prenatal records). How easy/difficult is it to find a specific data item? What do they do if they cannot find an item? A subsequent quantitative study is planned to evaluate the quality and completeness of medical and health data as collected on the revised birth certificate, by comparing birth certificate data to medical record data. These studies will contribute to our knowledge of issues to be addressed and best methods and practices for collection of birth certificate data.

The 2003 Revised U.S. Standard Certificate of Live Birth and the Facility Worksheet appear as Attachment 1a. These forms will be used as starting points in evaluating the collection of birth certificate data using focus group and cognitive interview techniques. The goal is to capture information from those who are on the front lines, actually collecting birth certificate data, about the collection birth certificate data. The moderator focus group guide appears as Attachment 1b and the introduction to the cognitive interview appears as Attachment 1c.

Initially, focus groups and cognitive interviews will be conducted in Washington State; the utility and value of this approach and these data will then be evaluated. Depending upon the results of this evaluation, as many as 20 states that now use the 2003 Revised U.S Standard Certificate of Live Birth may be asked to participate at a later time. The 28 states that currently use the 2003 Revised U.S. Standard Certificate of Live Birth are: California, Colorado, Delaware, Florida, Georgia, Idaho, Indiana, Iowa, Kansas, Kentucky, Michigan, Montana, Nebraska, New Hampshire, New Mexico, New York City, New York State, North Dakota, Ohio, Oregon, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Vermont, Washington, and Wyoming.

Focus groups and cognitive interviews will be conducted with adults (aged 18 years and older), in Washington State whose primary responsibility is collection of birth certificate data in hospitals and who have been doing this job for at least six months. The Washington State Department of Health, Center for Health Statistics, will provide NCHS with contact information (which is readily available) for the hospital administrators and the staff who collect birth certificate data (birth records specialists). An invitation letter and fact sheet will be used to recruit participants. An invitation letter and fact sheet about the study will be sent directly from NCHS to birth records specialists. At the same time, a slightly modified version of the invitational letter and the fact sheet will be sent to the appropriate hospital administrator/medical records supervisor, informing them about the study and asking for their support. See Attachment 2. There will be no coercion; birth records specialists will be told that their participation in the study is entirely voluntary.

As many as three 90-minute focus groups (consisting of a maximum of 8 people) and as many as fifty 60-minute cognitive interviews may be conducted. Participants may participate in both a focus group and a cognitive interview.

Focus groups and cognitive interviews will be conducted by QDRL staff members in a private room of the hospital facility or a community facility. With the consent of the participants, focus groups and the cognitive interviews will be recorded on audiotape. Participants will be informed of taping procedures (including observation if applicable) in the process of reviewing the consent forms, and the equipment will be turned on once it is clear that the procedures are understood and agreed upon. At the end of the interview/focus group, participants will be paid and provided with copies of all papers they signed.

We propose paying individuals participating in either the 90-minute focus group or the 60-minute cognitive interview $75.00 for their participation. Since birth clerks are extremely busy, and there is sometimes or often only one birth clerk per hospital, and hospitals are geographically spread out within a state, the incentive has been increased over and above our normal $40 incentive to increase participation, reduce the number of cancelations, and maximize time and travel. In total, for this project, the maximum respondent burden will be 86 hours of interviewing in addition to travel time. An updated burden table for this project is shown below:

Projects	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden
QDRL Interviews
2) Other questionnaire testing	50	1	1	50
5) Focus Groups	24	1	1.5	36

Attachments (4)

cc:

M. Moien

M. Daneshvar

S. Perryman

4