ATT E1 Board of Scientific Counselors Report of NSFG

NSFG 2015-2018 OMB Attachment E1 OMB No. 0920-0314

Report of the Review Panel on the National Survey of Family Growth,

to the Director of NCHS and the NCHS Board of Scientific Counselors

April 17, 2010

This report was written by the following Panel:

Panel Members

CHAIR: Wendy Manning, PhD, Professor of Sociology, Bowling Green State University, and
Co-Director, National Center for Marriage Research, Bowling Green, Ohio

Susan Averett, Ph.D., Charles A. Dana Professor of Economics, Lafayette College, Easton, PA

Sonya Borrerro, MD, MS, Professor of Internal Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA

Trena Ezzati-Rice, MA, Mathematical Statistician, Agency for Health Research and Quality, Rockville, MD

Lawrence B. Finer, PhD, Director of Domestic Research, Guttmacher Institute, New York, NY

Jennifer Manlove, PhD, Area Director & Senior Research Scientist, Child Trends, Inc., Washington, DC

Kathleen Mullan Harris, PhD, Professor of Sociology, University of North Carolina at Chapel Hill and Director, National Longitudinal Study of Adolescent Health, Chapel Hill, NC

BSC Liaison: Llewellyn Cornelius, Ph.D, Professor, University of Maryland School of Social Work, College Park, MD

Overview of the Report

The Board of Scientific counselors (BSC) of the National Center for Health Statistics (NCHS) commissioned a panel to review the National Survey of Family Growth (NSFG) as part of an on-going program review process and to report its findings to the BSC. This report

• summarizes the review process;

• provides background information on the NSFG, its history & current operations; describes the accomplishments, contributions and assets of the NSFG;

• describes the challenges it faces; and

• presents a series of recommendations.

Review Process

The NSFG Panel members (attachment 1) reviewed the BSC charge and the “Procedures for Reviewing the NCHS Program” (attachment 2) established by the BSC. These procedures call for the reviewers to examine the current status, scientific quality and responsiveness of each program within the context of its mission. Further, the review procedures require that the review takes into account future availability of financial and staffing resources, emphasizes forward thinking and future plans as well as assesses current operations, and conduct an interactive review that obtains information from written materials, presentations and discussion with program staff.

The Panel received advance information on the NSFG, including a comprehensive report on the program, prepared by NSFG staff and a number of additional references. Next, the Panel members had a conference call on January 26, 2010, to review the information received and to establish its approach for the review. At that time, the Panel submitted additional questions and information requests to the NSFG.

The Panel members met on February 22-23, 2010, at NCHS and followed an established agenda (attachment 3). During that two-day visit, they heard presentations by

--Dr. Edward Sondik, NCHS Director;

--Dr. Jennifer Madans, Associate Director for Science & Acting Deputy Director;

--Charles Rothwell, Director, Division of Vital Statistics;

--Stephanie Ventura, Chief, Reproductive Statistics Branch;

--Dr. William Mosher, NSFG team leader; and

--Peter Meyer, Director of the Research Data Center (RDC).

NSFG staff was available throughout both days to address and discuss various topics and respond to questions from the Panel. In addition to Dr. Mosher, NSFG staff present included Dr. Joyce Abma, Dr. Anjani Chandra, Dr. Casey Copen, Dr. Jo Jones and Dr. Gladys Martinez. At the end of the first day, the Panel drafted additional questions that were answered by the NSFG staff the next day.

Panel members reviewed and edited a detailed outline and specified additional topics to be addressed in the full report that was drafted, reviewed and revised. Panel Chair, Dr. Wendy Manning, presented the report to the BSC at the April 22-23, 2010 meeting. With BSC and further Panel input, the report was finalized and submitted to the BSC. A year later--at the April, 2011 BSC meeting-- the NSFG will report on the progress made on the recommendations and other actions taken as a result of the Panel’s report.

Description of the NSFG

For almost 40 years, the National Survey of Family Growth has collected, analyzed, and disseminated critical data on the nation’s fertility patterns and reproductive health status. It has provided a wide range of information to plan health and social services and health education programs, and to do statistical studies of families, fertility, and health.

Survey Overview and History

The National Survey of Family Growth collects information on family life, marriage and divorce, pregnancy, infertility, use of contraception, and men’s and women's reproductive health. The survey was first conducted by NCHS in 1973 and since that time has been conducted periodically in 1976, 1982, 1988, 1995, and in 2002. Beginning in mid-2006, the survey began continuous operations as an on-going survey. Over the past four decades the survey has grown in scope, expanding the sample population and the range of topics in the survey.

When originally conducted the survey was limited to ever-married women, then expanded to include never-married women in 1982. An independent, unlinked sample of men was added in 2002. Along with the expansion of the survey’s population was an expansion of the topics covered. The NSFG now collects more data on reproductive and sexual health, data on STD/HIV risk behaviors, fatherhood involvement and men’s and women’s attitudes toward marriage, parenthood and gender roles.

Survey Organizational Structure, Resources and Sponsors

Organizational Structure - The NSFG is located in the Division of Vital Statistics (DVS), Reproductive Statistics Branch. DVS manages and coordinates the National Vital Registration System, working with states on the reporting of births and deaths, and in earlier years, marriages and divorces. The placement of the NSFG in DVS supports the survey’s original role of producing data to help explain and interpret trends and group differentials in birth and pregnancy rates.

Staffing. The NSFG is designed and currently managed by 6 staff members. William Mosher is the team leader and has been with the survey for more than 30 years. Other staff members, Joyce Abma and Anjani Chandra have almost 20 years of service with NSFG. The remaining staff members are Gladys Martinez, who joined the NSFG in 2000; Jo Jones, who joined in 2004; and Casey Copen who started in 2008. The NSFG staff is a team with integrated functions and responsibilities. While there is some support for the claim “that everyone does everything,” staff members do have both functional and topical specialties. William Mosher is the project officer for the data collection contract and coordinates the work of the team in writing the contract, OMB clearance materials, IRB protocols, and review for the Disclosure Review Board (DRB). He also coordinates the team’s publications planning and work with outside funding agencies.

Most of the NSFG team members have design and implementation responsibilities for certain sections of the questionnaire corresponding to their research and analytical specialties. In addition, Joyce Abma is Alternative Project Officer, coordinating tasks related to producing the documentation for the public use files. Anjani Chandra serves as coordinator of the specifications and testing of all sections of both the male and female NSFG questionnaires for programming and oversees the design and implementation of the Automated Computer- Assisted Survey Interviewing (ACASI) sections of the questionnaires. Jo Jones works with users of the restricted data files in the Research Data Center and with the NCHS web site staff to upgrade and update the NSFG web pages. Gladys Martinez coordinates the translation, implementation and testing of the Spanish-language questionnaire and serves as the principal liaison to the Healthy People Family Planning Objectives Work Group. Casey Copen, the newest staff member, is assuming increasing responsibility for several topical sections of the questionnaire.

Resources and Sponsors. Since the first NSFG, data collection and survey field operations have been managed under contract. The survey staff work closely with the contractor and are involved in all phases of the survey operations. The University of Michigan Institute for Social Research was the contractor for the 2002 survey and is now the contractor for the current continuous survey through 2010. The NSFG has had long-term contractual relations with major national health survey organizations including the National Opinion Research Council, the contractor for the 1973 survey; Westat, the contractor for the next three cycles of the survey; and the Research Triangle Institute which was the contractor for the 1995 survey. A new contractor will be selected in 2010 to handle survey data collection for the next 10 years. A Request for Proposals (RFP) will be issued in the near future.

The current contract has an annual budget of $3.8 million per year of which NCHS funds $1 million. Funding sponsors provide the remaining $2.8 million. The funding sponsors are all Federal agencies. Along with NCHS, the National Institute for Child Health and Human Development (NICHD), National Institutes of Health (NIH) and the Office of Population Affairs (OPA) in the Department of Health and Human Services (DHHS) provide the bulk of the financial support. Additional support comes from three programs in the Centers for Disease Control and Prevention (CDC): Division of HIV/AIDS Prevention, Division of STD Prevention and the Division of Reproductive Health. In addition, the Office of the Assistant Secretary for Planning and Evaluation and the Agency for Children and Families Children’s Bureau, both in DHHS, also contribute.

Survey Operations

The NSFG is conducted through personal interviews conducted in the home using computer-assisted personal interviewing (CAPI) technology. Trained interviewers (all women) administer most of the survey; portions of the survey considered more sensitive are self-administered through ACASI. A nationally representative sample (ranging from 8,000 to 11,000) of women aged 15-44 was selected for each cycle of the survey from 1973 through 1995. In 2002, the sample was 7,600 women and 4,900 men. Beginning with the continuous survey in 2006, 5,000 women and men are selected for participation each year. An advance letter is mailed to selected households to make initial contact with residents. A screener identifies the individuals selected for the sample from each household and they receive additional information, explaining the purpose and importance of the survey. The NSFG currently oversamples African-Americans, Hispanics, and teenagers, 15-19 years of age, to improve the accuracy of the estimates for these groups. The survey obtains signed informed consent for adults and signed parental consent and signed assent for minors 15-17 years of age (or until age of majority as defined by state law). The interview is administered in English and Spanish. Usually about 6 percent of the questionnaires are administered in Spanish. The length of the interview has grown over time from about an hour in early cycles to an hour and 20 minutes for women in the continuous survey. The interview averages an hour for men. The NSFG now achieves a 75 percent response rate. The chart below summarizes the key facts about each survey cycle.

Survey Planning and Content

The survey is planned in coordination with the sponsoring agencies and selected outside experts to meet multiple and diverse purposes, from policymaking to program administration and a wide range of research and analytical applications. The NSFG maintains steady contact with funding agencies and obtains feedback through conferences, presentations and in regular interaction with data users. The NSFG held annual meetings with an informal, rotating group of outside advisors between 1990 and 2002 and again in 2008. The expert advisors at these meetings were chosen by the three primary funding agencies and representatives from each agency attended. These meetings served to provide insight into emerging data needs and how the NSFG could address them. The meetings often focused on questionnaire content to guide the major changes and additions made to the questionnaires.

Since Cycle 1 in 1973, the core of the NSFG’s content for women has included a birth and pregnancy history, a complete marriage history, partial histories of cohabitation and contraceptive use, and use of health care services for birth control and infertility. Since the 1980’s that content has also included data on nonmarital, noncohabiting sexual partners during the 12 months before the interview. For men, who were first interviewed in 2002, a history of births fathered, and information on marriage and cohabitation are collected, along with data on what the father does to help raise his children, whether they live with him or not. For both men and women, data are collected on social and demographic characteristics, health conditions related to reproduction, attitudes toward marriage and parenthood and sexual and drug use behaviors that affect the risk of HIV and other sexually transmitted diseases.

Data Access and Data Products

The NSFG produces several types of data products to meet the needs of researchers, program planners and administrators, policymakers, educators and students, clinicians, writers and reporters, and others. The NSFG web site is the most frequently used resource to obtain NSFG statistics. The web site at http://www.cdc.gov/nchs/nsfg.htm is the entry point to NSFG data resources and receives some 2,000-4,000 hits per month. About half of those web site visits are to “Key Statistics” available at http://www.cdc.gov/nchs/nsfg/abc_list.htm. “Key Statistics” is an alphabetized listing of basic findings on 80 key topics from the 2002 NSFG. Each entry gives a basic statistic, along with the source of the statistic, and a link to the report in which it was published for more information. “Key Statistics” is an effective way to refer users to the comprehensive and topical reports produced by NSFG staff on the data from each survey Cycle. All of the survey reports can be downloaded from the web site, where users can also find bibliographies of journal articles based on NSFG data. From Cycle 1 through Cycle 6, over 600 reports and journals articles based on NSFG data have been identified and referenced on the web site.

The public use data files offer the most extensive access to NSFG data available to all data users. Users have access to code books, data files and documentation from each of the NSFG Cycles. For example, for the 2002 Cycle 6 of the NSFG there are four public use data files: female respondent data file, female pregnancy data file, male respondent data files and a data file with household variables, each with corresponding codebooks. The web site also features variance estimation examples with programs and output in SAS, SUDAAN, Stata and WesVar for examples of many of the common types of variance estimation. Program statements to import each of the public use data files into SAS, SPSS and Stata are available as well.

A data file of the information collected in the ACASI portion of the survey is also available. The ACASI file includes those data items considered the most sensitive such as drug use, risk behaviors for HIV and other STD’s, nonvoluntary intercourse, anal and oral intercourse, and same-sex partners. To obtain the ACASI data file, requestors are asked to provide information on the proposed use of the data and a signed User Agreement to describe the specific protections in place that meet the NCHS standard for data users. There is no charge for the ACASI file.

Contextual (geographic) data files provide information on the context, or community, in which respondents live. Geographic data may include information for the region, state, county, census tract or block group in which the respondent lives A contextual data file for Cycle 6 is available now for use by the research community. These data are only available for use through the NCHS Research Data Center. The Research Data Center (RDC) was established to give public access to the full range of health and vital statistics data from NCHS, while protecting the confidentiality of the respondents. The RDC creates a file and offers basic analytic and methodological expertise to help analyze it. This can be done at the NCHS RDC in Hyattsville or Atlanta, in the several RDC’s operated by the Bureau of the Census, or through remote access. The data access process begins with a research proposal specifying the purpose, the restricted data measures to be used, and information on the analytical plan. Proposals are processed and reviewed, and this review process--done in concert with the requestor---may take from a few weeks to a few months. The charge for the RDC use includes a set-up fee ($500) and access fees of $200 per day for on-site access at an NCHS RDC or $500 per month for remote access. Staff assisted fees are $500 per day. The cost for an RDC project with NSFG data has averaged about $2,000.

There are listservs for the NCHS data systems, including NSFG. The listserv distributes announcements on releases, updates and technical assistance. Many users also contact the NSFG team directly and receive individual consultation and technical assistance with gaining access to the data, understanding the survey methodology and its impact on survey findings, or processing and analyzing the data.

Data Users and Uses

The NSFG data users are a diverse group of analysts, researchers and policymakers in a variety of settings, including demography, sociology and economics departments, schools of public health, non-profit research organizations, and many state and federal agencies. The results of analyses using these data are used to guide national policies and help direct federal programs. The data help shape programs in health and social services, both public and private. The funding agencies are primary data users and their applications of the data are illustrative of many others.

• NICHD uses NSFG data to set a research agenda, determine what topics or areas need to be investigated by grantees and contractors, and provide data sets for those researchers.

• OPA also uses NSFG data to shape its research agenda, as a data resource for grantees, and for program evaluation, particularly for family planning services funded under Title X, and for the Title XX teen pregnancy prevention programs.

• The Children’s Bureau of the ACF applies NSFG findings on adoption demand and preferences to develop and manage foster care and adoption policies and programs.

• The OASPE uses NSFG data, including new statistics on fatherhood, to develop and inform policies around families.

• The Division of HIV/AIDS Prevention of CDC uses the NSFG data on HIV/AIDS for behavioral surveillance and prevention activities.

• The Division of STD Prevention of CDC uses the data to study sexual behavior patterns, to better understand trends and differences in the rates of STDs in the United States.

Further, the Guttmacher Institute and Child Trends analyze and publish NSFG data in a myriad of applications and projects. The National Campaign to Prevent Teen and Unplanned Pregnancy relies upon NSFG findings to form its prevention strategies, identify groups at risk, evaluate initiatives and track progress. NSFG is the data source for nearly all of the Healthy People objectives on family planning and for some objectives on HIV and STD prevention and those on maternal and child health.

Responsiveness and Evolution of the NSFG

Methodological Enhancements. From the beginning of the survey, there was an emphasis on innovative and creative changes in survey methodology to improve the quality and scope of the data. After determining that it was both feasible and desirable, the survey sample was expanded to include all women ages 15-44 (regardless of marital status) in 1982 and, with a major restructuring, to include men in the sample in 2002. The questionnaires were modified in Cycles 3 and 4 to gain new information deemed important for a number of research and policy applications.

In 1990 the NSFG conducted a telephone re-interview with almost 6,000 respondents from Cycle 4 (1988) to gain some insight into longitudinal patterns. In 1995, the NSFG became one of the first national surveys to be administered by CAPI. A change to CAPI offers opportunities to improve the quality and standardization of interviewing and to offer opportunities to streamline and speed up data processing. After evaluation and assessment of the impact on response rates, the NSFG added incentives in 1995. Another change in Cycle 5 was the use of event histories to improve the completeness and accuracy of reporting on pregnancies and contraception practices, for example.

Continued Emphasis on Releasing More Timely Data. The NSFG staff recognizes the importance of improving the timeliness of the NSFG data and has taken some steps to try to do that. The long-standing goal of the NSFG is to release high-quality data products in a timely manner. Since Cycle 3 the highest priority has been to release a public use data file along with a report on a high-profile topic. For Cycles 3-5 it took about 18 months to release the public use data file, though the data file for Cycle 5 was considerably larger. For Cycle 6 the process took longer (21 months), due to the complexity of preparing two data files for the first time (on men and women) and dealing with several thousand new data items. With both publications and data files, the NSFG staff gives considerable thought to the sequencing of products to better meet the needs of the largest number of users in the shortest amount of time. The 2006-08 round is expected to be released 17 months after completion of data collection, which is later than originally planned.

Range of Data Products, Access Mechanisms, and Technical Assistance. NSFG does a thoughtful evaluation of its data products and explores ways to match those products to users’ needs and applications. The types of publications from NSFG staff have varied over time as the staff sought the best mix of detailed, comprehensive or short, topical reports and articles and most recently turning to a combination of the two. NSFG staff is conscientious in creating materials to help researchers, providing fully developed codebooks and documentation.

Methods of data release have improved over time—going from massive data tapes to CD’s and data files available over the Internet. The web has allowed access to several technical reports, questionnaires, and a comprehensive user guide. In addition, the documentation is fairly user friendly and complete. The FAQ’S is a new users’ tool which will be available soon. Providing answers to some of the basic questions, it will be a good resource especially at the start of a research project using NSFG data. Users value the one-on-one technical assistance they receive from NSFG staff. Staff members are well known and respected among researchers who frequently contact them for guidance on individual problems and specific applications. Staff members respond quickly to these requests, building a cadre of data users who expect and receive this customized assistance.

Accomplishments and Strengths of the NSFG

The NSFG has a long history of producing high-quality statistics on a wide range of topics related to family formation and reproductive health to meet critical data needs.

Valuable Data Source. The NSFG is a key source and, in some cases, the only national source of data on a number of key indicators. It offers a complete history of pregnancies and reports on outcomes not available from reporting systems. For example, the NSFG captures information on fetal loss regardless of the period of gestation. The National Vital Registration System is limited to late fetal deaths reported under state law. After the national marriage and divorce reporting system was discontinued, the NSFG became an important source of national data on marriage, divorce, and cohabitation and how those events relate to childbearing and other aspects of reproductive health. The survey provides information on fatherhood and the role of fathers that extends considerably the information available on American families. Given the current funding environment, the NSFG stands alone as the national survey of American family life. It has been referred to as America’s Fertility Survey.

Data for National Program and Policy Development. The NSFG provides the objective, scientific data to document the current status and monitor changes in reproductive practices, childbearing, and other aspects of family formation. The decline in teen birth rates over the past 15 years reported by NCHS, based on information from birth certificates, can be further explained by NSFG data which showed both a decline in teen sexual activity and an increase in the use of contraception at first intercourse, especially the use of the condom. For the recent increase in the teen birth rate and the decline in 2008, there were no NSFG data to help interpret the reasons for both of those changes--a data gap which will be addressed by the continuous NSFG.

The ability of the NSFG to contribute to health promotion and disease prevention initiatives increased greatly with the expansion of the ACASI questionnaire to collect more information on AIDS and STD risk behaviors and these data became important components of national prevention efforts.

Continuous Survey. Beginning in mid-2006, the NSFG moved from periodic data collection to a survey of continuous operations. This means that field work is virtually continuous and that survey planning, management, data release and analysis are performed concurrently rather than consecutively. This transition took several years of planning, most notably between 2000 and 2004, involving the NSFG team, co-funding agencies, and other researchers and experts. The factors that precipitated this change were declining response rates overall in health and social surveys. It was anticipated that the NSFG could also experience this decline, since NSFG response rates were being maintained only by extraordinary efforts accompanied by increasing survey costs. Also the NSFG needed to be conducted and data released more frequently to improve the timeliness of data and eliminate the long periods between surveys cycles when data are not available. Continuous operations with an essentially stable questionnaire reduce questionnaire redesign and pretesting costs and has resulted in a number of efficiencies in survey management. Fewer interviewers can collect data on a larger sample on a continuous basis and highly trained NSFG interviewers are kept on the job rather than recruiting and training new interviewers for each survey cycle.

So far, the continuous survey appears to be working very well. A larger survey sample has been interviewed with lower costs overall and a lower level of interviewer labor per completed interview. The weighted response rate is about 75 percent and almost 22,000 interviews will have been completed by June 2010, after four years of continuous operations. The current estimated date for the first release of data from the continuous survey is May 2010, a 17-month period between the end of data collection and release.

Paradata to Monitor and Improve Survey Management and Operations. An integral component of the continuous survey is the use of paradata (detailed information on field operations) that is reviewed on a daily basis. Paradata for the NSFG consists of:

• Level of effort – number of interviewers working, number of hours worked, percent of hours worked in peak times

• Characteristics of the active sample -- whether units are occupied and eligible, if interviewers are encountering locked buildings; cases which haven’t been contacted or worked; status of appointment; calculated propensity to respond.

• Productivity – interviews; hours worked per completed interview

• Data set balance -- response rates by subgroup

With this steady flow of information, survey operations can be fine-tuned and modified to identify and address problems swiftly and effectively. For example, there are methods of dealing with physical barriers, such as locked apartment buildings and gated communities. When an access problem is first encountered, building superintendents and community management staff are contacted and given an information folder to explain the access that is needed and why and to encourage their cooperation. The process of contacting and gaining support of building and community managers may take some time. Therefore it is essential to identify the access problem early in the 12-week interviewing period when it can be dealt with soon enough to prevent response rates from being adversely affected. Similarly, when response rates for certain population groups are lower than others, these interviews can be made a priority and slated early in the interviewing period, again, to give that extra time needed to gain their participation. Using paradata, changes can be made in real time to avert or deal with problems or factors adversely affecting efficient survey operations and to maintain the quality of data and response rates.

Paradata is also a mechanism to monitor costs and assign costs to specific aspects of survey operations and the cost impact of changes in survey practices. The NSFG is the first national survey to utilize paradata to this degree, and the use of paradata has contributed significantly to the continuous survey. The NSFG team has advanced Federal survey methodology with its application of paradata and the resulting improvements in continuous survey operations. NSFG staff members have been asked to share this information with others in the survey community.

NSFG Staff. Perhaps the single greatest asset of the NSFG is its staff. The staff, though small in number, is incredibly productive and has advanced the survey in ways that far larger groups might have failed to do. It is a highly organized and efficient team, with years of experience with this survey and in the field of reproductive health and family statistics. It brings together a depth of subject-matter expertise and skill in survey management. The staff is forward-thinking, seeking and obtaining solutions to current and anticipated challenges. It is proactive and strategic in its planning and its efforts have resulted in a survey which has much to be admired.

Challenges

NSFG Staff. Despite the fact that the staff is extraordinary, the NSFG staff is too few in number to maintain the survey’s continuous operations and meet the goal of more timely data release. The current staff of six is spread thin and burnout is a potential concern. With major responsibilities for survey management, staff has limited time for conducting their own research and analysis or for professional development. Two additional hires have been authorized and recruitment is underway. However, to fully achieve the potential of a continuous survey, maintain a sample size sufficient to produce the range of needed data and to speed the release of more complex data sets, a staff of 10-12 is needed. To guide future recruitment, staffing goals for the NSFG will need to be carefully defined to give the team the types of expertise and experience to address the survey’s future directions.

Budget and Survey Sponsorship. The NSFG is the nation’s fertility survey, and the level of funding does not seem to match this role. Within the NCHS family of surveys, the NSFG receives relatively low funding in comparison with other surveys. The NSFG has a smaller proportion of NCHS funding than any other NCHS data system. NCHS pays NSFG personnel costs, but only a third of the contract to conduct the survey is funded by NCHS. The remaining two-thirds is supported by other agencies thus resulting in a reliance on sponsors and an accompanying uncertainty of funding. The critical role of survey sponsors requires NCHS staff to expend a good deal of staff time to coordinate with sponsors and to ensure that the funders’ input in the survey is acknowledged, reviewed and implemented if possible. Solicitation, integration and management of additional sponsors would be time-consuming with possibly fewer benefits. While an additional concern is that the external funders’ support can be tenuous, the primarily funders for the NSFG have been remarkably steady.

A potential solution to some of the survey’s staffing concerns is to turn over some work to the contractor. However, contract funds are limited, thus precluding the transfer of more data management and documentation functions from NSFG staff to the contractor. This shift in some of the management duties could free NSFG staff for more complex substantive analytical and research activities.

The current level of funding is inadequate to reach and maintain the optimal sample size to produce reliable estimates for smaller population groups and thus increase the survey’s value in identifying disparities and differences which could call for policy and programmatic changes. A larger sample size would lend itself to more extensive trend analysis with reasonable standard errors. A higher level of financial support would enable the survey to respond to changing and emerging policy-related data needs. Significant changes, modifications, and expansion of the survey are dependent upon the resources for staff and/or contractor assistance.

Software Needs. A number of aspects of the survey from data collection through data release could be managed more efficiently with the use of customized software which is not used by the survey at this point. Better software is needed to enable the NSFG to respond more readily and easily to changes in the survey’s CAPI questionnaire and to test those changes. Currently, this is a laborious process that involves significant staff time to specify and test changes. A second kind of new software could potentially help to produce high-quality, detailed NSFG data file documentation with less direct staff involvement and manual effort and help speed the release of the public use data files. At present, the data preparation stage is a sequential process with each function performed before data release occurs. The care and precision in developing data files and the comprehensive data documentation result in high-quality data with appropriate utilization by well-informed data users, but the entire process works against the timely release of data.

Contractor. The data collection contract for the NSFG is being competed and the selection of a contractor for 2010-2020 will occur soon. It is essential that the new contractor be able to effectively handle the current survey operations and management as well as contribute toward future advancements and expansions and help ensure that the data are released in a timely manner.

Methodological Research and Support. Methodological research to explore ways to determine the impact and benefit of new options and directions for the NSFG is less extensive than desired due to limited NSFG staff time and staff resources in NCHS’s central research program, the Office of Research and Methodology (ORM). The survey has received valuable advice on sample design, weighting, variance estimation and other statistical issues from Karen Davis in ORM who spends 10-20 percent of her time on issues related to the NSFG. In addition, funding levels impact contractor resources which could be expended on methodological research. The methodological studies which have been performed by NSFG staff, contractors or others have focused—by necessity-- on improving aspects of the current survey rather than exploring new alternatives.

Future Survey Options and Expansion. A number of possible options for expanding the survey are being discussed. To varying degrees, all of the options would require an increase in staffing and resources, all would enhance the research and policy importance of the survey, and all would require an assessment of the impact on survey design, management and operations. However, some options are a marked departure from current survey operations and would require additional methodological analysis.

Expanding the survey’s age range is the option most frequently discussed by NSFG staff. The NSFG’s universe is men and women 15-44 years of age. Expanding the age range to 49 or 54 would make the NSFG data on marriage and divorce, cohabitation, infertility and infertility services, father involvement, adoption and step-parenting much more useful. As life events—pregnancy and childbearing, marriage and divorce—occur at later ages, ages 49 or 54 are more appropriate end points for the survey. For some socio-economic and demographic subgroups the expanded age range would be especially important. This survey option probably has the least impact in terms of survey design and operations. While it would be useful to add some new questions to take advantage of the higher age range, the survey overall would change little.

Inclusion of a prison or military sample would enrich the data, particularly on men. It would also provide insight into the behavior patterns of individuals in institutionalized settings. Data on the interaction of parents in jail with their children and the impact on family structure and characteristics would be important additions to family statistics. These data would also be valuable for looking at sexuality and STDs among this population. A military sample would be important to include as well. The rapidly increasing number of women in the military is only one justification. Greater insight into how the military experience affects marriage, divorce, cohabitation and childbearing would be important knowledge for policymakers and military leaders. Both of these additions to the sample could be challenging to reach, require extensive negotiation and planning with relevant institutions, generate logistical difficulties, and greatly increase the complexity of gaining informed consent.

Longitudinal follow-up surveys would have considerable analytical and scientific value. A short-term survey gathering information frequently, perhaps on a weekly basis, on contraceptive use and sexual activity would be particularly useful. It could provide accurate, timely information on key behaviors. This type of follow-up could be conducted by phone, Internet or a combination of those techniques. It could be administered to a subset of the original sample, either geographically or demographically defined.

A long-term longitudinal study also has merit but is perhaps a lower priority due to the level of funding and staffing needed to carry it out. The patterns of behavior and outcomes documented in the NSFG are continuous events and are affected by numerous factors and characteristics of the individual, family and community. A longitudinal study could provide the information to guide public health intervention and prevention efforts and to direct policies and programs supporting children and families. Which experiences lead to better outcomes for children, parents and families is a question that a longitudinal component to the NSFG could help answer. A telephone follow-up was conducted once (in 1990) and plans for another dropped because of resource constraints.

Including biomarkers in the NSFG would be a new venture, requiring survey design innovation, operational changes, and additional staff with appropriate expertise. While biomarkers have been added successfully to other household interview surveys (Add Health for one example), those surveys have usually focused on more general health topics. Collecting, processing and analyzing biomarker data would need to be included in the new or modified survey contract and/or NSFG staff would need to include those with expertise in these measures. However, biomarkers could add a new dimension to the interview data on STDs and could be introduced for a sub-sample of he population (e.g. teens and/or young adults).

It has been suggested that NSFG designate a core set of items to remain in the survey indefinitely and then supplement that core with rotating supplements or topical modules. The survey can respond to new topics and emerging data needs by replacing one module or supplement with another. Currently, because of financial constraints, less than 10 percent of the questions are new in each survey, so there is limited opportunity to add or modify the entire instrument. The core and module approach would allow the survey to address cutting-edge topics related to fertility behavior. As a practical matter, this approach would reduce the modifications required to the entire questionnaire and focus development and testing on the new module(s).

All of these options and possibly others could be considered for their potential contribution to the survey and the data it produces. However, even a serious assessment of the feasibility, cost, scientific and analytical merit would require staff time which would be difficult to divert from current responsibilities to take on this task.

Remote Access and Contextual Data. The full analytical potential of NSFG data requires effective access to restricted and contextual data. The NCHS RDC provides useful services at its facility or remotely. Since 2000, the RDC has handled about 30 research projects using NSFG data. RDC services have improved over the past few years with new procedures implemented to provide better and more efficient customer service. However, some users report that the process is at times cumbersome, lengthy, too costly, and fails to provide the level of support that analysts desire. In addition, some statistical packages are not an option when accessing these data nor do they include all of the information which would make the processing of data files more efficient. A review and modification of some of the products and practices may be timely as NSFG prepares to release the first data from the continuous survey.

Timeliness. One of the greatest challenges facing the survey is timeliness—releasing the data in the shortest period possible from data collection to first release. One of the major goals of adopting the continuous survey is to improve the speed at which the data are released. For the data collection period of July 2006-December 2008, the NSFG hoped to release the first public use data file by December 2009. This hasn’t happened and subsequent estimated release dates have not been met. The current release is delayed until May 2010 in part because it is the first time the continuous data are to be released. This delay poses problems for data users on two fronts—the sheer length of the time before data are ready and the uncertainty of when that will be and, to some extent, the lack of communication on the delays, reasons for the delays and revised release dates. It is important that the release dates are widely shared and accurate for future data releases.

Recommendations

The following recommendations are intended to build on the strong foundation established by the NSFG. Some of these recommendations require additional staff and funding resources, but others can proceed and can be implemented, at least to some degree, without that additional support. This list includes recommendations for action by NSFG staff and those which would involve support by NCHS.

1. Implement a Strategic Planning Process

• Establish an on-going overall strategic planning process to guide the survey in future directions by determining goals and priorities for the NSFG and the best way to reach those goals and support the priorities. This process would enhance current efforts and establish a process for all future planning.

• Incorporate a systematic method for getting input from the user community into the strategic planning process. In gaining input, NSFG should intentionally broaden user input, seek more interaction beyond the survey sponsor community, and seek diversity of user feedback. Continue the process of advisory input through research conferences and try to keep a regular and frequent schedule for those meetings.

• Use the strategic planning process to systematically evaluate the feasibility, advantages and disadvantages of future options, including-- but not limited to--expanding age range (both upper and lower); including incarcerated and/or military in the sample; adding biomarkers; conducting longitudinal follow-ups; and creating content core and rotating topical modules. Specifically, the process could provide opportunities to:

  • Determine new content topics and analytical opportunities for expanding the survey’s age range. On initial assessment, this survey option appears to contribute significantly to the quality and scope of the data with the least impact on on-going survey operations.

  • Evaluate the value and feasibility of expanding the survey with new population groups by including a sub-sample of military personnel or trying out biomarkers on a small sample of the full survey.

  • Include in the strategic planning process an assessment of future and emerging programmatic and policy data needs and consider methods of meeting those needs by changing or adding to survey content, including the use of a survey core plus rotating supplemental questions.

  • Explore strategies to incorporate a longitudinal component in the NSFG, including a smaller subsample on a long term basis or shorter term, such as a 6-month or one-year follow-up on sexual activity and contraceptive use. As part of this assessment examine why the previous follow-up (the 1990 round) didn’t yield the desired results and take those findings into account when deciding about future longitudinal efforts. Also, consider alternative data collection mechanisms, such as those that are web-based, for a follow-up survey.

• Consider appropriate possibilities for partnership for various future options, for example, with NICHD for longitudinal studies or new samples such as through an RFA mechanism

2. Develop a marketing plan

• Develop an approach to promote the NSFG and data produced by the survey to new and continuing audiences. Highlight new topics and those that could be considered “unexpected” in the NSFG data set to attract non-traditional NSFG users. Some of those “unexpected” topics might be those collected throughout the survey, such as child care, or newer topics such as father involvement or HIV/STD risk behaviors.

• Determine the best approach to market the continuous survey. With its expanded capabilities, the existence of the continuous survey offers new opportunities, such as trend data on fertility offered every two years, which should be widely disseminated to larger audiences and potential users.

• Utilize efficient and effective venues for communication. Try to maximize opportunities for exchange with larger audiences and reduce individual interchanges that may overextend existing NSFG staff. Also maximize electronic outreach, such as webinars, which can reach a large audience live as well as over time through archived webinars.

• Take advantage of existing NCHS-wide marketing efforts and request expanded or targeted and customized efforts for NSFG

• Consider obtaining marketing expertise through new staff hires or a contractor.

3. Improve the User Experience

A positive user experience is, in itself, one of the best marketing techniques. To that end, consider the following approaches:

• Provide downloadable NSFG data files in SAS, SPSS and Stata that are fully annotated, to make optimal use of the capabilities of the most popular statistical packages. Barriers to data would be significantly lowered if users could simply click on the dataset link on the NSFG website and have it open in their stats package. Concerns about software versions could be allayed by posting multiple versions for each package

• Embed details about how recodes were constructed, weights and complex survey design and imputation right in the data files, whenever possible using notes. Incorporating as much information and instructions as possible in the actual data files could reduce individual queries to staff and preserve the integrity of the data. Maximize the effectiveness of variable labels by including markers that indicate sections of the survey, whether variables are recodes or imputations, etc.

• Coordinate with the RDC team to ensure that NSFG data users understand the RDC process and to help them utilize the services in the way most advantageous to their research. Provide feedback on user experiences to the RDC staff to guide the RDC in improving services.

• Increase use and access to contextual data files. This can be facilitated in several ways:

  • Increase information on the NCHS web site on contextual data. Clearly describe the process and steps to accessing and using the data, as well as the necessary clearances. Add information on NCHS staff contact for more information and provide information on access modes.

  • Produce a report with contextual level data to demonstrate analytical possibilities, as well as the scope of the data set. Demonstrate the possibilities for regional analyses and specialized and policy-relevant analyses such as characteristics of communities with high poverty levels.

  • Improve the RDC remote data access by including Stata as a program in addition to SAS for remote access to contextual data files. This will open access to more users and facilitate adjusting for design effects in the analyses.

  • Prepare data files for the RDC which are standard restricted data files.

  • Create and post a dummy data set so that users can prepare in advance their research protocols to make the most productive use of RDC time.

  • Evaluate and explore how new research on maintaining confidentiality of data files could impact positively on the method and scope of data release. In addition, determine if technological advances can be employed to expand or improve data access.

  • Consider releasing documentation and program statements in advance of data release so analysts can prepare to use the data.

4. Study Ways to Improve Survey Processes

• Explore methods to streamline production of documentation, for example, the use of relational data- bases to develop documentation, which could reduce errors and remove inconsistencies between files.

• Develop a process of managing questions, keeping records and notes of the process of questionnaire development and modification. For both this and the previous bullet, consider ways to interface and share information with the contractor whenever possible to minimize information duplication.

5. Improve the Timeliness of Release of Data

• Employ all possible strategies to speed up the release of NSFG data while still maintaining accuracy. The continuous survey has great potential to reduce the survey management and data preparation time so that the interval between data collection and data release can be reduced. However, it has not been realized in the current planned release of 2006-08 data.

• Make the most realistic estimates of when data will be available. Allow time for unanticipated delays so that users will have the best information around which to plan their research and data applications. When delays occur, communicate early and often.

• Identify any functions which can be transferred to the contractor to free staff time.

6. Increase Funding for NSFG

• Emphasize the inappropriately low level of funding for NSFG. Make the case that the nation’s fertility survey merits a higher funding level as a priority NCHS data system. A higher level of funding is needed for a larger sample. The sample needs to be expanded to 15,000 every 2 years to allow adequate sample sizes for trend estimates and reliable estimates for critical sub-populations.

• Explore ways to increase funding from current sponsors, perhaps by highlighting the expanded capabilities and output from the continuous survey.

• Explore through the marketing program the possibility of expanding the number of sponsors, recognizing that additional sponsors will require staff support.

7. Enhance NSFG Staff Opportunities

• Request additional staff. Build the case for the optimum level of 10-12 staff members. This higher level of staffing is required to conduct the survey with sufficient sample size to produce data with the level of detail needed and to release that data in the timeframe desired.

• Explore ways that new hires can complement existing staff, by offering new skills and background and/or performing some of the “routine” tasks which consume a great deal of staff time.

• Determine ways to promote current staff development, including intramural research agenda and opportunities for training.

Conclusion

The NSFG is the nation’s fertility survey and as such provides essential information on key demographic measures and the status and trends in reproductive health. It provides the information needed for a range of programmatic and policy purposes and research needs. NSFG staff is doing an outstanding job, but clearly the NSFG is not funded at a level to meet its present responsibilities nor to advance and reach its full potential.

The innovative and carefully-designed continuous survey offers numerous advances in survey operations and data quality and may bring about the long-sought improvement in timeliness. The survey needs more resources and support to build upon that initial success, to fully employ technological advances, and to determine and design the optimal survey for the future.

10