NFSG Update: Resuming Data Collection with a New Multi-mode Design

National Center for Health Statistics

National Survey of Family Growth Update: Resuming

Data Collection with a New Multi-mode Design

Anjani Chandra, Ph.D.

NSFG Team Lead & Principal Investigator

NCHS Board of Scientific Counselors Meeting

February 10, 2022

NSFG

OMB Attachment F

OMB No. 0920-0314

NSFG Team

At NCHS:

NSFG Team within Division of Vital Statistics/Reproductive Statistics Branch:

Anjani Chandra, Team Lead/Principal Investigator

Joyce Abma, Contract Officer’s Representative

Gladys Martinez

Kim Daniels

Colleen Nugent

Jennifer Sayers

Hee-Choon Shin, Mathematical Statistician, Division of Research & Methodology

At RTI:

Andy Peytchev, Principal Investigator

Susan Kinsey, Project Director

Emilia Peytcheva, Instrumentation and Methodology Reports Task Leader

Taylor Lewis, Sample Design, Weighting, and Imputation Task Leader

Lilia Filippenko, Systems Development, Data Processing, Documentation, and Delivery Task Leader

William Savage, Information Security and Confidentiality Task Leader

Milton Cahoon, Preparation for Data Collection and Data Collection Task Leader

Overview of presentation



NSFG background



Progress since last BSC presentation in January 2020

–

Closeout of previous contract and final data release (2017-2019 NSFG)

–

Award of new contract September 2020 and preparation for data collection launch

–

Instrument development work – including adaptations for online mode

–

Multi-mode survey design envisioned for Year 1 data collection



Resumption of data collection in January 2022

–

Revisions made for Year 1, Quarter 1 due to COVID-19

–

Plans under review/consideration for Quarter 2+



Feedback and discussion

NSFG Background

NSFG Purpose and Uses



Address Section 306 of the Public Health Service Act, stipulating “NCHS

shall collect statistics on… family formation, growth, and dissolution”



Help explain variations in birth rates using intermediate or “proximate”

determinants of fertility



Assess factors that affect timing and consequences of sexual activity and

pregnancy



Describe relationships and families



Measure receipt of family planning and other medical services



Monitor risk of HIV and STIs in general household population

NSFG Background (1973-2019)



Each survey provides a nationally representative, cross-sectional snapshot

of the U.S. household “reproductive-age” population



Key changes since “Cycle 1” in 1973:

–

1982 – Inclusion of never-married women

–

1988 & 1995 - linked to NHIS sampling frame from several years prior

–

1995 – Conversion to CAPI and ACASI and 1st use of incentives

–

2002 – Inclusion of men (independent sample) & expansion of ACASI

–

2006 – Transition to continuous fieldwork design

–

2015 – Expansion of age range from 15-44 to 15-49



Public-use files have been released with every periodic survey 1973-2002

(Cycles 1-6) and roughly every 2 years since 2006 (under continuous

fieldwork design)

Year

Scope

N

Over Samples

Response Rates

Incentive

OMB Approved

Intvw Length

1973

Ever-Married Women

15-44

9,797

Black women

90.2%

None

60 min

1976

Ever-Married Women

15-44

8,611

Black women

82.7%

None

60 min

1982

All Women 15-44

7,969

Black women & teens 15-19

79.4%

None

60 min

1988

All Women 15-44

8,450

Black women

82.5%

None

70 min

1995

All Women 15-44

10,847

Black & Hispanic women

78.7%

$20

100 min

2002

All Women & Men

12,571

W = 7,643

M = 4,928

Black people, Hispanic people,

People 15-19

All - 79%

W – 80%

M – 78%

$40

W – 85 min

M – 60 min

2006-2010

All Women & Men 15-

44

22,682

W=12,279

M=10,403

Black people, Hispanic people,

People 15-19

All - 77%

W – 78%

M – 75%

$40

W – 85 min

M – 60 min

2011-2013

All Women & Men 15-

44

10,416

W=5,601

M=5,815

Black people, Hispanic people,

People 15-19

All – 72.8%

W – 73.4%

M – 72.1%

$40

W – 80 min

M – 60 min

2013-2015

All Women & Men 15-

44

10,205

W=5,699

M=4,506

Black people, Hispanic people,

People 15-19

All – 69.3%

W – 71.2%

M – 67.1%

$40

W – 80 min

M – 60 min

2015-2017

All Women & Men now

15-49

10,094

W=5,554

M=4,540

Black people, Hispanic people,

People 15-19

All – 65.3%

W – 66.7%

M – 63.6%

$40

W – 80 min

M – 60 min

2017-2019

All Women & Men 15-

49

11,347

W=6,141

M=5,206

Black people, Hispanic people,

People 15-19

All – 63.4%

W – 65.2%

M – 61.4%

$40

W – 80 min

M – 60 min

Current NSFG Cosponsors (all within DHHS)



National Center for Health Statistics (NCHS)



Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD)



Office of Population Affairs (OPA)



Administration for Children & Families (ACF)/ Children’s Bureau



Administration for Children & Families (ACF)/ Office of Planning, Research, & Evaluation (OPRE)



Office on Women’s Health



CDC/NCHHSTP/Division of HIV Prevention



CDC/NCHHSTP/Division of STD Prevention



CDC/NCHHSTP/Division of Adolescent & School Health



CDC/NCCDPHP/Division of Reproductive Health



CDC/NCCDPHP/Division of Cancer Prevention & Control



CDC/NCIPC/Division of Violence Prevention

Progress since last BSC update

in January 2020

Closeout of previous contract and final data release

(2017-2019 NSFG)



Previous 10-year contract with University of Michigan ended Dec 2020

–

Final public-use files for 2017-2019 NSFG released in October 2020

–

Restricted-use files made available through Research Data Center (RDC):

•

Contextual, interviewer observations, and paradata files

•

Additional data, formerly included on PUF, but now only available in RDC due to disclosure risk



Analyses with latest NSFG data:

–

Data Brief on current contraceptive status released with 2017-2019 PUF in October 2020

–

NSFG Key Statistics page updated with 2017-2019 data in November 2021

–

Several QuickStats with 2017-2019 data published in 2021, but most other analyses delayed due to

staff size and competing demands of new contract and 3 staff COVID deployments

•

Teen report using data for 2015-2017

•

Menarche report using data through 2017

Award of new contract to RTI and preparations for

Year 1 (2022) data collection



New 10-year contract awarded to RTI in September 2020, with plans to

resume data collection with a multi-mode survey design in January 2022



Oversight, review, and collaboration with contractor on:

–

Sample design and establishing quarterly sample areas

–

Interviewer training materials and other preparation for multi-mode data collection

–

Onboarding of contractor staff, hiring interviewers

–

Establishing IT systems that meet CDC security requirements

–

Modification of contract to move web survey mode earlier (to 1st year of data

collection) due to COVID

–

Obtaining clearances for OMB and ERB

Instrument development work



Producing programmer-ready full specifications based on “capi-lite” questionnaires in RFP

–

Streamlining and restructuring of content to reduce complexity and respondent burden

–

Improvements to accommodate respondents in same-sex marriages and cohabitations

–

Incorporation of new items from cosponsors and revisions prompted by CCQDER work***

–

Adapting survey instruments and interview aids to work for both FTF and online modes

–

Translating new or revised items into Spanish



Developing web screener to manage multiple survey tasks accomplished in FTF mode



Developing an electronic life history calendar for female survey



Extensive instrument testing in English and Spanish

***Collaboration with CCQDER on selected instrument issues:

–

Cognitive interviews to assess Life History Calendar, as well as selected question series

–

RANDS panel (in progress) to do split-panel testing of specific questions or series

Survey design envisioned for Year 1



Continuing to interview 1 person aged 15-49 per sample household



Building off success of continuous FTF survey design used 2006-2019:

–

Responsive survey design using paradata in real time with subsampling for nonresponse allowed for

nonresponse bias mitigation with cost control

Prior FTF design

•

Fieldwork organized into 4 12-week quarters/year

•

2-Phase design:

–

Phase 1 (weeks 1-10):



All sample worked



$0 screener, $40 main incentive

–

Phase 2 (weeks 11-12):



30% subsample of non-respondents



$5 prepaid screener; $40 prepaid for main,

and $40 at end for main

Current Multi-mode design

•

Fieldwork organized into 4 16-week quarters/year

•

3-Phase design:

–

Phase 1 (weeks 1-4): Web



$2 screener, $40 main incentive

–

Phase 2 (weeks 5-12): Web & FTF



No change in incentive

–

Phase 3 (weeks 13-16): Web & FTF



Up to 50% subsample of non-respondents



$5 prepaid screener; $40 prepaid for main,

and $40 at end for main

Multi-mode, multi-phase survey design for Year 1:

4 overlapping quarters

Multi-mode Comparison Study (originally planned for

Q1/2, but shifted to Q3/4)

New Multi-mode Design:

Face to Face, as used 2006-2019:

Web (CAWI)

(Phase 1)

Face to Face (Phase 1)

Face to Face (and Web continued) (Phase 2)

Face to Face, increased incentives (Phase 3)

Face to Face, increased incentives (Phase 2)

Mail eligibility or NRFU survey (Phase 4)

Resumption of data collection

in January 2022

Quarter 1 changes and challenges



The surge in COVID-19 rates, national staffing shortages, and contractor

policy precluded FTF interviewing in most NSFG sample areas at start of

Q1; policy changed as of 2/3/22 to allow some FTF during Q1.



As a result, for Quarter 1:

–

Only CAWI data collection in Phases 1 and 2

–

Field staff will conduct Phase 3 interviews with higher incentives, for a subsample of

non-respondents from Phases 1 & 2, as originally planned, and will follow up 100% of

breakoffs.

–

Seeking OMB and ERB approval for further changes in Quarter 2

Changes being considered for Quarter 2+ (1 of 2)



Mailed paper screener to augment CAWI screener

–

Most nonresponse in HH surveys relying on mailed invitations to a web survey is at

screener stage. Even if FTF interviewing can resume to originally intended levels in Q2,

mailed screeners could reduce reliance on in-person follow-up.

–

Paper screener would be mailed around week 2 (on 3

rd

mailing, approx. 1 week after 1

st

mailing)

–

Would only be used to select adult respondents, given complexity of parental

permission and minor assent process for teens 15-17

–

We would evaluate impact on response rates, demographic composition, & cost

indicators.



Conducting telephone prompting calls to complete the web main survey

Changes being considered for Quarter 2+ (2 of 2)



Experiment to test accelerated delivery of Phase 3’s higher incentives

–

Phase 3 protocol already approved by OMB & ERB:

•

$5 prepaid incentive for screener; $40 prepaid incentive before main and $40 upon

completion

–

Experimental condition would be assigned 50-50 at sample address level – half receiving

higher incentives at standard Phase 3 timing (weeks 13-16) and half receiving up front in

Phase 1.

–

Higher incentives have potential to increase response rates and decrease nonresponse

bias

–

Incentivizing participation from the beginning could be cost-efficient overall:

•

In areas where FTF is not possible, can reduce # of mailings needed per case

•

In areas where FTF is possible, can reduce # of sample addresses needing FTF

–

Some increased challenge for field staff juggling different incentive protocols

Feedback and Discussion