SSB_FertilityKnowledgeSurvey_OASH-OPA_Feb2019_V8_ROCIS_2-22-19

Supporting Statement
Parts B: Collection of Information Employing Statistical Methods
FERTILITY KNOWLEDGE SURVEY

February 22, 2019

Submitted to

Office of Management and Budget
Office of Information and Regulatory Affairs

Submitted by

Department of Health and Human Services
Office of the Assistant Secretary for Health
Office of Population Affairs

Contact Information:

Diane Foley, MD
Deputy Assistant Secretary of Population Affairs
Phone/Office: (240) 453-2888
Email: [email protected]

This page is intentionally blank.

Supporting Statement
Part B: Collection of Information Employing Statistical Methods
Fertility Knowledge Survey

Contents

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 1

1. Respondent Universe and Sampling Methods 1

2. Procedures for the Collection of Information 3

3. Methods to Maximize Response and Deal with Non-Response 5

4. Tests of Procedures or Methods to be Undertaken 6

5. Consultations on Statistical Aspects of Survey Design 6

EXHIBITS

Exhibit 1–Ipsos KnowledgePanel® and YouthPulse Panel: Eligible Population, Number of Eligible Panel Members Invited to Participant, Estimated Number of Completes, and Cooperation Rate 1

Exhibit 2–Power Analysis Assumptions and Inputs for Determination of Female and Male Sample Sizes 3

Exhibit 3–Consultations on Statistical Aspect of Survey Design 7

ATTACHMENTS

A. Authorizing Title X Program Regulations A–1

B. Fertility Knowledge Survey: Female & Male Versions B–1

C. Federal Register Notice C–1

D. Notice of RTI Institutional Review Board Study Approval D–1

E. Email Invitation to Adult KnowledgePanel® and All (Adults and Minors)
YouthPulse Panel Members E–1

F. Email Invitation to Parent KnowledgePanel® Member with
an Eligible Minor-Aged Child F–1

G. Consent Form: Adult KnowledgePanel® and YouthPulse Panel Members G–1

H. Parental Permission Form for KnowledgePanel® Members with
an Eligible Minor-Aged Child H–1

I. Assent Form for Minor-Aged (15-17 years) Children of KnowledgePanel® Members and Minor-Aged (17 years) YouthPulse Panel Members I–1

J. Email Reminders J–1

K. KnowledgePanel® Bill of Rights and “The Deal” K–1

L. Privacy Statement for KnowledgePanel® Members L–1

M. Statement of Ipsos Commitment to Privacy and Data Protection M–1

This page is intentionally blank.

Supporting Statement:
Part B Collection of Information Employing Statistical Methods
Fertility Knowledge Survey

2. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

The Fertility Knowledge Survey will be administered online using the Ipsos’s KnowledgePanel® and YouthPulse Panel, both of which are based on a scientific, probability-based sample of the U.S. population. The panels were created by GfK Custom Research, LLC (“GfK”), a consumer and marketing research firm. In 2018, Ipsos acquired four global divisions of GfK Research (Customer Experience, Experience Innovation, Health and Public Affairs), including the research panels (e.g., KnowledgePanel® and YouthPulse) that they manage.

1. Respondent Universe and Sampling Methods

Respondent Universe. Response for our survey will reflect the response rates that Ipsos obtains in recruiting, onboarding, and retaining its panels, as well as the rate of completion for the Fertility Knowledge Survey. Exhibit 1 presents estimates for the number of eligible population in the panels (i.e., potential respondent universe), the number of eligible panel members that is invited to complete the survey, and the estimated number of invited panel members that completes the survey (i.e., sample size).

Exhibit 1–Ipsos KnowledgePanel® and YouthPulse Panel: Eligible Population, Number of Eligible Panel Members Invited to Participant, Estimated Number of Completes, and Cooperation Rate

Respondents	Number of Eligible Population	Number of Eligible Population Invited to Complete the Survey	Estimated Number of Completes (Desired Sample Size)	Cooperation Rate (% Invited anticipated to Complete)
Females
15-19	2,717	2,221	700	32%
20-24	5,433	1,921	700	36%
25-29	2,560	2,013	700	35%
Subtotal	10,710	6,155	2,100	34%
Males
15-19	2,273	2,186	700	32%
20-24	3,954	1,907	700	37%
25-29	1,393	1,393	500	36%
Subtotal	7,620	5,486	1,900	35%
Total	18,330	11,642	4,000	34%

The estimated cooperation rate for this survey is the percentage of eligible panel members invited that completes the survey. The number of the eligible population invited to complete the survey is calculated by dividing the desired number of completed interviews by the expected cooperation rate, which is based on Ipsos’s previous experience with similar studies. A small buffer will be provided in both the number invited and the number of completed responses collected. Non-response is the complement to the cooperation rate: Non-response rate plus cooperation rate equals 100%.

Sampling Method. Ipsos will take a stratified random sample of eligible persons from the KnowledgePanel® and YouthPulse Panel. The target overall sample size of 4,000 will be stratified by sex with the goal of obtaining approximately 1,900 male respondents and 2,100 female respondents. Ipsos’s patented sample selection methodology, which has been used since 2000, ensures that the panel samples closely track the U.S. population. The sample selection methodology was developed in recognition of the practical issue that different surveys target different subpopulations. Often, only panel members with certain characteristics are selected for a survey. This can skew the remaining panel sample and affect the sample representativeness of later surveys. The patented methodology also attempts to adjust or correct for nonresponse and noncoverage error in the panel sample. Ipsos’s primary sampling rule is not to assign more than six surveys per month to members with the expectation that a panel member will complete, on average, four surveys a month.

Sample size and statistical power analysis. To obtain the nationally representative sample of English-speaking females and males aged 15 to 29 years, and to have a sufficient sample for detecting differences between three age subgroups (15 to 19, 20 to 24, and 25 to 29) for a three-category (female) or two-category (male) dependent variable, RTI obtained a desired sample size (completes) of 2,100 female and 1,900 male respondents by conducting a power analysis with the assumptions shown in Exhibit 2. The minimal detectable difference is 8.1 percentage points for female sample and between 8.1 and 9.5 percentage points for the male sample. Previous research was used to inform the RTI study team’s thinking about the minimal detectable effect size. While there are no directly comparable fertility knowledge studies of the 15 to 29-year-old U.S. population, a 2014 fertility knowledge study¹ based on a non-probability sample of females aged 18 to 40 years, found differences greater than 8.1 percentage points on knowledge outcomes (e.g., age and time to conception or obesity as a fertility risk factor) when comparing 18 to 24 and 25 to 29-year-olds. The Office of Population Affairs staff, with input from statistical and substantive experts, determined that the minimal detectable differences with the sample size shown in Exhibit 2 would be sufficient to detect meaningful and policy relevant differences. Smaller point differences, in addition to being harder to detect, would also not necessarily be actionable differences.

Exhibit 2–Power Analysis Assumptions and Inputs for Determination of Female and Male Sample Sizes

Assumption	Female Sample	Male Sample
Sample size	N=2,100 females 15-19 = 700 20-24 = 700 25-29 = 700	N=1,900 males 15-19 = 700 20-24 = 700 25-29 = 500
Design effect	1.5	1.5
Estimated (%) of outcome of interest	50%	50%
Power (Type II)	80%	80%
Alphaa (Type I)	0.05	0.05
Alternate hypothesis	Two-sided	Two-sided
Minimum detectable difference	8.1 percentage points	> 8.1 points and < 9.5 points

^a Adjusted for multiple comparisons.

2. Procedures for the Collection of Information

Ipsos will begin online data collection after the RTI team has tested and approved the final online version of the OMB-approved survey.

Inviting eligible panelists to participate. Ipsos will invite (“assign”) the 11,642 eligible panel members (6,155 females and 5,486 males) selected to complete the survey. When Ipsos “assigns” (i.e., eligible panelist selected to be invited) a survey to a panel member, the panelist receives a notification or invitation (Attachments E and F) in their password-protected email account that a survey is available for them to complete. This email notification/invitation contains a custom link that, when clicked, will take the panelist to the consent page and survey link without requiring any further login or password verifications. Panelists can also access their assigned surveys from their password-protected personalized landing page on the panel website.

For minor-aged (15 to 17 years) children of KnowledgePanel® members (“parent KP panelist”), all communication (invitation, nonresponse reminders) regarding the participation of these minors will be with the parent and not the minor. Ipsos will send an email invitation to the parent KP panelists (Attachment F) requesting permission for their minor child’s participation in the survey. The survey will also be listed on the parent’s personalized landing page.

All invitations will include an FAQ section that will provide panelists and parent KP panelists with enough information on which to base their consent decision. The FAQs will address questions about the survey sponsor and purpose, topics addressed, potential risks, voluntary nature of the survey participation and the right to skip questions, privacy and confidentiality of data, and contact information if the parent has questions. The text of the email invitations for all participants are presented in these attachments:

• Attachment E: Email Invitation to Adult KnowledgePanel® and All (Adults and Minors) YouthPulse Panel Members

• Attachment F: Email Invitation to Parent KnowledgePanel® Member with an Eligible Minor-Aged Child

Informed consent and assent Procedures. Ipsos will obtain informed consent from adult panelists aged 18 to 29 years, parental permission from parent KP panelists, and assent from respondents aged 17 or younger. Parental permission and respondent consent and assents will be obtained and documented electronically using links at the bottom of the forms. The consent, permission, and assent forms are written in plain language, and have been reviewed and approved by the RTI International IRB. These forms are presented in these attachments:

• Attachment G: Consent Form: Adult KnowledgePanel® and YouthPulse Panel Members

• Attachment H: Parental Permission Form for KnowledgePanel® Members with an Eligible Minor-Aged Child

• Attachment I: Assent Form for Minor-Aged (15 to 17 years) Children of KnowledgePanel® Members and Minor-Aged (17 years) YouthPulse Panel Members

Parental permission and minor assent procedures. Ipsos will obtain electronic parental permission (Attachment H) from KnowledgePanel® members whose minor-aged children have been selected to participate (“parent KP panelists”) and, if the parent gives permission, electronic assent (Attachment I) from their minor-aged child. In addition to granting permission by clicking on the “Yes” action button at the bottom of the permission form, the parent must either forward the survey link to their child or provide their electronic device to the child so that the minor child can review the assent form and click the “Yes” button to register their assent and launch the survey.

Ipsos has already obtained permission from parents of minor-aged YouthPulse panelists (aged 17 years) for their participation in surveys on a range of topics, including those that address sensitive topics and risk behaviors. Therefore, no additional permissions will be sought from the parents of minor-aged YouthPulse panelists.

Data collection and nonresponse follow-up. The data collection period, including nonresponse follow-up will be 14 to 21 days. Survey respondents can break off and return to complete an interview during a second or later session. To encourage response to the survey, Ipsos will send up to two email reminders (Attachment J) to those who have not started the survey and those who have started but not completed it.

Participant questions or issues. Ipsos has a system in place whereby panel members may call a panel support line (1-800-782-6899), maintained and staffed by Ipsos, to ask questions and communicate problems related to a study. The toll-free phone number for the panel support line is provided in the study email invitations (Attachments E and F); parental permission, assent, and consent forms (Attachments G through I); and the KnowledgePanel® Member Privacy Statement (Attachment L). In addition, there is a Support Center email address ([email protected]) for panelists to communicate with Ipsos. Ipsos logs into a panel relations database for each contact made or received.

Survey participants who contact Ipsos with a question or concern about the study will receive contact information for the RTI principal investigator (PI), Dr. Christina Fowler, and the RTI IRB. In addition, if a study participant reports an adverse event or serious problem, Ipsos will promptly notify the RTI PI. In cases of an adverse event or serious problem, RTI will inform OPA. All permission, consent, and assent forms include contact information for the hotline, RTI PI, and RTI Office of Research Protections.

Sample weighting. Once the data are collected, Ipsos will compute weights that will improve the representation of the study respondents. While reflecting the selection probabilities of sampled units, the weights will aim to compensate for any differential nonresponse (i.e., cooperation) encountered during survey administration. The resulting weights will be adjusted to the benchmark distributions of the target population using an iterative proportional fitting (raking) procedure. Moreover, calculated weights will be examined to identify and, if necessary, trim outliers at the extreme upper and lower tails of the weight distribution. The final weights will be scaled to sum to the total number of completed surveys.

Quality control procedures. The Ipsos quality control process for data collection includes multiple procedures performed at different stages of the process. The Ipsos Quality Control manager supervises the following procedures:

• Review Microsoft Word version of the questionnaire

• Review logic flow

• Review behavior of sample variables (used in skip logic)

• Estimate number of test cases to be performed

• Complete QC checklist/perform test cases on computer

• Perform data verification

• Re-test for corrections

• Configure sample variables

• Assign survey to panelists’ accounts

• Test on Internet platform

• Assign survey to panel members for one or more pretests

• Assess pretest results

• Make any corrections as needed

• Assign survey to main survey sample according to the sampling plan

3. Methods to Maximize Response and Deal with Non-Response

Incentives. As described in Section A.9, survey respondents will receive a small incentive payment for completing the Fertility Knowledge Survey.

• YouthPulse Panel members will receive a general incentive of $10, and no study-specific incentive.

• KnowledgePanel® members who Ipsos provides with equipment (e.g., laptop) or internet access as a general incentive for participating in surveys will receive the equivalent of $5 in the form of points (5,000).

• KnowledgePanel® members who do not receive equipment (e.g., laptop) or internet access receive a general incentive a 1,000 points and entry into a sweepstakes, and 5,000 points in a study-specific incentive.

Length and flexible features of data collection. The survey field period will be from 14 to 21 days to maximize the survey completion rate. Each active member has a personalized home page that lists all the surveys that have been assigned to that member and have yet to be completed. Survey respondents can break off and return to complete an interview during a second or later session.

Nonresponse follow-up. Ipsos will send up to two email reminders (sent after 3 days) to non-responders (i.e., those who have not started the survey and those who have not completed it) to encourage response (Attachment J).

Weighting for differential nonresponse. Once the data are collected, weighting will be used to compensate for differential nonresponse. Ipsos will provide information on the characteristics of non-responders.

4. Tests of Procedures or Methods to be Undertaken

Cognitive and usability testing. As described in Supporting Statement Part A, Section A.8, in December 2018 RTI completed cognitive and usability testing of the online survey with nine males and females, aged 16 to 29 years, to assess whether they understood the survey questions, whether they could answer them as intended, and whether the visual layout, design, and navigational features of the online survey were easy to follow. In response to participant feedback and in consultation with RTI survey methodologists, the RTI team made the following changes to the survey: defining or describing key terms and phrases on every page where they appear; using the definitions of “female fertility” and “male fertility” instead of the terms themselves; placing descriptive text before versus after the survey question; defining vague terms and phrases; revising the instructions for true/false statements; and bolding or underlining words for emphasis.

Quality control pretest prior to launch. As a final quality control step prior to the survey’s full administration, Ipsos will pretest the OMB-approved, online instrument with no more than nine panelists. The pretest results will be analyzed and any final corrections will be made, if needed.

5. Consultations on Statistical Aspects of Survey Design

Exhibit 3 provides the name, affiliation, telephone number, and email address for each individual consulted on statistical aspects of the design and their role in design, collection, or analysis of the data. The list also includes the name of personnel responsible for receiving and approving contract deliverables.

Exhibit 3–Consultations on Statistical Aspect of Survey Design

Name	Telephone Number Email Address	Role
		Design	Collect	Analyze	Other
Michael Lawrence, PhD Ipsos	(202) 370 6345 [email protected]				Subcontract management, deliverable review
Christina Fowler, PhD RTI International	(919) 316-3447 [email protected]				Subcontract management, deliverable review
Jamie Ridenhour, MS RTI International	(919) 541-6567 [email protected]				Deliverable review
Helen P. Koo, DrPH RTI International	(919) 493-1207 [email protected]				Deliverable review
Julia Gable, MS RTI International	(770) 234-5018 [email protected]				Deliverable review
Carol Ward, DrPH MITRE Corporation	703-983-0388 [email protected]				Subcontract management, deliverable review
Stefanie Schmidt, PhD MITRE Corporation	703-983-4074 [email protected]				Subcontract management, deliverable review
Samruddhi Thaker, MBBS, MHA, PhD MITRE Corporation	919-619-8705 [email protected]				Subcontract management, deliverable review
Kate Ahrens, PhD Office of Population Affairs*	(240) 453-2802 [email protected]				Deliverable review
Karen Silver Office of Population Affairs	(240) 453-2802 [email protected]				Deliverable review and approval
Diane Foley, MD Office of Population Affairs	(240) 453-2888 [email protected]				Deliverable review and approval
Alicia Richmond Office of the Assistant Secretary for Health	(240) 453-2816 [email protected]				Deliverable review and approval

* Dr. Ahrens left OPA in July 2018.

References

1. Lundsberg LS, Pal L, Gariepy AM, Xu X, Chu MC, Illuzzi JL. Knowledge, attitudes, and practices regarding conception and fertility: a population-based survey among reproductive-age United States women. Fertility and sterility 2014;101:767-74.