OMB Memo

Generic Information Collection Request (ICR):
2020 Census Integrated Communications Campaign (ICC) –
Creative Testing – Quick Idea Platform Testing (QIPT)

Request: The U.S. Census Bureau plans to conduct additional research under the Generic Clearance for Internet Nonprobability Panel Pretesting (OMB Control Number 0607-0978). The Census Bureau’s Integrated Partnership and Communications (IPC) program plans to conduct online testing and focus groups as part of the 2020 Integrated Communications Campaign (ICC) Quick Idea Platform Testing (QIPT). Results will drive creative development for the 2020 ICC. In this current submission, we are seeking approval to conduct nonprobability, non-generalizable quantitative and qualitative information collection to inform creative development activities. Nonprobability is an appropriate approach for experimental design, since generalizability to the entire population is not necessary nor required.

Purpose: The results of QIPT online testing and focus groups will serve as inputs for creative development, and more specifically for the selection of the 2020 ICC creative platform. The creative platform is the unified campaign concept, designed not only to reach a “diverse mass” audience, but also to include those with a lower propensity to respond or otherwise difficult to reach.¹ This testing is designed to both assess and collect feedback on three unique platform ideas developed to promote the 2020 Census. Results will provide direction for selecting the ultimate campaign platform concept, as well as insights for adapting the platform concept for specific audiences. The platform will then be developed into draft creative advertisements that we will subsequently test. We will submit another similarly structured package to OMB outlining the request, purpose, and methodology for the second phase of 2020 ICC Creative Campaign Testing in the coming weeks.

Stimuli Tested in QIPT

We will test three potential campaign platform concepts, which will be developed in the 2020 ICC Creative Workshop, to be held in fall 2018. The platforms to be tested will consist of creative element ideas such as tagline, graphical layout, and imagery. The stimuli are an industry standard concept called “ad-like objects” (adlobs), which resemble a print advertisement but do not include all elements that might be in the final ad.

Each adlob will contain a required set of features including:

• Tagline

• Paragraph of text

• Image

• Graphical layout

QIPT Online Testing

Online creative testing is considered an essential tool for major creative development projects and is standard throughout the industry, and will allow us to get a broad representation of the U.S. population. Integrating online testing into the creative development process helps inform both the selection of the most effective communications platform concept and subsequent creative development.

Populations of Interest: QIPT online testing sample aims to reflect the general U.S. population and a sample of the HTC audience, to hear a broad range of perspectives. HTC will be a separate sample of people who have elevated risk of nonresponse.

Timeline: Tentative fielding dates are October 29-November 2, 2018.

Language: English only

Method: QIPT online testing respondents will be selected on a non-probability basis since generalizability to the entire population is not necessary, as this is an experiment. According to the Association for Public Opinion Research (AAPOR), a non-probability based sample occurs when “a sample is chosen without specifying the target population and without knowing the probabilities of selecting a given respondent.”² In other words, the “sample” consists of people who are in the survey panel database and decide to participate. We will establish soft quotas based on demographic characteristics to ensure a broad range of perspectives from the online community.

To maximize feasibility, respondents will be recruited from a variety of commercially available online survey panels, who recruit U.S.-based audiences and specialize in demographic sub-groups that are of interest to the study. Participation in online testing is opt-in and voluntary— that is, participants will be individuals who have already opted in to survey panels and are familiar with the online testing process. QIPT online testing will include two samples to be analyzed separately: general population and HTC.

Previous Census Bureau research has identified a number of factors that may be associated with nonresponse, which will be used to determine eligibility for the HTC sample. The combination of those factors lowers the response rates for individuals (Word, 1997). We will operationalize the risk of nonresponse to the census using a point system for eight characteristics that have been identified in previous Census Bureau research (Erdman & Bates, 2017). The risk factors are:

• Rents home

• Households with young children (ages 4 years old or younger)

• Low household income (less than $35,000/year)

• Large household (more than six people in household)

• Moved in the past year

• Female householder, no husband

• Low education (less than a high school diploma)

• Multi-unit or mobile home structure

Each risk variable will be included as a screening question. Potential HTC respondents will receive a “priority audience point” for each self-response risk factor or inclusivity characteristic they demonstrate (e.g., one point for living in a household with young children). To be eligible, participants must receive a minimum of two “priority audience” points. These points may come by way of a combination of risk factors to ensure that QIPT findings represent a range of views within the HTC audience. We will use a two-point threshold for eligibility to maximize sample, as those with hard-to-count traits are less likely to be represented in online panels.

Online testing will use a sequential monadic technique, where respondents will be exposed to all three platform ideas one at a time. To mitigate order bias, we will assign the sample into three groups, among which the order of the first idea is randomized.

The online testing questionnaire will explore the following topics:

• Awareness and familiarity with the 2020 Census

• Individual assessment of three unique platform stimuli

Incentives: We will enlist several nonprobability panel providers to recruit and distribute invitations to participate in the online testing questionnaire. Panels incentivize their panel members to complete studies via a points-based incentive system. Panel members redeem points earned over time for a variety of prizes or cash incentives. The average range for the points-based incentive value is $0.75-$3.00 for diverse mass sample, and $3-$5 for the HTC sample. All incentives are distributed upon completion via virtual currency in the form of a correlated point value.

Length of Interview: The online testing questionnaire is approximately 15 minutes in length, after a five-minute eligibility screener to determine if a potential respondent is to be included in the general population or HTC sample. The incidence rate for general population sample is 60%, and the incidence rate for HTC sample is 30%. Thus, the estimated maximum total respondent burden for this study is 741.67 hours, inclusive of screening (see Burden Hour Calculation below).

Enclosures: The following is enclosed:

• Appendix A: Online Testing Questionnaire

QIPT Focus Groups

Populations of Interest: We will conduct focus groups with audiences that are HTC and unreachable online due to their HTC nature. Focus groups will gather insights from specific hard-to-reach populations not reachable in online panels, including Alaska Native, American Indian, and Native Hawaiian and Pacific Islander (NHPI), and specific segments of Asian and Hispanic populations. Because of timeline limitations, all focus groups will be held in English.

We will conduct 18 focus groups with the audiences described in Table 1:

Table 1. QIPT Focus Group Audiences

Description	Focus Groups
Alaska Native	2 focus groups
American Indian	6 focus groups
Native Hawaiian and Pacific Islander	4 focus groups
Asian	2 focus groups
Hispanic	2 focus groups
Rural	2 focus groups
Note: Focus groups will include participants with a mix of ages, genders, and education levels.

Focus groups will be conducted across nine proposed locations. These locations were selected in collaboration with Census Bureau subject matter experts (SMEs) and multicultural partner advertising agencies that are part of the contractor team in order to provide sufficient concentration of the audience targeted for particular focus groups. The locations also provide geographic diversity. Other factors considered in location selection include the mix of country-of-origin demographics for certain audiences, the potential for participation from Census Bureau regional offices for observation, and minimizing travel costs for the government.

The proposed locations are identified in Table 2.

Table 2. Proposed Focus Group Locations

Location	Audiences (Groups)
Fairbanks, AK	Alaska Native
Phoenix, AZ	American Indian
Portland, OR	American Indian
Saulte Ste. Marie, MI	American Indian
Honolulu, HI	Native Hawaiian and Pacific Islander
Seattle, WA	Native Hawaiian and Pacific Islander
San Gabriel Valley / Pasadena, CA	Asian
Houston, TX	Hispanic
Polk County, MI	Rural

Timeline: Tentative fielding dates are November 12-19, 2018.

Method: Focus groups provide a way to include insights from HTC audiences, who by nature are hard-to-reach and unavailable in online panels. To be eligible for participation, potential participants are screened using a screening questionnaire (see Appendix C for Audience Screener Questionnaires). While each audience has specific screening criteria, all participants across all audiences must meet three overarching criteria to be eligible. First, participants will be screened to be at risk for nonresponse (HTC criteria – see above description and citation in Online Testing section), and eligibility will be contingent upon risk for nonresponse. Second, participants must meet soft quotas for characteristics like gender, age, and education to ensure that there is an appropriate mix of people within each focus group (see Appendix B for Quota Tables). Finally, participants must have not participated in any other focus group session in the past three months. Eligible participants must also be willing to contribute to the research and must be available to attend the focus group in person at a specific location on a particular date and time. Appendix B includes Audience Criteria and Quota Tables for each audience, while Appendix C includes each audience-specific screener questionnaire.

Within each of the specific audiences for focus groups, we will seat participants who have elevated risk of nonresponse, or who are HTC - demographic, family composition, or housing tenure factors that are associated with lower response rates - and to hear a broad range of perspectives. Participants must receive a minimum of two “priority audience” points to be eligible for participation in any of the audience focus groups. That is, only potential participants with a priority audience score of three or more will be scheduled to ensure we are reaching the target audiences of this phase of research.

In addition to establishing eligibility rules based on priority audience factors, we have established soft quotas for focus groups to recruit diverse subpopulations. Soft quotas help ensure that the study sample represents individuals from a variety of demographic backgrounds. Census Bureau SMEs and the ICC contractor multicultural agencies provided input for the soft quota proportions for this research study (see Appendix B for Quota Tables). The team will evaluate the confirmed recruiting list and determine throughout the data collection process if any subgroup is over- or under-represented based on the study’s soft quotas. We will adjust recruitment strategies accordingly, and every effort will be made to ensure a strong mix of participants in each group.

We will recruit 16 participants to seat eight per group for AIAN and NHPI and 14 to seat eight for Hispanic, Asian, and Rural groups. These numbers take into consideration the recruiting efforts and show rates from the 2020 Census Barriers, Attitudes, and Motivators Study focus groups. The number recruited for each focus group may fluctuate by audience, following guidance from Census Bureau’s SMEs. QIPT focus groups will rely on an on-the-ground, in-the-community based recruitment strategy, in which the multicultural agencies tap into their network of community and organizational leaders to disseminate information and recruit potential participants.

Each focus group will be approximately 90 minutes in duration. Upon arrival at the facility, participants will receive a focus group information sheet and informed consent form (enclosed as Appendix F). A member of the research team will be available to answer any questions prior to the start of the discussion. Discussion will be semi structured and aligned with a discussion guide. With guidance from contractor researchers, multicultural specialist agencies, and Census Bureau SMEs, we have developed a discussion guide for the focus groups that covers key topics and identifies areas to probe for further information. Discussions will be facilitated by professionally trained moderators with demonstrated expertise in focus group facilitation and with specific audience groups.

The topics for discussion are as follows:

• Section A: Introduction and Icebreaker (10 min.)

• Section B: Warm Up (5 min.)

• Section C: Individual Platform Review (65 min.)

• Section D: Conclusion (10 min.)

The complete discussion guide and screenshots for group activities are enclosed (see Appendices D and E respectively).

Incentives: We are requesting that OMB authorize the Census Bureau to approve an incentive amount of $75.

Length of Interview: Focus groups will last approximately 90 minutes. For each group, we will recruit twelve participants to seat six participants. Thus, the estimated maximum total respondent burden for this study is 676 hours, inclusive of screening and informed consent activities (see Burden Hour Calculation below).

Enclosures: The following are enclosed:

• Appendix B: Focus Group Screening Questionnaires by Audience

• Appendix C: Focus Group Audience Criteria with Quota Tables

• Appendix D: Focus Group Discussion Guide

• Appendix E: Focus Group Activities

• Appendix F: Focus Group Consent Form

• Appendix G: Example Focus Group Recruitment Advertisement

Focus Groups ($75 incentive):

BURDEN HOUR COMPUTATION Number of respondents (X) estimated response or participation time in minutes (/60) = annual burden hours:

Type/Category of Respondent	No. of Respondents	Participation Time (minutes)	Burden (hours)
Screened Potential Participants
Questionnaire	2,760	5	230
Consent		5	230
Focus Group Participants
Maximum Focus Group Participants	144	90	216
Total Burden (Screened Participants and Focus Group Participants)			676

Assumption Notes:

• Based on 2020 CBAMS recruitment and show rate data, we recommend recruiting 16 to seat 8 participants for AIAN (8 groups) and NHPI (4 groups) focus groups, and recruiting 14 to seat 8 participants for Asian (2 groups), Hispanic (2 groups), and Rural (2 groups) focus groups (276 total recruits).

• For every 10 potential participants contacted; 1 will be interested, eligible, and available to participate in the focus group.

Online Testing Questionnaire:

BURDEN HOUR COMPUTATION Number of respondents (X) estimated response or participation time in minutes (/60) = annual burden hours:

Type/Category of Respondent	No. of Respondents	Participation Time (minutes)	Burden (hours)
Screened Potential Respondents
Screening – General Population	2,500	5	208.33
Screening – Hard-to-Count	1,000	5	83.33
Completed Respondents
Maximum Completes – General Population	1,500	15	375
Maximum Completes – Hard-to-Count	300	15	75
Total Burden (Screened Participants and Completed Respondents)			741.67

Assumption Notes:

• For every 10 potential respondents invited, 6 will be interested, eligible, and able to complete the questionnaire as a general population respondent.

• For every 10 potential respondents invited, 3 will be interested, eligible, and able to complete the questionnaire as a hard-to-count respondent.

Point of Contact: The contact person for questions regarding data collection design of this research is listed below:

Monica Vines

Researcher

Center for New Media and Promotion (CNMP)

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-8813

[email protected]

1 Erdman, C., & Bates, N. (2017). The Low Response Score (LRS): A Metric to Locate, Predict, and Manage Hard-to-Survey Populations. Public Opinion Quarterly, 81(1).

2 American Association for Public Opinion Research (2007). “Why Sampling Works.” Retrieved from: https://www.aapor.org/Education-Resources/For-Researchers/Poll-Survey-FAQ/Why-Sampling-Works.aspx

8