Knowledge Panel Recruitment Procedures

KnowledgePanel Recruitment Procedures

When GfK Custom Research’s Government and Academic Unit (formerly Knowledge Networks) began recruiting in 1999, the company established the first online research panel (now called KnowledgePanel^®) based on probability sampling covering both the online and offline populations in the U.S. Panel members are recruited through national random samples, originally by telephone and now almost entirely by postal mail. Households are provided with access to the Internet and hardware if needed.  Unlike Internet convenience panels, also known as “opt-in” panels,  that include only individuals with Internet access who volunteer themselves for research, KnowledgePanel recruitment uses dual sampling frames that include both listed and unlisted telephone numbers, telephone and non-telephone households, and cell-phone-only households, as well as households with and without Internet access.  Only persons sampled through these probability-based techniques are eligible to participate on KnowledgePanel.  Unless invited to do so as part of these national samples, no one on their own can volunteer to be on the panel. 

RDD and ABS Sample Frames

KnowledgePanel members today could have been recruited by either the former random digit dialing (RDD) sampling or the current address-based sampling (ABS) methodologies. In this section, we will describe the RDD-based methodology; the ABS methodology is described in a separate section below. To offset attrition, multiple recruitment samples are fielded evenly throughout the calendar year.

KnowledgePanel recruitment methodology has used the quality standards established by selected RDD surveys conducted for the Federal government (such as the CDC-sponsored National Immunization Survey).

GfK employed list-assisted RDD sampling techniques based on a sample frame of the U.S. residential landline telephone universe.  For purposes of efficiency, GfK excludes only those banks of telephone numbers (a bank consists of 100 numbers) that had fewer than two directory listings.  Additionally, an oversampling was conducted within a stratum of telephone exchanges that had high concentrations of African American and Hispanic households based on Census data. Note that recruitment sampling is done without replacement, thus numbers already fielded do not get fielded again. 

A telephone number for which a valid postal address can be matched occurred in about 67-70% of each sample.  These address-matched cases were all mailed an advance letter informing them that they had been selected to participate in KnowledgePanel. For purposes of efficiency, the unmatched numbers were most recently under-sampled at a rate of 0.75 relative to the matched numbers. Both the minority oversampling mentioned above and this under-sampling of non-address households are adjusted appropriately in the panel’s weighting procedures.

Following the mailings, telephone recruitment by trained interviewers/recruiters begins for all sampled telephone numbers.  Telephone numbers for cases sent to recruiters were dialed for up to 90 days, with at least 14 dial attempts for cases in which no one answers the phone, and for numbers known to be associated with households. Extensive refusal conversion was also performed.  The recruitment interview, about 10 minutes in length, begins with informing the household member that the household had been selected to join KnowledgePanel.  If the household does not have a computer and access to the Internet, the household member is told that in return for completing a short survey weekly, the household will be provided with free monthly Internet access and a laptop computer (in the past, the household was provided with a WebTV device).  All members of the household were enumerated, and some initial demographic and background information on prior computer and Internet use was collected.

Households that informed recruiters that they had a home computer and Internet access were asked to take KnowledgePanel surveys using their own equipment and Internet connection.  Incentive points per survey, redeemable for cash, are given to these “PC” (personal computer) respondents for completing their surveys. Panel members provided with a laptop computer and free Internet access do not participate in this per-survey points-incentive program.  However, all panel members do receive special incentive points for select surveys to improve response rates and/or for all longer surveys as a modest compensation for the extra burden of their time and participation.

For those panel members receiving a laptop computer, each unit is custom-configured prior to shipment with individual email accounts so that it is ready for immediate use by the household.  Most households are able to install the hardware without additional assistance, although GfK maintains a toll-free telephone line for technical support. The KnowledgePanel Call Center contacts household members who do not respond to e-mail and attempts to restore both contact and participation. PC panel members provide their own e-mail addresses, and we send their weekly survey invitations to that e-mail account.

All new panel members receive an initial survey for the dual purpose of welcoming them as new panel members and introducing them to how online survey questionnaires work.   New panel members also complete a separate profile survey that collects essential demographic information such as gender, age, race, income, and education to create a personal member profile. This information can be used to determine eligibility for specific studies and is factored in for weighting purposes. Operationally, once the profile information is stored, it does not need to be re-collected as a part of each and every survey.  This information is also updated annually for all panel members. Once new members have completed their profile surveys, they are designated as “active,” and considered ready to be sampled for client studies.  [Note: Parental or legal guardian consent is also collected for the purpose of conducting surveys with teenage panel members, aged 13 to17.]

Once a household is recruited and each household member’s e-mail address is either obtained or provided, panel members are sent survey invitations linked through a personalized e-mail message (instead of by phone or postal mail). This contact method permits surveys to be fielded quickly and economically, and also facilitates longitudinal research. In addition, this approach reduces the burden placed on respondents, since e-mail notification is less intrusive than telephone calls and allows research subjects to participate in research when it is convenient for them. 

Address-Based Sampling (ABS) Methodology

When panel recruitment began in 1999, the conventional opinion among survey experts was that probability-based sampling could be carried out cost effectively through the use of a national RDD samples. The RDD landline frame at the time allowed access to 96% of U.S. households.  This is no longer the case. In 2009, the use of the ABS sample frame for panel recruitment was introduced to reflect the real changes in society and telephony over recent years. Those changes that have reduced the long-term scientific viability of landline RDD sampling methodology are as follows: declining respondent cooperation in telephone surveys as reflected in “do not call” lists, call screening, caller-ID devices, and answering machines; dilution of the RDD sample frame as measured by the working telephone number rate; and finally, the emergence of cell phone-only households (CPOHH) because such households are excluded from the RDD frame because they have no landline telephone.

According to the Centers for Disease Control and Prevention (January-June 2010), approximately 28.6% of all U.S. households cannot be contacted through RDD sampling—26.6% as a result of CPOHH status and 2% because they have no telephone service whatsoever. Among some age segments, the RDD non-coverage would be substantial: 40% of young adults, ages 18–24, reside in CPOHHs, 51% of those ages 25–29, and 40% of those ages 30–34.¹

After conducting an extensive pilot project in 2008, the decision was made to move toward address-based sample (ABS) frame in response to the growing number of cell-phone- only households that are outside the RDD frame. Before conducting the ABS pilot, we also experimented with supplementing its RDD samples with cell-phone samples. However, this approach would was not cost effective—and raised a number of other operational, data quality, and liability issues (for example, calling cell phones while respondents were driving).

The key advantage of the ABS sample frame is that it allows sampling of almost all U.S. households. An estimated 97% of households are “covered” in sampling nomenclature. Regardless of household telephone status, those households can be reached and contacted through postal mail. Second, the KNABS pilot project revealed several additional advantages beyond expected improvement in recruiting adults from CPOHHs:

• Improved sample representativeness for minority racial and ethnic groups

• Improved inclusion of lower educated and low income households

• Exclusive inclusion of the fraction of CPOHHs that have neither a landline telephone nor Internet access (approximately four to six percent of US households).

ABS involves probability-based sampling of addresses from the U.S. Postal Service’s Delivery Sequence File. Randomly sampled addresses are invited to join KnowledgePanel through a series of mailings and, in some cases, telephone follow-up calls to non-responders when a telephone number can be matched to the sampled address. Operationally, invited households have the option to join the panel by one of several ways:

• Completing and returning a paper form in a postage-paid envelope,

• Calling a toll-free hotline maintained by GfK, or

• Going to a dedicated KnowledgePanel web site and completing an online recruitment form.

After initially accepting the invitation to join the panel, respondents are then “profiled” online by answering key demographic questions about themselves. This profile is maintained through the same procedures that were previously established for RDD-recruited panel members. Respondents not having an Internet connection are provided a laptop computer and free Internet service. Respondents sampled from the ABS frame, like those sampled from the RDD frame, are offered the same privacy terms and protections that we have developed over the years and that have been reviewed by dozens of Institutional Review Boards.

Large-scale ABS sampling for KnowledgePanel recruitment began in April 2009. As a result, sample coverage on KnowledgePanel of CPOHHs, young adults, and non-whites has been increasing steadily since that time.

Because KnowledgePanel members have been recruited from two different sample frames, RDD and ABS, several technical processes were implemented to merge samples sourced from these frames. This approach preserves the representative structure of the overall panel for the selection of individual client study samples. An advantage of mixing ABS frame panel members in any KnowledgePanel sample is a reduction in the variance of the weights. ABS-sourced samples tend to align more closely to the overall demographic distributions in the population, and thus the associated adjustment weights are somewhat more uniform and less varied. This variance reduction efficaciously attenuates the sample’s design effect and confirms a real advantage for study samples drawn from KnowledgePanel with its dual frame construction.

1 Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, January–June 2010. National Center for Health Statistics. December 2010. Available from: http://www.cdc.gov/nchs/nhis.htm.