Class Action Study Supporting Statement FTC Revised Per OMB 09_27_17

Supporting Statement for a Paperwork Reduction Act
Submission to OMB
Class Action Notice Study
OMB Control No. 3084-XXXX
A. JUSTIFICATION
(1)

Necessity for Collecting the Information

The Class Action Fairness Project strives to protect injured consumers from settlements
that provide them with little to no benefit and to protect businesses from the incentives such
settlements may create for the filing of frivolous lawsuits. As part of that initiative, the FTC is
studying the effectiveness of class action settlement notice campaigns. While class action
settlement notices historically have taken the form of long documents mailed to each class
member, class action administrators increasingly have used email notifications, especially for
large, national classes. Given the increasing use of these types of notices, the Commission plans
to study what variables affect class members’ willingness to open and read an email notice
informing them of a class action settlement and whether consumers understand such notices and
the options they provide.
(2)

Use of the Information

The FTC proposes to conduct an Internet-based consumer research study to explore
consumer perceptions of class action settlement notices. Based on marketplace trends identified
through review of class action settlements and discussions with notice administrators, the study
will focus on notices sent to individual consumers via email. Using a treatment-effect
methodology, the study will examine whether variables, such as the email address of the sender
and the email subject line, affect respondents’ perception of and willingness to open an email
notification. The proposed study will also gauge consumer comprehension of the options
conveyed by the notice, including participating in the settlement and the implications of a
consumer’s choice whether to participate or do nothing. The FTC plans to study notices derived
from actual notices sent to class members in various nationwide class action settlements as well
as streamlined versions designed by the FTC. We plan to use the study results, along with other
information, such as public comments, to guide the FTC’s Class Action Fairness Project.
Having considered the costs and benefits of various data collection methods, FTC staff
has concluded that an Internet panel with nationwide coverage will provide the most efficient
way to meet the research objectives. We will draw participants from an Internet panel
maintained by a commercial firm. All participation will be voluntary. While the results will
not be generalizable to the U.S. population, comparing responses across various treatments
should provide useful insight into consumer understanding of the claims and statements made in

the class action settlement emails.1 The FTC has contracted with Great Lakes Marketing, a
consumer research firm with substantial experience assessing consumer communications via the
Internet and other alternative protocols, to administer the Internet study.
(3)

Consideration of the Use of Information Technology to Reduce Burden

Consistent with the aims of the Government Paperwork Elimination Act, 44 U.S.C. §
3504 note, the proposed study will use the Internet for data collection. The Internet was
selected as the means to collect data, in part, to minimize burden on respondents and to collect
data in a cost-efficient manner. For example, people who choose to participate in the study will
be able to both view the questionnaire and submit their responses via computer, at a time and
location of their choosing.
(4)

Effort to Identify Duplication

The FTC has issued a prior request for comment on the above-noted issues, including a
request for relevant study findings, published in the Federal Register (as described in Part A.8(a)
below). The FTC has reviewed the relevant academic literature as well as industry sources for
studies of consumer perception of the claims at issue. We know of no other studies that provide
the specific information sought here.
(5)

Efforts to Minimize Burden on Small Organizations
Not applicable. Only individual consumers are being surveyed.

(6)

Consequences of Not Conducting the Collection of Information

If this information is not collected, the Commission may lack sufficient information to
address important issues and to target more effectively future law enforcement and consumer
education actions. The study’s scope and burden have been reduced as much as possible, short
of sacrificing the value of the information to be collected.
(7)

Circumstances Requiring Collection Inconsistent With Guidelines

The collection of information in the proposed study is consistent with all applicable
guidelines contained in 5 CFR § 1320.5(d)(2).

1

See FED. TRADE COMM’N, APPLIANCE LABELING RULE: PROPOSED RULE, 72 FR 6836, 6838-39,
6841, 6843-51, 6854 (Feb. 13, 2007) (codified at 16 CFR pt. 305) (discussing results of
consumer research involving treatment groups).
2

(8)

Public Comments/Consultation Outside the Agency
a.

Public Comments

As required by section 3506(c)(2) of the PRA, the FTC published a notice seeking public
comment on the proposed collection of information. See 80 FR 25676 (May 5, 2015). The
FTC received two comments, neither of which substantively addressed the proposed survey:2
Class Action Trolls, Inc. requested updates as information becomes available on issues
related to this study. Information will be made public as appropriate on FTC.gov and through
other means.
An Individual Commenter expressed support for the FTC’s work in this area.
b.

Consultation Outside the Agency

FTC staff has consulted with Great Lakes Marketing and the FTC’s survey consultant,
Manoj Hastak, Ph.D., Associate Professor of Marketing at American University’s Kogod
College of Business Administration in Washington, DC. FTC staff has also consulted with
various class action administrators.
(9)

Payments and Gifts to Respondents

Great Lakes Marketing will provide incentives to members of its Internet panel who
participate in the online study. Specifically, these incentives are similar to frequent flyer miles
that can be redeemed for rewards. The total dollar value of the points that will be awarded to a
combined 8,100 respondents to the pretest and Internet questionnaire will be $17,820 (or an
average of $2.20 per person).
(10) & (11) Assurances of Confidentiality/Matters of a Sensitive Nature
Responses to the study questionnaire will not include any identifying information, and
the FTC will not receive any information about the identity of individual respondents. The
respondent’s email address is stored in a database, and a unique ID number is then assigned to
that email address. Respondents’ responses will be stored in a separate database that includes
the unique ID number but not the underlying email address. The contractor will be prohibited
from using any email address to retrieve response records, even if records may be maintained
2

The Commission received comments from the Class Action Trolls, Inc. (#615-00004) and one
individual commenter (#615-00005), available at
http://www.ftc.gov/policy/public-comments/initiative-615.
3

and retrieved by ID number. Because the Privacy Act of 1974 may be implicated, the
contractor will provide all survey respondents with appropriate notice of the authority,
purpose, and use of their information and, pursuant to the nondisclosure agreement and other
restrictions set forth in the FTC’s survey contract, the data shall remain protected from public
disclosure to the extent permitted by law. See, e.g., Freedom of Information Act Exemption 6,
5 U.S.C. 552(b)(6).
The contractor also will be required to have sufficient procedures in place to prevent
unauthorized access to respondent information, such as: storing personally identifying
information on separate servers from questionnaire response data; using firewalls to secure
those servers; and maintaining audit records of log-ins, file accesses, and security incidents, if
any. Moreover, the proposed data collection will not include sensitive questions.
(12)

Estimated Annual Hours Burden

As before, staff estimates that respondents to the Internet questionnaire will require, on
average, approximately 20 minutes to complete. Staff also will pretest the questionnaire with
approximately 100 respondents to ensure that all questions are easily understood. Allowing for
an extra three minutes for questions unique to the pretest, staff estimates that the pretest will take
approximately 23 minutes. Cumulatively, those completing the questionnaire will require
approximately 2,667 hours (8,000 persons x 20 minutes each), and those completing the pretest
will require approximately 38 hours (100 respondents x 23 minutes each).
Staff also anticipates that some respondents will not complete the questionnaire. Staff
projects that those who will prematurely end the process will do so in less than one minute.
Great Lakes anticipates that 60 percent of those invited to participate in the study will complete
the questionnaire. Accordingly, Great Lakes might contact as many as 13,333 people to achieve
the study’s goal of surveying 8,000 respondents, which would result in an additional 89 hours
total. [(13,333 total contacts – 8,000 people completing the questionnaire) x 1 minute each)].
For the pretest, Great Lakes estimates an additional 67 people will prematurely end the process,
which totals an additional 1 hour [(167 total contacts – 100 people completing the pretest) x 1
minute each)]. Cumulatively, then, complete and partial surveying of 13,333 people will total
about 2,756 hours and complete and partial pretesting will total 39 hours, for an overall total of
2,795 hours.
(13)

Estimated Annual Cost Burden

The cost per respondent should be negligible. Participation is voluntary and will not
require any labor expenditures by respondents. There are no capital, start-up, operation,
maintenance, or other similar costs to the respondents.

4

(14)

Estimated Cost to the Federal Government

The total cost to the Federal government for the information collection will be
approximately $124,387. Staff projects it will cost $66,235 to pay Great Lakes Marketing to
review the study questionnaire, program it into an online format, execute the pretest and online
study, and analyze and report the data. The estimated cost of FTC staff time is $58,152.3 This
is necessarily an estimate because several factors in this calculation may vary, including the
amount of staff involved and the actual time required.
(15)

Program Changes or Adjustments
Not applicable. This is a proposed new study.

(16)

Plans for Tabulation and Publication

The projected duration of the information collection is approximately eight weeks. The
FTC staff will consider publication of the results following the project’s completion.
(17)

Display of Expiration Date for OMB Approval

The FTC will display the expiration date alongside the assigned OMB control number on
the first screen of the pretest and on the first screen of the actual study questionnaire.
(18)

Exceptions to Certification
Not applicable.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
(1)

Description of Study Design, Respondent Universe, and Sampling Methodology

Because class action lawsuits are so pervasive, it is likely that a wide segment of the U.S.
population has received notice of a class action settlement or will receive such a notice in the
future. Therefore, the research targets a broad sample of consumers. The study will use a
3

This estimate is based on 675 hours of staff time (Attorneys: 10 hours per week x 21 weeks x 2
attorneys = 420 hrs. x $ 90.30 per hr. = $37,926; Economist: 10 hours per week x 15 weeks x 1
economist = 150 hrs. x $95 per hr. = $14,250; User Experience Designer: 5 hours per week x 21
weeks x 1 specialist = 105 hrs. x $56.91= $5,976).

5

nationwide Internet panel to identify potential respondents, and the questionnaire will be
administered online. The FTC will contract with Great Lakes Marketing, which has substantial
experience assessing consumer communications using Internet protocols and alternative
protocols. By randomly assigning respondents across treatment groups, the study will provide
useful information about whether consumer perception of the class action settlement notice
studied differs across the various inbox and email conditions the study will examine.
a.

Study Design

The study’s goal is to gather information on consumer perception of emailed class action
settlement notices. Specifically, the study seeks to evaluate whether various email
characteristics—including the sender address, the subject line, and the email body—influence
respondents’ understanding of the class action claims process.
The study will consist of two parts. The first part will gauge the importance of various
inbox characteristics (such as the sender address and the subject line), and the second part will
address the importance of various email body characteristics (such as the format of the notice
and the presence of a court seal in the email). The study randomly will assign each respondent
to a sender, subject line, and email body cell. Each respondent will view only one inbox
(consisting of a sender and subject line combination) and one email body. FTC staff will
compare responses across cells to determine if consumer perception of the class action settlement
notice studied is sensitive to differences in the scenario presented.
The first part of the study (the “inbox” part) seeks to understand if various inbox
characteristics influence: (1) respondents’ stated likelihoods of opening a fictitious class action
settlement notice email; and (2) their perceptions of the type of information contained in the
email. Respondents will be presented with a static image of an inbox, in the format of a Gmail
inbox. The inbox will contain a total of 10 inbox entries, one of which is a class action
settlement notice email about a fictitious company named “Sonoro Technologies.” The nine
other inbox entries also concern fictitious companies and have various subject lines, such as
promotional subject lines and subject lines indicating online order confirmations. Respondents
will be instructed to assume that the static image of the inbox contains emails from companies
they have done business with. The position of the class action settlement notice email will be
randomly assigned to mitigate any bias resulting from the email’s location within the inbox: for
half of the participants it will appear in the third position, and for the other half, it will appear in
the eighth position. The inbox part of the study will test 18 sender address / subject line
scenarios. Specifically, it will test 3 sender addresses (“Sonoro,” “SonoroJetSettlement,” and
“[email protected]”) across 3 subject line prefixes (“Notice of Class Action Settlement,”
“Notice of Refund,” and “Lavin v. Sonoro Technologies Class Action Settlement”) and two
subject line refund information conditions (presence or absence of the potential refund amount).4
4

FTC staff selected “Jet” as the fictitious name of the product and “Lavin” as the fictitious name
6

Each respondent will see only one inbox, i.e., one sender address / subject line combination.
The portion of the questionnaire relevant to the inbox part of the study will ask
respondents which emails, if any, they would be likely to open, as well as ask respondents to
determine the nature of the email. This series of questions seeks to understand the importance
of sender address and subject line for respondents’ decisions to open an emailed class action
settlement notice, and, relatedly, for respondents’ takeaway of the type of information that the
email might contain. FTC staff will compare responses across scenarios to test for differences
in the proportion of respondents who indicate they would be likely to open the class action
settlement notice email and the proportion of respondents who indicate that it provides
information on a class action settlement or a refund.
The second part of the study (the “email body” part) seeks to understand if various email
body characteristics influence: (1) respondents’ understanding of the information contained in a
fictitious class action settlement notice; (2) their comprehension of the steps required for
receiving a refund; and (3) their perceptions of the claims and refund process. After completing
the inbox part of the study, the study will instruct respondents to assume that they have opened
the class action settlement notice email. Respondents will be presented with a static image of an
email body, depicting a class action settlement notice related to the company “Sonoro
Technologies.” The email body will appear in the format of the Gmail interface. The sender
and subject line will be blurred so that these components of the email do not drive responses in
the email body portion of the study. This part of the study will test six email body scenarios.
Specifically, it will test three email formats across two court seal conditions. The three email
formats are: (1) a “typical” emailed class action settlement notice derived from notices sent to
class members in various nationwide class action settlements; (2) a condensed settlement notice
developed by FTC staff; and (3) a further streamlined settlement notice developed by FTC staff.
The two court seal conditions are: (1) the presence of the seal, and (2) the absence of the seal.
The court seal, when presented, will be outlined in blue and will be displayed in combination
with the text “This is a LEGAL NOTICE approved by the UNITED STATES DISTRICT
COURT FOR THE CENTRAL DISTRICT OF CALIFORNIA” along with the fictitious case
name and docket number. Each respondent will see only one email body, which will be
removed from the respondent’s view when he or she is answering questions.
The portion of the questionnaire relevant to the email body part of the study will ask
respondents to determine the nature of the email and will ask respondents about actions they
might need to take to receive the refund, according to the email. These actions include incorrect
actions—such as taking no action, filing a customer service complaint with the company, and
hiring a personal attorney—as well as the correct action, filling out the claims form. FTC staff
will compare responses across email body cells to gauge whether there are significant differences
of the plaintiff involved in the class action lawsuit against Sonoro Technologies. The fictitious
refund amount is $100.
7

in the proportion of respondents who indicate that the email provides information on a class
action settlement or a refund and the proportion of respondents who comprehend that they must
fill out a claims form to receive a refund. The questionnaire also will ask respondents about
their personal opinions of the claims and refund process according to the email they viewed,
including their impressions of the likelihood of receiving the refund, the ease of meeting the
requirements to receive the refund, the proportion of refund applicants likely to receive refunds,
and the time it will take to apply for the refund. Again, FTC staff will compare responses across
email body cells to assess whether there are significant differences in respondent takeaway of the
expected costs and expected benefits of applying for the refund by filling out a claims form.
Finally, FTC staff will compare responses across the 108 sender address / subject line /
email body cells to assess the interactive effects of the various inbox and email body
characteristics on consumer perception of the class action settlement notice studied.
b.

Sampling Frame

After considering the costs and benefits of various data collection methods, FTC staff has
concluded that the most efficient way to collect the data needed to meet the research objectives
within a feasible budget is to employ an Internet panel with nationwide coverage. The FTC has
contracted with Great Lakes Marketing to design an Internet study that, while not technically
representative of the nation as a whole, nonetheless reflects the views of a broad population.
This study’s sampling frame consists of members of the contractor’s Internet panel.
Great Lakes Marketing is contracting with a provider of an Internet panel consisting of
approximately 6 million individuals drawn from throughout the country, derived from a series of
convenience sampling procedures, rather than true probability sampling. As also discussed in
Part A of this Supporting Statement, FTC staff, in consultation with its contractor, has
determined that a screening sample of up to 13,500 respondents will be needed to ensure 8,100
completed responses – 100 for the pretest and 8,000 for the main study.
FTC staff understands that an Internet panel is not a nationally representative probability
sample and, therefore, recognizes that the study results will not be projectable to the general
population. Accordingly, the study will focus on comparing respondent interpretations across
various inbox and email body characteristics and will not seek to project the percentage of the
population at large that holds a particular view. FTC staff will consider this point further in the
ensuing analysis of the data and note this limitation in any discussion of the findings.
Nevertheless, given that the study is part of a broader policy analysis considering information
from this as well as other relevant sources, the Internet panel should be suitable for the study’s
purpose.

8

c.

Sampling Methodology

The FTC has determined that a sample size of 8,000 consumers is appropriate for the
study based on several considerations, including the funds available for the study, the cost of
different sample sizes, the number of inbox and email body cells into which the sample will be
divided, and a power analysis.
The contractor will randomly assign each of the 8,000 respondents to one of 108 sender
address / subject line / email body cells. FTC staff will analyze differences in responses across
the 108 cells. Additionally, FTC staff will “collapse” the data to various inbox
characteristic-levels (resulting in 3 cells based on sender address, 3 cells based on the prefix of
the subject line, 2 cells based on refund information condition, or 18 cells based on all interactive
effects of the inbox characteristics). Similarly, FTC staff will collapse the data to various email
body characteristic-levels (resulting in 3 cells based on email format, 2 cells based on presence
of the court seal, or 6 cells based on all interactive effects of the email body characteristics). In
other words, in this 3X3X2X3X2 between-subjects factorial design, FTC staff will analyze main
effects and interactive effects across the various inbox and email body conditions.
For simplicity, the power analysis below is conducted for the 108-cell design, which
accounts for interactions across all characteristics in both parts of the study, as well as for the
18-cell design related to the inbox part of the study and the 6-cell design related to the email
body part of the study. With a total sample size of 8,000, each cell in the 108-cell design will
contain approximately 74 participants; each cell in the 18-cell design will contain approximately
444 participants; and each cell in the 6-cell design will contain approximately 1,333
participants.5 The primary goal of the study is to gauge whether the proportion of various
responses—e.g., responses that indicate that respondents would be likely to open the class action
email, or responses that indicate that respondents understand that they need to fill out a claims
form to receive a refund—differs significantly across scenarios.
With a sample size of 74 per cell, there is better than an 80 percent probability of finding
a statistically significant difference using a two-tailed test with a 5 percent significance level if
the true difference between the proportions of a particular response in the baseline and
comparison scenarios is at least 23 percentage points.
With a sample size of 444 per cell, there is better than an 80 percent probability of
finding a statistically significant difference using a two-tailed test with a 5 percent significance
5

A fewer number of cells are required to study main effects, e.g., only 3 cells are required to
analyze differences in responses across the various sender address conditions. This implies that
power for the main effects analysis will be higher than for the interactive effects analysis
presented here.
9

level if the true difference between the proportions of a particular response in the baseline and
comparison scenarios is at least 10 percentage points.
With a sample size of 1333 per cell, there is better than an 80 percent probability of
finding a statistically significant difference using a two-tailed test with a 5 percent significance
level if the true difference between the proportions of a particular response in the baseline and
comparison scenarios is at least 6 percentage points.
(2)

Description of the Information Collection Procedures

As discussed more fully in Part A of the Supporting Statement, the FTC has selected
Great Lakes Marketing, a consumer research firm, to recruit 8,000 individuals 18 years of age or
older for the study. The FTC’s questionnaire will include both open-ended and closed-ended
questions to learn how consumers viewing a class action notice email interpret the information
contained in the email.
The procedure for administering the questionnaire will be determined in consultation
with the contractor. The contractor will develop an online program that will allow respondents
to view appropriate questions based on their answers to prior questions. Therefore, an Internet
methodology can be an improvement over mail questionnaires, where respondents can preview
questions and materials that are best asked in a controlled sequence.
The contractor will provide the FTC with raw data as well as tabulated data. The FTC
staff economist will conduct all statistical analyses of the data.
(3)

Methods to Maximize Response Rates/Reliability of Sample Data

The existing Internet panel used for sampling consists of people who have expressed
interest in sharing their opinions via the Internet and who do so regularly. To help maximize the
participation rate, FTC staff and/or the FTC’s contractor will:
·

Design an experimental protocol that minimizes burden (short in length and clearly
written);

·

Test the draft protocol with a pretest of 100 respondents to ensure that the protocol
minimizes burden, and then refine the protocol as appropriate; and

·

Administer the experiment to individuals who have expressed interest in participating in
Internet studies; email reminders to respondents who do not complete the protocol soon
after the original invitation to participate is sent; and provide incentives similar to
frequent flyer miles that can be redeemed for rewards.
10

(4)

Testing of Procedures or Methods Undertaken

Again, staff will pretest the questionnaire in the online format using 100 respondents to
ensure that all questions are easily understood and that the online procedure is sufficient to
generate reliable data. This pretest is also discussed in Part A and is part of the collection of
information for which staff seeks OMB approval.
(5)

Individuals Consulted on the Design of the Study

The study design has been reviewed internally by James A. Kohm, Associate Director of
the Enforcement Division, Bureau of Consumer Protection (202-326-2640); Laura Koss
Assistant Director of the Enforcement Division, Bureau of Consumer Protection (202-326-2890);
Robin Moore, an attorney in the Enforcement Division, Bureau of Consumer Protection
(202-326-2167); Colin MacDonald, an attorney in the Enforcement Division, Bureau of
Consumer Protection (202-326-3192); Tara I. Koslov, Acting Director of the Office of Policy
Planning (202-326-2386); Elizabeth Jex, an attorney in the Office of Policy Planning
(202-326-3273); Janis K. Pappalardo, Assistant Director of the Division of Consumer Protection,
Bureau of Economics (202-326-3380); Tim Daniel, Deputy Assistant Director of the Division of
Consumer Protection, Bureau of Economics (202-326-2928); Shiva Koohi, an economist in the
Bureau of Economics (202-326-3416); Jessica Skretch, a User Experience Designer in the
Division of Consumer and Business Education (202-326-3004); and Manoj Hastak, Ph.D.
(202-326-2613), a faculty member in the Kogod School of Business at American University and
a consumer research consultant with the Bureau of Consumer Protection’s Division of
Advertising Practices. Dr. Hastak has served as a consultant on experimental and survey studies
for the FTC on numerous occasions. The contractor, Great Lakes Marketing Research, is
experienced in conducting statistically rigorous Internet-based studies. The contractor has
conducted numerous studies using its Internet panel in the last five years.

11