SS A v10_4 27 2015

Focus Group Testing to Effectively Plan and Tailor a Communication Campaign about Young Women and Breast Cancer

Submitted under OMB No. 0920-0800

Focus Group Testing to Effectively Plan and Tailor

Cancer Prevention and Control Communication Campaigns

Generic Information Collection

Expiration Date 12/31/2017

Supporting Statement Part A

April 27, 2015

Technical Monitor:

Karena Sapsis

Division of Cancer Prevention and Control

Phone: 770-488-3080

Fax: 770-488-3040

E-mail: [email protected]

4770 Chamblee Tucker Road | Bldg. 107

Chamblee, GA 30341

MailStop F76

Supported by:

Centers for Disease Control and Prevention

National Center for Chronic Disease Prevention and Health Promotion

Division of Cancer Prevention and Control

TABLE OF CONTENTS

A. Justification

A1. Circumstances Making the Collection of Information Necessary 4

A2. Purpose and Use of the Information Collection 7

A3. Use of Improved Information Technology and Burden Reduction 8

A4. Efforts to Identify Duplication and Use of Similar Information 8

A5. Impact on Small Businesses or Other Small Entities 8

A6. Consequences of Collecting the Information Less Frequently 8

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 9

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 9

A9. Explanation of Any Payment or Gift to Respondents 9

A10. Assurance of Confidentiality Provided to Respondents 10

A11. Justification for Sensitive Questions 11

A12. Estimates of Annualized Burden Hours and Costs 11

A13. Estimates of Other Total Annual Cost Burden to Respondents or
Recordkeepers 13

A14. Annualized Cost to the Government 14

A15. Explanation for Program Changes or Adjustments 14

A16. Plans for Tabulation and Publication and Project Time Schedule 14

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate 15

A18. Exemptions to Certification for Paperwork Reduction Act Submissions 15

References………………………………………………………………………………………16

List of Tables

Table A1-A: Materials to be Tested

Table A1-B: Focus Groups, by Audience Segment, Family History of Breast or Ovarian Cancer,

and Location

Table A12-A: Estimated Annualized Response Burden for each Method

Table A12-B: Estimated Annualized Cost to Respondents

List of Attachments

Attachment A. Legislative Authority – EARLY Act Reauthorization

Attachment B1. Discussion Guide for Ashkenazi Jewish Young Women (Family History)

Attachment B2. Materials for Testing

Attachment C1. Discussion Guide for Ashkenazi Jewish Young Women (No Family History)

Attachment C2. Materials for Testing

Attachment D1. Discussion Guide for African American Young Women (Family History)

Attachment D2. Materials for Testing

Attachment E1. Discussion Guide for African American Young Women (No Family History)

Attachment E2. Materials for Testing

Attachment F1. Discussion Guide for Other Young Women (Family History)

Attachment F2. Materials for Testing

Attachment G1. Discussion Guide for Other Young Women (No Family History)

Attachment G2. Materials for Testing

Attachment H. Screening Instrument for Ashkenazi Jewish Young Women

Attachment I. Screening Instrument for African American Young Women

Attachment J. Screening Instrument for Other Young Women

Attachment K. Participant Information Sheets

A. JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

CDC requests OMB approval to collect information under an existing generic clearance, “Focus Group Testing to Effectively Plan and Tailor Cancer Prevention and Control Communication Campaigns,” OMB No. 0920-0800 (exp. 12/31/2017). The respondent universe for this GenIC aligns with that approved under OMB 0920-0800.

The information collection for which approval is sought is in accordance with CDC’s mission to conduct, support, and promote efforts to prevent cancer, reduce its risk, increase early detection and better treatment, and improve the quality of life for cancer survivors authorized by Section 301 of the Public Health Service Act (PHSA, 42 U.S.C. 241).

Approximately 11% of all breast cancer cases in the United States occur in women under 45 years of age. Occurrences of breast cancer among these women are often accompanied by higher risks of recurrence and death, compared to older women with the disease. These women also face unique and significant long-term, treatment-related side effects such as infertility, cognitive dysfunction, muscular and skeletal issues, and cardiac and vascular complications. They are also at an increased risk for developing new cancers and other co-morbid conditions.

In 2009, Congress established the Education and Awareness Requires Learning Young (EARLY) Act, section 10413 of the Patient Protection and Affordable Care Act (Public Law 111-148). The EARLY Act legislation specified the need to create an education and outreach campaign to highlight the breast cancer risks facing young women. In 2014, Congress reauthorized this legislation (Attachment A– Legislative Authority), re-emphasizing the importance of educating young women about breast health and breast cancer risk. Specific aims include:

1. “Increase public awareness regarding breast cancer in young women of all ethnic and cultural backgrounds, including particular risks faced by certain ethnic and cultural groups;” and

2. “Promote educational awareness, early detection, and risk-reducing practices among young women and increase the number of young women with breast cancer warning signs who seek immediate care.”

In response, the Division of Cancer Prevention and Control (DCPC) of the Centers for Disease Control and Prevention (CDC) is launching a public education and awareness campaign to promote breast health for young women ages 18-44 years. The campaign will be branded under a unique campaign name and logo, and will be executed through digital and social media due to the target audience’s increasing presence online and in social media. Campaign tactics will include a paid digital ad campaign; a campaign website with stories about real women who have a family history of breast cancer; digital video vignettes and accompanying educational materials; and social media content to engage the target audience with real stories about women like themselves. All of the campaign content is meant to encourage the target audience to click for more information, share the content with friends and family, “like” the content and otherwise engage with it in the digital sphere as they learn more, become educated, and take steps to protect their health.

A number of campaign materials have been drafted and must be tested. The draft materials under development are summarized below in Table A1-1.

Table A1-A: Materials to be Tested

Item Code	Item Type			Item Descriptor
	Storyboard	One-pager	Factsheet
A1	X			Jennifer
A2	X			Elana
B1	X			Jackie
C1	X			Michelle
C2	X			Lauren
C3	X			Danielle
D1		X		Five Things You Need to Know about BRCA Genes (Ashkenazi Jewish Women)
D2		X		Five Things You Need to Know about BRCA Genes
E1			X	Bring Your Brave. Breast Cancer in Young Women

Materials will be tested with the target audiences identified by the EARLY Act: young women in the general public, and young women with an increased risk for developing breast cancer. Populations at increased risk include Ashkenazi Jewish women, who have been identified by CDC as a group at increased risk for breast cancer at a young age due to an increased risk of having a BRCA 1 or BRCA 2 gene mutation, and African-American women, as this population has been identified as having increased prevalence of breast cancer at younger ages and as having increased mortality rates from breast cancer. We aim to recruit approximately 180 respondents (36 Ashkenazi Jewish women aged 18-44, 72 African American women aged 18-44, and 72 “other” young women aged 18-44) to participate in focus group discussions. Focus groups for “other” young women may include women of all races and ethnicities, with the exception of African American women and Ashkenazi Jewish women who will participate in focus groups specifically for these populations. We will further segment audiences according to whether participants have a known family history of breast or ovarian cancer. Separate focus groups will be held for each audience segment. We will replicate focus groups for audience segments that represent proportionally larger populations in the U.S. (other young women and African American women) in order to obtain broader geographic representation.

Each focus group will be conducted with 9 or fewer respondents and will last approximately 2 hours. A summary of respondents by audience segment is provided below in Table A1-2. The materials presented to each focus group, and/or the order of presentation of materials, will vary so that CDC obtains preliminary, qualitative information needed to assess ad acceptability and placement for each audience segment.

Table A1-B: Focus Groups, by Audience Segment, Family History of Breast or Ovarian Cancer, and Location

Audience Segment	Family History of Breast or Ovarian Cancer		Focus Group Location (one group per location)
	Yes	No
Ashkenazi Jewish women ages 18-29 4	X		New York City
		X	Chicago IL
Ashkenazi Jewish women ages 30-44 4	X		Chicago IL
		X	New York City
African American women ages 18-29	X		Birmingham
	X		Chicago
		X	Birmingham
		X	Chicago
African American women ages 30-44	X		Birmingham
	X		Chicago
		X	Birmingham
		X	Chicago
Other women ages 18-29	X		Sacramento
	X		Phoenix
		X	Sacramento
		X	Phoenix
Other women ages 30-44	X		Sacramento
	X		Phoenix
		X	Sacramento
		X	Phoenix
Total	10	10

The focus groups will assess numerous qualitative dimensions that include, but are not limited to, breast cancer knowledge, attitudes, beliefs, behavioral intentions, information needs and sources. Information on the tone, feel, and content that would most appeal to these women is needed to guide the finalization of campaign materials, as well as assist in determining best strategies for dissemination of campaign materials and messages. Discussions will be tailored to each audience segment. See Attachments B1-G2 for discussion guides and supplementary materials.

Identification of Web Site(s) and Web Site Content Directed at Children Under 13 Years of Age

No web-based data collection methods will be used and, thus, there is no web content directed at children less than 13 years of age.

A2. Purpose and Use of the Information Collection

As part of the health communication process, CDC will conduct focus groups to pre-test concepts, messages, and materials with target audiences. These materials will be branded under a campaign name and logo.

Formative evaluation will center around two main areas of inquiry:

• Qualitative assessments of the target audiences’ baseline knowledge of breast cancer in young women, the related risk to specific populations based on ethnic, racial, or family medical history factors, and actions young women can take to reduce their risk of developing breast cancer. Improved understanding in this area will allow the project team to tailor media campaign content to fill gaps in the target audiences’ existing knowledge.

• Exploring what content and design elements the target audiences find most compelling within the context of a young women and breast cancer media campaign. It is the goal of the project team to develop media advertisements and materials that not only appeal to the target audience, but also motivate them to respond appropriately to presented calls to action.

During focus group sessions, participants will be asked questions at the beginning of the session that specifically relate to the key central messages of the young women and breast cancer campaign to ascertain their general knowledge and attitudes toward breast cancer. After a facilitated discussion of a few campaign materials, questions on appeal, saliency, and understanding of the key central messages will be asked again in order to determine qualitative increases in knowledge and behavioral intentions. This qualitative data collection and analysis would determine whether the existing materials are adequate in communicating the key concepts. Insights gained from the focus groups will determine if the campaign materials under development will motivate women – those with average risk and those with increased risk of developing breast cancer – to respond to the calls to action and engage with activities promoted by campaign messages (clicking on ads, seeking additional information, sharing) and the digital mode of delivery. Along with information collected on saliency and clarity, increases in knowledge at the end of the session would suggest that the materials are appropriate in delivering the key central messages. Persisting deficits in knowledge following the facilitated discussion would indicate that the materials are not clear, or may be culturally inappropriate for the particular audience. It is anticipated that the information collected will lead to refinement of existing materials, and development of new, targeted and more culturally appropriate materials.

By conducting formative evaluation through this information collection, campaign materials can be tailored to suit the audiences’ preferences and educational needs. This will improve acceptance of the campaign materials and the success of the campaign overall.

A3. Use of Improved Information Technology and Burden Reduction

Electronic data collection methods have limited applicability to focus groups, other than video- or audio-taping discussions. However, whenever possible, DCPC staff will employ electronic technology to collect and process data in order to reduce respondent burden and aid in data processing and reporting efficiency.

Efforts have been made to design discussion questions that are easily understandable, not duplicative in nature, and least burdensome. In all instances, the number of items posed will be held to the minimum required in order to elicit the necessary formative or materials-testing data.

A4. Efforts to Identify Duplication and Use of Similar Information

Based on a comprehensive review of activities in CDC’s Division of Cancer Prevention and Control, CDC has determined that the planned information collection efforts do not duplicate any other current or previous data collection efforts.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A6. Consequences of Collecting the Information Less Frequently

As the health communication process illustrates, formative evaluation is a critical segment of a scientifically sound campaign effort. Formative evaluation, often encompassing concept, message, and materials testing activities, is essential in pre-testing materials to evaluate a wide variety of dimensions that include, but are not limited to, appeal, saliency, clarity, cultural appropriateness and readability/understandability. If a concept and/or a message is not tested, then resources could be expended without necessary attention and preparation paid to the overall communication objective. Forgoing testing can also increase the likelihood of unintended consequences from an irrelevantly perceived message and/or decreased credibility of an organization and/or a Federal health official (Wallendorf, 2001 & Harris-Kojetin et al., 2001). Finally, if materials are not tested with the intended audience, a poor execution strategy could weaken a sound concept.

There are no legal obstacles to reducing the burden.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances. The activities outlined in this package fully comply with all guidelines of 5 CFR 1320.5.

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A8.A Federal Register Notice

This information collection is being conducted using the Generic Information Collection mechanism of Focus Group Testing to Effectively Plan and Tailor Cancer Prevention and Control Communication Campaigns, OMB No. 0920-0800. As required by 5 CFR 1320.8(d), a notice for public comments was published in the Federal Register on June 25, 2014 (Vol. 79, No. 122, pages 36064-36065). No public comments were received. The current submission does not require publication of an additional Federal Register Notice.

A8.B Efforts to Consult Outside the Agency

The proposed protocol and discussion guides were developed and reviewed extensively by DCPC staff and others directly involved in implementing the DCPC communications campaigns. There were no external consultations.

A9. Explanation of Any Payment or Gift to Respondents

Incorporating modest incentives to aid in recruitment acknowledges participants’ time and effort, boosts response rates, may improve the quality of information collected. Each focus group participant will be provided with a modest gift of $75 for their participation in a two-hour focus group. This incentive is based on market rates commensurate with the cities in which the data collection is to take place.

The most important aspect of an incentive plan may be its potential for reducing response bias, underreporting bias, and similar sources of error. Findings from the National Survey of Family Growth (a study in which childbearing and family planning patterns are collected from young women) demonstrated that incentives not only had positive effects on response rates, but they also increased the accuracy of reporting. Incentives are necessary for qualitative information collections such as the proposed materials testing in order to ensure that those who are willing to participate are as representative as possible of the target audience, which in this case includes participants of hard-to-reach racial and ethnic subpopulations, and young women who may have responsibilities for child care, etc. Failure to provide a basic incentive is likely to bias samples in the direction of well-educated individuals who are generally predisposed to be helpful (http://www.cdc.gov/nchs/nsfg.htm).

A10. Assurance of Confidentiality Provided to Respondents

1. Privacy Act Determination

The National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) has reviewed this OMB submission and determined that the Privacy Act is not applicable. CDC will not create a record system for this project. CDC will not be privy to the last names, mailing addresses, telephone numbers or email addresses of any of the focus group participants. These individuals will be recruited using the proprietary databases of a professional recruiting firm. Eligibility criteria will be established for all focus group participants, and potential participants will be screened during a telephone interview (see Attachments H-J for screening instruments). No personal identifying information used in the recruitment process will be linked to the data collected in the focus group discussions. Thus, no personal information in identifiable form will be collected by CDC.

2. Safeguards

A minimum amount of demographic information may be retained in focus group notes for purposes of analysis, but will not be sufficient to identify respondents. Participants will be informed that focus groups will be audio-taped and transcribed, and that tapes will be destroyed after completion of each report on findings. DCPC staff, in conjunction with Oak Ridge Associated Universities (ORAU), will collect and evaluate the audience research data.

Information provided during the groups will be kept private and secure to the extent allowed by law. Participants’ names or images will not be used in the final report. No statements made by participants will be linked to them by name. Only members of the research staff will be allowed to look at the records. Participants’ names or other personally identifiable information will not be shown or used in the presentation of findings.

3. Consent

All information provided by respondents will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law. Informed consent will be obtained from respondents (see Attachments B1 – G1 – Discussion Guides) and they will be informed that participation is voluntary; they do not have to answer questions if they do not want to, their responses will be treated in a secure manner, and they can stop participating at any time. It has been determined that these information collection activities are not generalizable and do not qualify as human subjects research and will therefore not require IRB review and approval.

4. Nature of Response

Respondent participation is entirely voluntary, as noted in the participant information sheet (Attachment K).

A11. Justification for Sensitive Questions

The majority of questions asked will not be of a highly sensitive nature. However, some respondents may find thinking about and discussing the disease of cancer unpleasant. A portion of respondents could consider questions about race, ethnicity, or other demographic characteristics to be sensitive, although such questions are unlikely to be highly sensitive. Additionally a portion of respondents may feel uncomfortable answering some questions about their individual cancer experiences, level of disease awareness, and/or adopted preventive behaviors (or lack thereof) associated with cancer. Questions about family history of breast or ovarian cancer are specifically needed for audience segmentation. Such questions, if asked, would be necessary for the purposes of a targeted communication campaign and thus to the information collection. To minimize psychological distress, the moderator will inform participants that they do not have to respond to any questions they do not want to answer and they may stop participating at any time.

A12. Estimates of Annualized Burden Hours and Costs

1. CDC plans to collect information from 180 women ages 18-44 years, segmented into 20 focus groups according to:

   1. target audience (African American women, women of Ashkenazi Jewish heritage, and other young women),

   2. age group, and

   3. family history of breast or ovarian cancer.

Because the questions for each audience segment vary slightly, there are 6 customized versions of the focus group Discussion Guide (see Attachments B1-G1). Each Discussion Guide is specific to both the target audience ([a] above) and participants’ relevant family medical history ([c] above). Each focus group will last 2 hours and will be comprised of 9 respondents.

Each focus group Discussion Guide includes an example Response Sheet. Each participant will be asked to provide feedback on 3 storyboards and will complete one Response Sheet for each storyboard. Because the moderator will coordinate the distribution, completion, and collection of Response Sheets within the context of the focus group discussion, the burden associated with the Response Sheets is included in the overall 2 hour burden for the focus group.

Potential participants will be recruited through standard commercial recruiting practices. Similarly, potential respondents will be screened for interest and eligibility using a screening form (see Attachments H-J). There are 3 versions of the screening instrument (one for each target audience: Ashkenazi Jewish women, African American women, and Other women). Based on previous experience recruiting focus group participants from master lists of eligible or interested persons, it is estimated that twice the target number of needed respondents must be screened in order to yield the targeted number of respondents. The total number of respondents screened will be 360. The estimated burden per response for screening is three minutes.

The estimated burden to respondents is 378 hours, as summarized in Table A12-A below.

Table A12-A: Estimated Annualized Burden to Respondents

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Ashkenazi Jewish Young Women	Screening Instrument for Ashkenazi Jewish Young Women	72	1	3/60	4
	Discussion Guide for Ashkenazi Jewish Young Women (with a Family History of Breast Cancer)	18	1	2	36
	Discussion Guide for Ashkenazi Jewish Young Women (with No Family History of Breast Cancer)	18	1	2	36
African American Young Women	Screening Instrument for African American Young Women	144	1	3/60	7
	Discussion Guide for African American Young Women (with a Family History of Breast Cancer)	36	1	2	72
	Discussion Guide for African American Young Women (with No Family History of Breast Cancer)	36	1	2	72
Other Young Women	Screening Instrument for Other Young Women	144	1	3/60	7
	Discussion Guide for Other Young Women (with a Family History of Breast Cancer)	36	1	2	72
	Discussion Guide for Other Young Women (with No Family History of Breast Cancer)	36	1	2	72
	Total				378

Information will be collected over a one month time period. There are no costs to respondents except their time to participate in the focus groups. The total annualized burden to respondents is 380 hours.

B. Approximately 40% of respondents will be African American, 20% will Ashkenazi Jewish women, and 40% of respondents will be other young women. Table A12-B presents the calculations for cost of respondents’ time using two categories of mean hourly wages. Hourly mean wage information is from the U.S. Department of Labor, Bureau of Labor Statistics Web site (http://www.bls.gov/oes/2013/may/oes_nat.htm#29-0000) specifically originating from the Occupational Employment Statistics May 2013 National Occupational Employment and Wage Estimates, United States, Bureau of Labor Statistics. The total estimated annualized respondent cost (including the screening form) is $8,441.

Table A12-B: Estimated Annualized Cost to Respondents

Type of Respondent	Form Name	No. of Respondents	Total Burden Hours	Average Hourly Wage Rate	Total Cost
Ashkenazi Jewish Young Women	Screening Instrument for Ashkenazi Jewish Young Women	72	4	$22.33	$89
	Discussion Guide for Ashkenazi Jewish Young Women (with a Family History of Breast Cancer)	18	36	$22.33	$804
	Discussion Guide for Ashkenazi Jewish Young Women (with No Family History of Breast Cancer)	18	36	$22.33	$804
African American Young Women	Screening Instrument for African American Young Women	144	7	$22.33	$156
	Discussion Guide for African American Young Women (with a Family History of Breast Cancer)	36	72	$22.33	$1,608
	Discussion Guide for African American Young Women (with No Family History of Breast Cancer)	36	72	$22.33	$1,608
Other Young Women	Screening Instrument for Other Young Women	144	7	$22.33	$156
	Discussion Guide for Other Young Women (with a Family History of Breast Cancer)	36	72	$22.33	$1,608
	Discussion Guide for Other Young Women (with No Family History of Breast Cancer)	36	72	$22.33	$1,608
	Total				$8,441

A13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers

None.

A14. Annualized Cost to the Government

The estimated average annual cost to the Federal government for the proposed focus group activities is $229,320.00. This figure encompasses the salary of two GS-14 employees, communication contract costs, as well as fees for identifying and recruiting participants, incentive payments, facility rental, and transcription.

Estimated Annualized Cost to the Government, per Campaign and Total
Cost Category	Estimated Annualized Cost
Federal employee costs (adjusted for Atlanta) (10% FTE of 1 GS-14 Step 5 @ $116,808/yr) (2% FTE of 1 GS 14 Step 5 @ $116,808/yr)	$11,681 $2,336 Subtotal $14,017
Contractual costs for focus group facility rental, focus group moderator, participant recruitment, and report on findings, per campaign	$215,303
Total	$229,320

A15. Explanation for Program Changes or Adjustments

This is a new, one-time information collection.

A16. Plans for Tabulation and Publication and Project Time Schedule

Statistical methods will not be employed to analyze focus group data, as it is not appropriate to report the percentage of focus group participants who expressed a particular view (Carey, 1995; Morgan, 1995; National Cancer Institute, 2002; Webb & Kevern, 2001). Typically, not every participant in a group comments on every issue discussed (Carey, 1995), and the course of discussion will vary across groups, with some topics emerging in one group and not in another (Carey, 1995; Morgan, 1995). Qualifiers such as “many,” “several,” and “few” will be used to describe the number of participants who expressed a particular view.

Project Time Schedule

Table A16-1 presents the estimated timeline for conducting focus groups following receipt of OMB clearance.

Table A16-A: Project timeline for cancer communication campaigns

Activity	Time Schedule
Focus group recruitment	4-5 weeks after OMB approval
Focus group testing	6-12 weeks after OMB approval
Analysis of focus group results (topline reports)	12-20 weeks after OMB approval
Report Writing/Recommendations to CDC based on Findings	3-6 months after OMB approval

Focus group findings will inform campaign planning efforts, provide guidance on efforts to refresh existing materials, and aid in the sound development of new communication products for specific cancer communication initiatives. Additionally, findings will be disseminated through presentations and/or posters at meetings and publications in peer-reviewed journals. All abstracts, poster presentations, and manuscripts will undergo CDC clearance review prior to submission to conferences or journals.

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

The OMB expiration date will be displayed.

A18. Exemptions to Certification for Paperwork Reduction Act Submissions

No certification exemption is being sought.

References

Berlin, M., Mohadjer, L., Waksberg, J., Kolstad, A., Kirsch, I., Rock, D., & Yamamoto, K. (1992). An Experiment in Monetary Incentives. In the American Statistical Association (ed.), Proceedings of the American Statistical Association Section on Survey Research Methods (pp. 393-398). Alexandria, VA: American Statistical Association.

Cancer survivors—United States 2007. Morbidity and mortality weekly report 2011;(60)9:269-272.

Church, A.H. (1993). Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta‑Analysis. Public Opinion Quarterly, 57, 62‑79.

Harris-Kojetin, D., McCormack, L.A., Jael, L.A., Sangl, E.F., & Garfinkel, S. A. (2001). Creating more effective health plan quality reports for consumers: Lessons from a synthesis of quality testing. Health Services Research, 36(3), 447-476.

Krueger, R.A. (1994). Focus Groups: A Practical Guide for Applied Research. 2^nd ed. Thousand Oaks, CA: Sage Publications.

Krueger R.A., Casey M.A. (2000). Focus Groups: A Practical Guide for Applied Research. 3rd ed. Thousand Oaks, CA: Sage Publications.

National Cancer Institute. (2002). Making Health Communication Programs Work (NIH Publication No. 02-5145). Bethesda, MD: Department of Health and Human Services.

Wallendorf, M. (2001). Literally literacy. The Journal of Consumer Research, 27(4), 505-511.