OMB Package
Taxpayer Advocate Service 2009 External Focus Groups
The Taxpayer Advocate Service (TAS) is looking to identify ways to improve Business Results within the IRS by working with taxpayers identified as underserved by TAS. Communications and Liaison (C&L) is working with an outside vendor, Porter Novelli, to better understand how to communicate with these audiences about receiving TAS assistance. TAS wants to hold discussion groups with these underserved audiences to obtain their thoughts and opinions, particularly to gauge relevance and comprehension of our mission statement and other messages, and how different segments of TAS target audiences interpret this information. TAS will share different versions of our mission statement and other draft messages with the participants to determine which language most clearly resonates with these audiences.
To meet this need, the Taxpayer Advocate Service would like to conduct ten triads (i.e., mini-focus groups with three people per session) in September 2009. The triads will be held in Baltimore, MD and Los Angeles, CA. Four triads will be held in Baltimore, and six will be conducted in Los Angeles. Although the specific dates for the sessions have not been determined, the triads will be conducted concurrently in each city (i.e., separate moderators will conduct individual sessions simultaneously) in each of the two cities so the testing can be done in one day.
Triads represent an effective methodology to examine reactions to written materials in that the reduced number of participants allows a professional moderator to probe for specific thoughts and feelings that underlie preferences for each statement. In addition, smaller sessions provide a less intimidating environment than traditional focus groups, which will allow the research to explore the consumer mindset around tax problems and participants’ potential previous interactions with the IRS. The results of the focus groups will be presented to the appropriate areas of TAS.
The C&L division within TAS has been working with Porter Novelli to identify the four priority underserved audiences of TAS. Porter Novelli reviewed two previous TAS studies conducted by Russell Research as well as other datasets (e.g., Simmons Two-Year Combined Study) to identify a set of TAS target audiences defined by socio-demographic segmentations, with the primary determinants being income and marital status. The segments are as follows:
Middle-Income Families (27% of the TAS Underserved)
Affluent Families (24% of the TAS Underserved)
Low-Income Families (18% of the TAS Underserved)
Financially Distressed (12% of the TAS Underserved)
TAS would like to conduct ten triads within these underserved audiences:
Four with mixed Middle/Affluent-Income Families
Four with Low-Income Families
Two with Financially Distressed Individuals
Effectiveness of Taxpayer Advocate Service (TAS) Mission Statement and Messages
TAS is interested in improving the effectiveness of its mission statement and other messages. This will involve testing different versions of the mission statement and messages for 1) ease of understanding, 2) readability, 3) effectiveness in conveying how TAS can help and 4) establishing trust. TAS wants to ensure that taxpayers understand our role in serving as an advocate on behalf of taxpayers. Additionally, this effort will hopefully improve taxpayers’ willingness to use TAS to help them solve their tax problems.
The overall objective of this project is to provide qualitative data to TAS on views of taxpayers relating to the effectiveness and ability to understand our mission statement and messages, and how effectively these convey the purpose and capacity of TAS to assist taxpayers with their tax problems. The specific business questions to be explored by this project include:
What are the opinions of the underserved audiences regarding the effectiveness of the TAS mission statement and are there alternative versions that might better communicate this information?
How relevant and comprehensible are TAS current messages and how might they be improved?
Is there an understanding among our underserved audiences regarding commonly used terms such as “tax problem” or “economic hardship?”
The project will encompass: recruiting participants, moderating the triads, and preparing a report for TAS summarizing the qualitative feedback from underserved participants. Comments will be used to shape our mission statement, messages, and marketing materials.
To accomplish the objectives of this project we will conduct ten triads in conjunction with our vendor, Porter Novelli. These triads will be held in Baltimore, MD and Los Angeles, CA.
Triad interviews are a directed discussion on specific messages and language used by TAS with a small group of three people. As a type of small group interview, triads use group interaction to stimulate relatively spontaneous responses to the supplied topics. The goal is to explore the feelings, opinions, and beliefs people hold, and to learn how these feelings shape overt behavior. Triads provide insight and direction rather than quantitatively precise or absolute measures. They are useful for gathering information and generating hypotheses based on participants’ opinions. The main advantage triads offer is the opportunity to observe considerable interaction on a topic in a limited period of time. In fact, given the small number of individuals per session, it affords more interaction than traditional focus groups provide. An important aspect of triads is that they give rise synergistically to insights and solutions that may not come about without them.
Triads are also a useful methodology in that they create a comfortable environment in which respondents are not daunted by many individuals in the same room. The reduced number of participants per session also reduces the possibility of “group think” and it creates a dynamic that enables the moderator to better control respondents that seek to dominate the discussion.
The market segments for this project consist of the identified underserved TAS audiences: Middle-Income/Affluent Families,1 Low-Income Families, and the Financially Distressed. TAS executives and managers will use the data gathered during the focus group interviews to better shape an effective mission statement, language, messaging, and development of marketing materials.
Our step by step approach follows:
Step 1 – We met with Porter Novelli to determine the objectives for this project. After clarifying objectives for the qualitative sessions Porter Novelli developed screening questionnaires (see Appendices A, B, and C) to recruit participants and moderator’s guides (see Appendices D and E) to be used during the focus groups.
Step 2 – So that we obtain a diverse group of participants Porter Novelli prepared a demographic sheet (see Appendix F) to be used by a professional recruitment facility to track recruitment. The facility will use the demographic sheet and the screener’s guide to solicit and select focus group participants.
Step 3 – Professional moderators contracted by Porter Novelli will conduct the triads. All focus groups will be audio-taped and video-taped (participants informed in advance). Their identities will be kept confidential.
Step 4 – Notes taken while watching all focus group sessions and on-site recordings will serve as the data collection method and the foundation of the analysis. Porter Novelli will review each set of data for recurring themes identified during the review. More specifically, study analysts will review notes and responses to handout exercises to identify patterns in participants’ answers to the questions in each segment of the discussion guide. (See Appendix G for handouts used during the discussions.) Recordings will be reviewed to explore additional themes and to confirm results. Differences across gender and target audience will be noted in the report when appropriate; however, given the small number of participants in this research, analyses focus on commonalities across groups, as opposed to within specific demographic characteristics. No respondents’ names will be used in reporting documents. Upon conclusion of the research, Porter Novelli will submit a summary report of findings.
It should be noted that focus groups are a qualitative research technique. Qualitative research provides valuable insight into a particular group’s thoughts, feelings, and perspectives and is especially useful in identifying “red flags” or unintended interpretations of creative material. Recruiting techniques and small sample sizes, however, mean that results are not statistically representative of a larger population. Consequently, findings will be considered descriptive and directional but not definitive.
We want to include a diverse mix of our population based on the following aspects:
Gender
Self-identified as “bad money managers” (assessed via responses to screening items)
Self-identified as having a “life-changing event” in past 12 months (assessed via responses to screening items)
Household income
Marital status
The discussions will gather qualitative data only. The data will not be, nor will it be presented to be, representative of the population. The study will obtain geographic diversity via the different cities used for the focus groups.
TAS proposes to use the qualitative sessions to identify and capture a range of ideas regarding the TAS mission statement, language, and TAS messaging. Different versions of these communications elements will be presented to the participants. This data will be used to revise our mission statement, language, messaging and marketing materials.
The participants’ comments will be used to provide insight into TAS underserved audiences’ understanding of our mission statement, language and TAS messaging, and will be used to improve the content of our messages to taxpayers. The results are not designed to, nor is it expected that they will, be projected to the population for formulation of conclusions about the general population. It is anticipated that the study objectives can be met in a qualitative, not quantitative measure.
The triads are planned for mid to late September 2009. They should be completed on one day.
Porter Novelli is responsible for recruiting and screening participants, preparing screener and moderator guides, moderating the groups, and compiling and summarizing the results. They have research analysts trained in all aspects of qualitative research, including moderating the focus groups, analyzing the qualitative data, and preparing report findings. Triads will be conducted in Baltimore, MD and Los Angeles, CA.
The contract with the vendor is for $68,500. There is also an estimated cost of $1,600 for travel of IRS personnel for a total estimated cost of $70,100. This includes all vendor costs and IRS costs for travel, printing, postage, and overtime payments (but not regular salary payments).
Participants will receive a cash stipend for their participation ($75-$100), intended to reimburse for expenses such as transportation, child care services, etc. The incentive amount will be standard for the locations and a recruitment of this nature.
The vendor will be responsible for recruiting the focus group participants. The vendor has agreed to use specially designed screeners (see Appendix A,B, & C) to ensure the participants meet the requirements of our study.
The Taxpayer Advocate Service would like to conduct ten triads (i.e., mini-focus groups with three people per session) in September 2009. The triads will be held in Baltimore, MD and Los Angeles, CA. Four triads will be held in Baltimore, and six will be conducted in Los Angeles. ) The triads will be held in facilities selected by the vendor. All facilities will be suitable for conducting focus groups.
The expected response rate is 10%. This is based upon the vendor’s prior experience in conducting similar research.
A stipend will be provided to all participants who show up and participate in the focus groups. This incentive has proven itself as a valuable tool for increasing participation rates.
Triad interviews are a directed discussion on specific messages and language used by TAS with a small group of three people. As a type of small group interview, triads use group interaction to stimulate relatively spontaneous responses to the supplied topics. The goal is to explore the feelings, opinions, and beliefs people hold, and to learn how these feelings shape overt behavior. Triads provide insight and direction rather than quantitatively precise or absolute measures. They are useful for gathering information and generating hypotheses based on participants’ opinions. The main advantage triads offer is the opportunity to observe considerable interaction on a topic in a limited period of time. In fact, given the small number of individuals per session, it affords more interaction than traditional focus groups provide. An important aspect of triads is that they give rise synergistically to insights and solutions that may not come about without them.
Triads are also a useful methodology in that they create a comfortable environment in which respondents are not daunted by many individuals in the same room. The reduced number of participants per session also reduces the possibility of “group think” and it creates a dynamic that enables the moderator to better control respondents that seek to dominate the discussion.
There has been no official research conducted to date to inform message development for TAS among consumers. In an effort to determine whether such an endeavor would be necessary, we conducted informal discussion sessions with eight individuals among the Financially Distressed audience. The results from these informal discussions suggested that TAS materials were not communicating as intended. As a result, we would like to conduct a more comprehensive research program to assess how TAS primary audiences interpret (or potentially misinterpret) current TAS messaging. Triads are an excellent vehicle for gauging message relevance and comprehension.
We acknowledge that we are conducting qualitative research among tax professionals during our tax forums this summer to assess that specific audience’s reactions to our messaging. It is important to note, though, that the research we are proposing in this package is among consumers only (i.e., excludes tax professionals from participation).
We will include a diverse mix of our population based on the following aspects:
Gender
Self-identified as “bad money managers” (assessed via responses to screening items)
Self-identified as having a “life-changing event” in past 12 months (assessed via responses to screening items)
Household income
Marital status
TAS will ensure that the utmost scrutiny is given to privacy, security, and disclosure when reporting the comments from the triads. Only first names will be used in both recruiting and groups (first initial of last name will be used if there are duplicate names). No individual taxpayer data will be used for this project. Only qualitative data derived from discussions will be used in the report. We will limit and control the amount of information that we collect to those items that are necessary to accomplish the research objectives.
Porter Novelli will be responsible for recruiting and screening participants, preparing screener and moderator guides, moderating the groups, and compiling and summarizing the results. The interviews will be conducted at professional testing facilities in both cities. We will protect the privacy of the triad participants by not using names in the report. We will also control official access to the information and will not allow public access to the information.
We will apply fair information and record-keeping practices to ensure protection of all participants. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its release to authorized recipients.
The estimated time to complete the participant screening is 15 minutes and the estimated time for each triad attendee is 1.5 hours. We will assume a 10% percent success rate in soliciting qualified participants.
Screening Burden |
||
Total number of potential participants screened: (participants screened for each group and 10 groups held = 50 screenings per focus group X 10 focus groups = 500 participants)
|
500 |
People |
Estimated time to complete screening |
15 |
Minutes |
Estimated participant screening burden (500 X 15 minutes = 7500 minutes / 60 = 125 hours) |
125 |
Hours |
Triad Participation Burden |
||
Estimated number of participants: There will be 2 locations and 10 groups conducted overall (6 in Los Angeles; 4 in Baltimore) for a total of 30 participants. |
30 |
People |
Time to conduct the triad (1.5 hours) |
1.5 |
Hours |
Estimated focus group participant burden (30 X 1.5 =) |
45 |
Hours |
Travel Time (50 participants X .5 hours) |
25 |
Hours |
Total burden (screening and focus group participation (screening burden of 125 hours + focus group burden of 45 hours + travel time of 25 hours = 195) |
195 |
Hours |
The following information will be provided within 60 days after the close of the focus group data collection operations:
Findings: a brief summary of significant (important) findings that were evidenced in the results.
Lessons learned: a brief summary of lessons learned because of the findings.
Number of participants screened.
Number of triad participants.
Date the data collection began.
Date the data collection ended.
Cost: Vendor-related costs, travel, stipend payments, and any other costs incurred as direct result of the focus groups.
Burden hours.
Attachments
Appendix A: Screening Questionnaire – Financially Distressed
Appendix B: Screening Questionnaire – Low-Income Families
Appendix C: Screening Questionnaire – Middle/High-Income Families
Appendix D: Moderator Guide – Financially Distressed
Appendix E: Moderator Guide –Low-Income/Middle to High-Income Families
Appendix F: Demographic Sheets
Appendix G: Participant Handouts
Appendix H: Script for follow-up phone call (i.e., for those taxpayers who accept the initial invitation to participate)
1 Given the limited income differences that distinguish Middle-income Families and Affluent Families, these groups are combined. This will also reduce the overall burden associated with the project and allow for more efficient data collection.
| File Type | application/msword |
| File Title | Level 2 Plan 1-04-20-053N |
| Subject | Standards for IRS Notices: Language and Layout |
| Author | ecschu74 |
| Last Modified By | qhrfb |
| File Modified | 2009-08-09 |
| File Created | 2009-07-31 |