Various Customer Feedback Activities

Appendix A: Moderator Guide

ACS Deliberative Focus Groups

Discussion Guide

Final

November 21, 2013

Purpose:

To identify messages about the American Community Survey (ACS) that resonate with audiences that have cynical, suspicious, distrustful or ambivalent attitudes toward the U.S. government, with the goal to inform materials creation and help increase survey self response rates among these types of audiences.

Overview of Jury Groups:

Time	Group
5:30 - 6:00 PM	Part 1	First group session with 6-8 advocates/opponents
6:15 - 6:45 PM		Second group session with 6-8 advocates/opponents
7:00 - 7:30 PM		Top advocates and opponents from first and second group sessions develop arguments for jury session
7:30 - 9:30 PM	Part 2	Jury session with 2 advocates, 2 opponents, and 8 jurors

Screening Criteria:

• All participants are identified to be ambivalent or cynical/suspicious with regard to the government during pre-screening.

• Participants may be principally opposed to completing the ACS because they find the questions intrusive or burdensome, or are uncomfortable with the use of administrative records.

• Participant groups should demonstrate diversity on the bases of race/ethnicity and gender.

• Participants’ resistance to the ACS may or may not be politically motivated.

PART 1: Group Session Materials

1. Preliminary Briefing For Mini-GROUP Participants

Before the start of each small group session, the moderator will brief participants on the structure and the objectives of the mini-group about to begin.

Before initiating the session, the moderator will ask participants about their present level of support for/opposition to ACS.

He or she will explain that an assortment of materials and messages (including existing ACS materials, talking points for and against ACS, answers to “Why Do We Ask These Questions,” and op-eds) that discuss the ACS are displayed throughout the room. The moderator will encourage participants to review these materials to help in developing arguments for their side.

Participants will be briefed as follows:

I want to thank you all for taking the time to be with us today to share your insights and opinions. Before we get started, I want to first mention a few guidelines for our discussion.

I am an independent market researcher. I am not here to sell you anything or lend a particular point of view. I’m open to both positive and critical views.

We are recording the discussion so that afterwards we can double-check our notes. We will only use the recording as a reference point as we write our report. Once the report is finalized, the recording will be deleted.

The session we are about to begin is actually a discussion about the value of the United States Census Bureau’s American Community Survey (ACS). The group of you will be assigned to take a side {in support of/against} the ACS.

Before we start, how many of you have heard of the ACS? How would you describe your present level of support for or opposition to the ACS?

The goal of the discussion is to develop arguments to convince me that the ACS {is/is not} a worthwhile pursuit, of benefit to the United States. You will place yourself in the roles of either advocates or opponents of ACS. You must make a strong case for your assigned side to convince me. I may ask you to originally argue one side, and then switch to another.

An assortment of materials that make the case {for/against} the ACS are spread throughout the room. At the beginning of the session, please spend a few minutes looking at these materials to help you develop arguments for your side.

Considering the materials you looked at, you will then need to convince me:

1. What are the best arguments {for/against} the value of the American Community Survey?

Please consider:

• What do you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How did these arguments make you feel?

• Which of the arguments you saw were least convincing in support of/against the ACS?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any compelling arguments about the ACS missing from the materials you saw?

• For those of you actually less inclined to support the ACS than you were at the beginning of this session: did you learn anything that might have a positive effect on your likelihood to support it?

Next, you will be asked to imagine that you’ve been chosen to participate in the ACS:

2. Based on the materials you reviewed, which arguments most strongly motivated you to {complete/not complete} the survey?

Please consider:

• What do you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How did these arguments make you feel?

• Which of the arguments you saw were least convincing in support of/against the ACS?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any compelling arguments about the ACS missing from the materials you saw?

• For those of you actually less inclined to support the ACS than you were at the beginning of this session: did you learn anything that might have a positive effect on your likelihood to support it?

The moderator will encourage participants to keep these questions in mind as they peruse the sample materials at the start of the session.

Keep these questions in mind as you review the materials. You’ll want to write down arguments for your perspectives (pro/con), and prepare defenses for any counter arguments you may hear. You can consult with your group members after you’ve completed your own review of the materials.

2) Moderator guide for GROUP Sessions

Each group session will last 30 to 45 minutes. The moderator should use the prompts provided below and intervene only if the discussion gets off-track.

To begin the group session, the moderator should bring all participants into the room and quickly explain the goals.

Before we get started, I want to quickly explain how this discussion will go. This session is meant to lay the groundwork for a debate between two sides: those in support of the ACS versus those against it. One or two of you will be selected to participate in a second session later this evening, for an additional cash payment.

In this session, each of you will be given time to make your points. I will then tell you when it’s time for the group to begin discussing or questioning these points.

Remember, as you make your cases to me, your objective is to convince me that the ACS {is/is not} a valuable pursuit for the United States and that it {is/is not} worthwhile for individuals to fill it out. You may be asked to debate a side you don’t personally agree with – or debate both sides – for the purposes of this session.

The moderator should encourage participants to peruse the materials in the room to get ideas when developing their arguments.

To begin, go ahead and spend a few minutes looking at the materials spread throughout the room. You should reference these as you begin to make the case for your side. Feel free to write down the strongest arguments you find to share with the group.

The moderator should now ask the participants to present their arguments for Question 1:

Now, each of you will have an opportunity to make your cases to me for Question 1 only:

1. Considering the materials you looked at, what are the most convincing arguments you can identify {for/against} the usefulness and value of the American Community Survey?

Then, you will get a chance to discuss with other group members, and they will get to question you about the points you have made. After that, we will discuss Question 2.

The participants will then present their arguments for Question 1. Prompt with the following only if the arguments are off-topic.

• What do you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How did these arguments make you feel?

• Which of the arguments you saw were least convincing in support of/against the ACS?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any arguments about the benefits of the ACS missing from the materials you saw?

• For those of you less inclined to support the ACS, did you learn anything that might have a positive effect on your likelihood to support it?

The group will then question or challenge each other on the points they have made.

The moderator should now ask the participants to present their arguments for Question 2:

2. Imagine that you have been selected to respond to the ACS. Based on the materials you reviewed, what arguments motivated you the most to want to complete the survey?

The participants now will present their arguments for Question 2. The moderator will prompt with the following questions only if the arguments are off-topic.

• What did you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How do these arguments make you feel?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any arguments about the benefits of the ACS missing from the materials you saw?

• For those of you less willing to complete the ACS, did you learn anything that had a positive impact on your willingness to participate?

The group will then question or challenge each other on the points they have made.

IF TIME REMAINS:

The moderator will lead a discussion to conclude the group session.

Thinking about everything we discussed today, were any arguments particularly persuasive to you? Did they make you re-evaluate your own position at all? Why or why not?

In your final estimation, what are the most compelling reasons to complete the ACS?

Finally, at what point in the discussion did you feel the most confident about your support/opposition of the ACS and your motivation to complete the survey? When did you know that your group had built a good case?

PART 2: jury session materials

1. Preliminary Briefing for advocates and opponents

Before the 1.5 to 2 hour jury session begins, the teams arguing for and against the ACS as well as the jury will need to be briefed on the structure and the objectives of the session.

First, the advocates and opponents will be briefed and given handouts so that they can prepare their arguments. Each team will need approximately 30 minutes to fill out the handouts and prepare for the jury session. Participants should work with their partner to prepare their arguments.

Advocates should be briefed as follows:

The discussion you’re about to participate in is modeled after a courtroom trial with a jury. You are going to play the role of the lawyers. There are two sides in this trial— advocates for the ACS and opponents against the ACS. Each of you will be representing one of those sides and will have a partner to work with during the “trial.”

There will be a panel of jurors who will listen to your arguments and your opponents’ arguments. Following these arguments, they will discuss among themselves the points you’ve made and decide which team has won the trial—which side has made the better arguments and persuaded them about the ACS. Each member of the winning side will receive an extra cash bonus, so make your arguments as persuasive as possible.

Your arguments will focus on the two key questions from your smaller group session centering on the value of the American Community Survey and reasons to complete it. Please refer to your handouts for the questions and the sub-points you’ll need to address.

Please review these sheets again carefully with your partner. Feel free to ask me any questions.

At this point, the moderator will answer any questions on the briefing so far.

Take time to think about this issue with your partner and talk about how you will address each of the points in the handouts. You and your partner should write out the arguments you will be making. Remember you’re a team, so you should work together.

One final point on how the set-up for this “trial” works. When you’re first taken in front of the jurors, you’ll only be talking about the first question. You and your partner will speak for a few minutes. Then, the opposing team will speak. After each team has made its arguments, the teams will be cross-examined by each other and by the jury during a cross-examination period. You will ask each other questions and be challenged on the arguments you’ve just made. Then you’ll be taken out while the jurors actually discuss that point, and you’ll be brought in again about 15 minutes later to do the same for the second question. The winning team is the team that gets most votes for both debates.

So take about 30 minutes to work on this apart from the other team, and then we’ll take you to the jury.

The pro- and anti-ACS teams are now given handouts to complete with their partners and discuss the arguments they will make in the jury session.

2. Preliminary Briefing for Jury Members

After the pro- and anti-ACS teams have been briefed and begin preparing their arguments, the moderator should brief the members of the jury on the rules of the debate and the questions they will be deciding. Jury members should be briefed all together. This briefing should take place in the same room where the trial will be held, so that the jury members are already seated and ready once the advocates and opponents come into the room and begin making their arguments.

The jury members should be briefed as follows:

The discussion that you’re about to participate in is modeled after a courtroom trial. You are going to play the role of the jury, or judges. There are two sides in this trial— Advocates for the United States Census Bureau’s American Community Survey (ACS), and opponents arguing against the ACS. A team of 2 debaters, essentially like lawyers, will be representing each of those sides during the actual “trial.”

Your job is to listen to all the arguments that each side makes, the attacks they make on the other sides, and each side’s defense of its own point of view. In the end, you as judges will decide which team has convinced you that their side is the winner. And you should know what the stakes are: Both members of the winning team will receive a prize based on your decision—so think carefully about how you vote.

There are two debates you’ll be hearing, which I’ll review with you in a moment. Your job is to vote for a winning side in each debate, but you don’t have to decide in favor of a single side in both debates. In other words, you could decide that the side in support of ACS is the winner of debate one, but the opposing side wins in debate two. To help in your decision, the specific questions you are considering are included in the following handouts. Please read through this carefully so that you know what the topics are you’ll be asked to decide.

The first debate will focus on the overall value of the ACS and the second debate will focus on reasons to respond to the ACS.

The teams will only be addressing one question at a time, and you’ll only be deciding one question at a time, so we’ll give you forms for the second debate later, just before the debaters are ready to make their arguments on those points.

At this point, the moderator gives each jury member a copy of the questions handout.

As you listen to the teams make their points, attack their opponents’ points, and defend their position against attacks, you will need to fill out the form I just handed you. This form is to help you make your decisions in the end. You should write down the most persuasive points and the least persuasive points in the spaces provided.

Also, there are spaces where you should write down any questions or challenges of your own that you would like to pose. After each team has spoken, the jury will have time to cross-examine the “lawyers.” Be sure to write down here the questions that you will want to ask. If a team makes a point that you do not agree with, be sure to write down why you would challenge that point. Then, you can pose your challenge during the cross-examination.

Ok, any questions? If not, I’ll bring in the teams to begin the first debate.

3. Moderator guide for Jury Sessions

A single jury session consists of two debates. During each debate, the debaters will present their cases and the jury will vote. After the two debates are over, the jury will decide which side has won overall—which side has made the most persuasive arguments during all debates. The verdict will then be followed by a Verdict Debriefing Session involving members of the jury and the debaters together.

This moderator guide can be used for each of the debates that will happen during the jury session. The format and time allotments for each debate are the same. Below, specific prompts have been provided for each debate to help guide the discussion and make sure participants stay on the topic.

1. The jury session begins with the Pro-ACS Advocates making their opening statement for Question 1 based on the arguments they prepared during the briefing. The team will speak for 15 minutes.

The moderator will also have a copy of the question handouts during these opening statements. He will intervene only as necessary in the openings of each team – as a judge in a trial would – to keep the lawyers’ statements focused on the core issues, and prompt them using the following:

Prompts for Debate 1:

1. What are the best arguments {for/against} the value of the American Community Survey?

• What do you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How did these arguments make you feel?

• Which of the arguments you saw were least convincing in support of the ACS?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any compelling arguments about the ACS missing from the materials you saw?

• For those of you actually less inclined to support the ACS, did you learn anything that might have a positive effect on your likelihood to support it?

Prompts for Debate 2:

2. Based on the materials you reviewed, which arguments most strongly motivated you to {complete/not complete} the survey?

• What did you like best about the arguments you’ve chosen?

• What is it about these arguments that most resonates with you?

• How do these arguments make you feel?

• Which of these arguments are must-haves? Which are nice-to-haves, and why?

• Are there any arguments about the benefits of completing the ACS missing from the materials you saw?

• For those of you less willing to complete the ACS, did you learn anything that had a positive impact on your willingness to participate?

2. After the Pro-ACS side makes its opening statements, remind the jury and the debaters that they will have an opportunity to cross-examine each team later.

Remember, both the jury members and the debaters will have a chance to cross-examine their opponents after each team has made its opening statements. You should be taking notes about any arguments that don’t seem fully believable or that you would like to further discuss with each team.

3. The Anti-ACS team will have 15 minutes to make their arguments.

The moderator will again intervene only as necessary to keep the opponents’ statements focused on the core issues and will prompt them with relevant points.

4. After both teams have made their opening statements, the moderator will lead a 10-minute cross-examination period where he/she will suggest challenges that the other teams can pose to each other.

5. After the teams have cross-examined each other, the moderator will instruct the jury members that it is now their turn to ask any of the questions that they have written down. The questions may be for either of the teams. This cross-examination period will last 10 minutes.

Jury members, take a minute now to review the questions that you wrote down while the lawyers were making their cases. Now is your opportunity to pose any challenges to the teams or ask follow-up questions based on what you heard. Be sure to ask about any key points that are influencing your decision over which side to vote for.

The moderator should keep track of time so that each jury member is allowed to pose his/her questions.

Intervene only if a jury member’s question is not in line with the key questions outlined in the handouts.

6. Now the debaters will leave the room to review their arguments for the second question. Once they are gone the moderator will act as the jury foreman while the jury deliberates and casts their votes.

Please take a moment to add any important notes to the form you were given prior to the debate. Also be sure to mark your initial vote for Question 1 on the form. Which side would you vote for and why?

How many of you voted for ACS? How many of you voted against ACS?

The moderator will use an easel to take notes on the jury deliberation. First, he will write down the number of initial votes for each team. Then, he will moderate a discussion between the jury members.

First, I’d like to hear from the people who voted for ACS. What made you pick this side and not the other? What are the most important reasons to {support ACS/fill out the survey}?

Now, I’d like to hear from the people who voted against ACS. What made you pick this side and not the others? What are the most important reasons not to{support ACS/fill out the survey}?

Once all jury members have offered their reasoning: Has what the other members said convinced anyone to change their vote? Why?

For the jurors who voted against ACS, are any of these reasons compelling enough for you to consider changing your initial vote? Why or why not? Are any of these arguments things you did not consider? Do they change your position at all?

What would you say to your fellow jury members to convince them that they should vote in support of {ACS/filling out the survey}? What factors have they ignored that make the case for or against {ACS/filling out the survey}?

7. After the jurors have deliberated, the moderator will take a final tally of the votes.

8. Once the final vote is over, the jurors are given a new handout to take notes on the next question.

Take a minute to read about the next issue we will be debating. The format will be the same as the last debate. You should take notes on this form in the spaces provided. Once everyone has read this form, I will bring in the lawyers and we’ll begin the next debate.

9. Repeat Steps for Question 2.

10. Once the jury members have voted on the winner of Question 2, they will be asked to vote on which side they think won taking into consideration all the arguments that were made.

11. The winning team is the set of debaters that receives the most votes across both debates (a tie is possible). Once the vote from the second debate is over, the advocates are told who won and told they will be given their additional prize after the Verdict Debriefing Session.

4. Verdict Debriefing SESSION

1. After both debates, the advocates and opponents are dismissed, and the moderator will conduct a final discussion of the “trial” and verdict. Jury members will be given the last handout to see which side of the argument they’re on.

We are now going to have a discussion about the debates we had today and your decision. Please take a moment to fill out the last form of the evening. In it, you should indicate which side of the argument you came down on and why.

2. Moderator will lead a discussion to wrap up the session.

Now, thinking about what you just wrote on your forms:

What were the most persuasive arguments in support of the ACS/completing the survey? Why?

What were the most persuasive arguments against the ACS/completing the survey? Why?

What was the most damaging evidence to the contrary side?

If you could only pick one side for both of the questions we have been debating, which would it be? What makes you choose this side?

5. Conclusion

Based on everything we discussed today, what is the one key takeaway that would make you most likely to consider supporting the ACS?

If you were going to persuade a friend to consider completing the survey, what would you say to him or her – what argument would you use?

Thanks and closing

18