My PAA Focus Groups

D ate: ____________ Moderator: ____________

PBGC – Focus Group Guide

Draft August 30, 2011

Version 0.4

Prior to session:

• Write introduction questions on the board.

• Ask participants to complete the background questionnaire; collect these for reference before beginning the session

• Give participants response sheet

Session Introduction

[Moderator’s Note: Give participants the Participant Background Information sheet (next page) to complete when they arrive. Collect before beginning discussion.]

Thank you for agreeing to participate in our study. The purpose of this study is to learn about your experience with the My Pension Administration Account (My PAA) application on PBGC’s website. The ultimate goal is to make the application easier to use.

Paperwork Reduction Act notice: This session has been approved by the Office of Management and Budget under OMB Control No. 1212-0066 (expires 06/30/2014). We need to obtain OMB approval and to tell you we obtained that approval in order to conduct this session. Your participation is voluntary and should take about 90 minutes.

Confidentiality: Your responses will be used by PBGC solely for customer service improvement and will be treated with the confidentiality provided by the Privacy Act and the Freedom of Information Act.

During this session we will look at some My PAA screens and some website pages. I will ask for your feedback on these screens and pages, I might ask you to write down some ratings on a sheet of paper, but most of the time we’ll be talking out loud as a group.

As you can see, we have video cameras taping the session, so I can go back to the tape if I forget anything that we talked about. We are also using the cameras to project into another room where my colleagues are observing and taking notes.

I would like to ask you to be as honest as you can. This information will not be shared with anyone other than the Pension Benefit Guaranty Corporation, and will be used only as information to help them improve My PAA and the website.

Here are some rules for our discussion today.... I’m interested in what each of you has to say, so I would like only one person to speak at a time. If you are talking while someone else is talking, I will ask you to wait your turn. This is a discussion, so feel free to comment on what others say. Just remember that everyone’s opinion counts, and there are no right or wrong answers.

Participant Background Information

Paperwork Reduction Act notice: This session has been approved by the Office of Management and Budget under OMB Control No. 1212-0066 (expires 06/30/2014). We need to obtain OMB approval and to tell you we obtained that approval in order to conduct this session. Your participation is voluntary and should take about 90 minutes..

Confidentiality: Your responses will be used by PBGC solely for customer service improvement and will be treated with the confidentiality provided by the Privacy Act and the Freedom of Information Act.

Please complete the following information:

1. Your name (first name only): _______________________

2. Where you work (for example, a large actuarial consulting firm): ________________________

3. Are you a(n) (you may select more than one):

   1. Third-party administrator (TPA) or benefit consultant

   2. Actuary

   3. Employee of a plan sponsor

   4. Other (Specify____________)

4. Who typically oversees the overall e-filing process (you may select more than one):

   1. You

   2. Your staff

   3. Your client’s staff

   4. Others (explain)

5. Who uses My PAA (you may select more than one):

   1. You

   2. Your staff

   3. Your client’s staff

   4. Others (explain)

6. What do you, your staff, or your clients typically use My PAA for (you may select more than one)

   1. To prepare premium e-filings using the My PAA screens

   2. To import premium e-filings created with private-sector software

   3. To upload premium e-filings created with private-sector software

   4. To view plan account histories

   5. To view plan filing receipts to verify what was previously submitted

   6. Other (explain)

7. For screen or imported filings, what My PAA roles typically apply to you (you may select more than one):

   1. Actuary

   2. Filing Coordinator

   3. Preparer

   4. Payer

   5. Plan Administrator

   6. Other

   7. I am not involved in screen or imported filings

8. For uploaded filings, how are you typically involved (you may select more than one):

   1. Preparer using private-sector software

   2. Uploader

   3. Payer

   4. Other

   5. I am not involved in uploaded filings

Participant Introductions (5 Min)

Before we get started, I would like us to get to know each other a little better. Let’s go around the room and introduce ourselves.

I will start. I’m _______________. I work for User Centric, an independent consulting firm that helps other companies improve their products. I don’t have nearly as much knowledge about your jobs as you do, so I may ask you questions throughout today. Observing from the adjacent room we have [introduce observers and their roles].

So now let’s go around the room and introduce ourselves. Please say your name, where you work, and how you typically use My PAA.

Overall Premium e-Filing Experience (Do not exceed 10 Min)

To start, I want you to think about your experiences using My PAA and provide some overall feedback. We will ask some questions about PBGC’s website, PBGC.gov, later in our discussion, but for now we are going to focus on My PAA, the secure application you log in to for premium e-filing.

[Moderator: Show blank screen.]

Using the sheet of paper you have in front of you, I’d like you to write answer question 1.

1. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate your overall experience using My PAA? [Moderator’s note: worksheet is only for the numerical scores. All other questions will be for group discussion.]

Ok, let’s talk about the My PAA application.

What do you like the most?

Responses:

What do you like the least? What have you had difficulties with?

Responses:

General Usability (Do not exceed 60 min)

[Moderator’s note: This is a walkthrough of the main screens that a user would encounter: Log-In Page, Home Page, Create a Filing Screen, the Filing Manager Page, the Plan Page, and a potential new Launch Page.]

Now we are going to look through a set of pages. I should note that depending on what kinds of tasks you perform on the site, you might not have seen all of these pages before. [Moderator’s note: If time is tight, can omit listing of pages; perhaps show on a flipchart.] Still, the types of pages that we look at should be familiar to many of you, they are:

• The Log-In Page

• The My PAA Home Page

• A Create a Filing Screen

• The Filing Manager Page

• The Plan Page

• A potential new Launch Page

We’ll be asking the same basic set of questions for each of these screens and pages to help PBGC focus on the aspects that are most challenging for premium filers. As you look at each page, please let me know both what is working well and what could be improved.

Let’s take a look at the Log-In Page to start. Go ahead and answer question 2 on your sheet:

2. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this page?

What do you like most about this page?

Responses:

How can this page be improved? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

Now let’s look at the My PAA Home Page. Go ahead and answer question 3 on your sheet:

3. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this page?

What do you like most about this page?

Responses:

How can this page be improved? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

If you have questions while you are on this page, what do you typically do? [Prompt if needed: Do you use the links? When do you use the links? Do you call or email PBGC? Note that this question is a general question that applies across all the pages in this section. This question needs to be addressed at least once as we go through and at any other point where a respondent makes a specific remark about the links. ]

Responses:

[If not covered in the question above] Do you use the Instructions or About Account Histories links or the resources linked in the header (the Users Manual) and footer (About Online Premium Filing or Contact Us)? Why or why not? How can they be improved? [Note that this question is a general question that applies across all the pages in this section. This question needs to be addressed at least once as we go through and at any other point where a respondent makes a specific remark about the links. ]

Responses:

[Show Create a Filing screen]

Ok now we’re looking at one of the My PAA screens used to create a filing. Please answer question 4 on your sheet:

4. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this screen and subsequent My PAA screens to create a filing?

What do you like most about this screen and the other screens used to create a filing?

Responses:

How can this and other filing creation screens be improved? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

[Show Filing Manager]

Ok now we’re looking at the Filing Manager Page. Please answer question 5 on your sheet:

5. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this page?

What do you like most about this page? What is most useful?

Responses:

How can this page be improved? What is least useful? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

[If not covered in questions above] Do you use the in-line instruction links on this page? Why or why not? How can they be improved?

Responses:

[Show Plan Page]

Ok now we’re looking at the Plan Page. Please answer question 6 on your sheet:

6. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this page?

What do you like most about this page? What is most useful?

Responses:

How can this page be improved? What is least useful? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

[Show prototype Launch Page] This screen is being considered to help filers, especially new or infrequent filers, navigate My PAA more easily. We would like to get your thoughts as to whether it would be helpful to see this after you log in, prior to the Home Page. Please answer question 7 on your sheet:

7. On a scale from 1-5, where 1 is completely unnecessary and 5 is very helpful, how would you rate usefulness of adding such a page?

What do you like or not like about this page?

Responses:

[Show Online Premium Filing With My PAA page on PBGC’s website]

Ok, now we’re going to shift gears a little and go outside the My PAA screens and look at the Online Premium Filing Page on PBGC’s website, PBGC.gov. Please answer question 8 on your sheet:

8. On a scale from 1-5, where 1 is poor and 5 is excellent, how would you rate the overall ease of using this page?

What do you like most about this page? What do you use the most?

Responses:

How can this page be improved? [Prompt if needed: Is there anything that should be here but isn’t? Anything that’s here but hard to find or inconveniently placed or labeled in a confusing way?]

Responses:

Communication From PBGC (5 Min)

Now let’s talk a little about the communications you receive from PBGC.

What do you receive from PBGC? Is it helpful? Is there information you need but aren’t getting?

Responses:

If there are issues with the premium filing, PBGC typically mails a letter to the plan administrator using the contact information provided in part 1 of the filing.  PBGC has been told that sometimes these letters do not always get delivered to the person who needs to handle it. Do you have any suggestions to help PBGC get filing information to the right party?

Responses:

[Time check for the moderator. If there is sufficient time to cover optional items, skip to optional items and discuss. Leave at least 5 min to close out discussion.]

Closing Comments (5-10 Min)

Would you like to use My PAA for any additional tasks or filings? If so, what else would you like to be able to do in My PAA?

[Moderator’s Note: Let participants volunteer; if no one mentions it, ask about stand-alone payments, paying statements of account (SOAs or bills), and updating names and addresses.]

Responses:

Considering everything, what are the one or two changes you would most like to see in My PAA? What are the most important improvements to you?

Responses:

Any final thoughts or comments that you would like to communicate to the Pension Benefit Guaranty Corporation?

Responses:

Please give me a moment to discuss with my colleagues and see if they have any final questions.

Ok, that’s all the time we have today. Thank you for coming in to talk with us today, your feedback is appreciated and will help make My PAA better!

Optional Questions

Let’s talk about getting filing assistance. If you needed additional filing assistance, how would you contact the Pension Benefit Guaranty Corporation? [Moderator note: Prompt if needed: Do they call the practitioner toll-free number or send an e-mail to “premiums”, or other?]

Responses:

Take a minute to think about your customer support needs. What could be done to better support those needs?

[Moderator’s Note: Try to uncover any support changes that could be made, even non-website related.]

Responses:

Thinking about system performance, are there any areas that need to be improved? What screens or functions are involved?

[If discussion is not moving forward, probe further: for example, speed of loading pages, outages, technical difficulties messages, etc.? Try to get specifics on where the performance issues are, that is, which particular screens or functions.]

Responses:

Information on PBGC.gov

Now we’re going to talk briefly about some of the information on PBGC’s public website, PBGC.gov. .

[Moderator: Show PBGC.gov Home Page.] . PBGC made changes to the look and feel of pbgc.gov earlier this year. What comments do you have about the changes?

Responses:

Premium Payments

We’re going to talk briefly about premium payments. In your experience, are premium payments typically made via:

• My PAA (whether that be through automated clearing house (ACH) or electronic check)

• Outside of My PAA (ACH, wire, or paper check)

Responses:

Tell me more about your experience with how premium payments are made. How can the payment process be improved?

Responses:

For those of you who make payments or work with people who make payments outside of My PAA, what would motivate you or your clients to submit payments via My PAA?

Responses:

Look and Feel of My PAA

Is there anything about the look and feel (amount of text, graphics, and white space; colors; text style and size; layout) that you would like to see changed?

Responses:

Response Sheet

First Name: ______________________________________________

Question #	How would you rate the overall ease of using this page (1=Poor, 5=Excellent)	Comments
1
2
3
4
5
6
7
8

© 2011 - User Centric, Inc. - PBG-600 Proprietary and Confidential DRAFT v0.4 - Page 11 of 11