Survey Template

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF MESSAGE TESTING FOCUS GROUP COLLECTIONS

TITLE OF INFORMATION COLLECTION:

(the collection that is the subject of the 10-day review request)

[ ] SURVEY [X] FOCUS GROUP [ ] WEBSITE USABILITY TESTING

DESCRIPTION OF THIS SPECIFIC COLLECTION

Specify all relevant information, including

1. intended purpose,

2. need for the collection,

3. planned use of the data,

4. date(s) and location(s),

5. collection procedures,

6. number of focus groups, surveys, usability testing sessions,

7. description of respondents/participants

Purpose of the Research, Need for the Collection, and Planned Use of the Data

FSA’s goal is to increase the number of borrowers in repayment status and to lower default rates. They seek to accomplish this by communicating more proactively, plainly and concisely with borrowers so they better understand what financial aid is, that they have options, and what those options are. This online focus group study will evaluate the appeal and usefulness of a selection of message themes developed by Market Connections and approved by FSA on topics related to student financial aid. The results from the focus groups will help to educate financial aid borrowers to make the best decision for their personal financial situation, and learn how to make debt manageable. The results will also guide subsequent survey development and, in the end, will provide additional insights into the “whys” behind the online survey responses. 

Specific objectives for the focus group phase of the research include: 

• Exploring general comprehension and perceptions of federal student aid communications among four target audiences.

• Evaluating a selection of written message themes focusing on federal student aid information, advice and solutions. Participants will gauge the messages based on perceived usefulness, value, and likelihood to take action, and will also provide feedback on the timing of the message given their needs and objectives. Anticipated areas for deeper exploration include: 

• Probing aspects of the messages that may encourage or impede users from fully reading the entire text or deeming it worthwhile (e.g., based on length, terminology, tone).

• Determining whether the messages influence users’ impressions of the topic (e.g., the types of aid available, repaying loans).

• Evaluating the extent to which users believe they have clear instructions regarding next steps for taking action or seeking additional information, and the impact of the messages on their likelihood to take action.

Dates, Locations, and Collection Procedures

The sessions will be conducted June 10, 11 and 12, 2014 (with dates subject to change pending date of OMB approval). The research will be conducted via six online focus groups (each 60 minutes in length).

Description of Respondents/Participants

The research will be conducted among four core audiences: student loan borrowers in repayment, borrowers in deferment/forbearance, borrowers that are delinquent/in default, and borrowers in grace.

Total Sample: 6 Online Focus Groups with 5-8 Participants per Group

1 Borrowers in repayment

2 Borrowers in deferment/forbearance

2 Borrowers that are delinquent/in default

1 Borrowers in grace

Borrowers in repayment must be currently repaying all student loans on time.

Borrowers in deferment/forbearance must have at least one federal student loan in deferment or forbearance.

Borrowers delinquent or in default must have at least one federal student loan in delinquency or default.

Borrowers in grace must have at least one federal student loan in grace, with repayment not yet required at this time.

Additional Information: Proposed Incentive

To ensure Market Connections’ ability to maximize the number of messages that FSA wishes to evaluate in the most cost-efficient manner possible, we had originally recommended an online focus group length of 90 minutes. Commensurate with this recommendation, we advised an incentive payout to participants of $75 each for borrowers in repayment, grace, deferment/forbearance, and delinquency/default.  (This is in line with the $75 incentive OMB has approved for this type of evaluation for previous FSA research among borrowers in repayment, current college students, high school seniors, and parents of high school seniors.) In all cases, the segments targeted for research represent different facets of the current or potential borrower community.

However, understanding that OMB has deemed a $75 incentive to be unacceptable for this study, we recommend the alternative of conducting a one hour group, paying each segment $50, which we feel is an adequate minimum amount. A highly respected online focus group facility, with whom we will partner for the recruiting effort, estimates that if we were to offer a reduced incentive of $50 for a 90-minute online focus group, show rates are likely to drop by one third, while the necessarily higher recruiting charges would negate any savings, take the project beyond budget, and still afford a very low probability that anyone would show for the groups. It is for this reason that we recommend a shorter one-hour duration at the $50 incentive level, and also advise against any further reduction of the incentive (i.e., to $40) for a one hour group. If limited to $40, we will make a best effort but can no longer guarantee our ability to recruit the participants. Please note that given the limited time available and the topics to be discussed in a one hour group, Market Connections will be able to evaluate only half the number of stimuli FSA would ideally like to test, thus undercutting the research effort.

Payment of a reasonable monetary incentive is a critical component of a qualitative research investment, and both our ability to recruit and the participant's likelihood to show up to the session depends primarily on an adequate incentive amount. For this study we will be screening distinct populations that must meet specific requirements for eligibility. It is therefore important to offer a reasonable incentive that encourages these harder to recruit people to show up to the session. 

The recruiting challenge applies especially to those in delinquency or default, both of which are targets of particular interest to FSA given the sensitive nature of their borrower status. With regard to the payment of an incentive for delinquent or defaulting borrowers, we strongly believe that like any other participant of a market research interview or group discussion, they are not likely to show unless they are paid. The principal behind compensating participants for their time is widely recognized by consumers, and is a decades-long practice in the market research industry. Indeed, not paying the delinquent or defaulting borrowers could be interpreted as insensitive or even discriminatory on the part of FSA to not provide a monetary incentive to those that happen to be in an awkward financial position, even if the rationale for not paying them was made with the best of intentions, such as not appearing to “reward” delinquent borrowers with a payment for their participation; or perhaps to avoid the appearance of swaying their responses during the group.

However, in this particular research study, where message statements are being evaluated, the objectives and the lines of questioning, do not boil down to any of the participants choosing one message over another, since it is possible for them to like or dislike both. More importantly, the value lies in including their opinion and understanding what themes, formats and types of communications are most likely to give them the impetus to take action to solve their loan repayment challenges. Without their insights, it will be very difficult if not impossible for FSA to develop and deliver communications based on anything other than guesswork.

Ultimately, if we were not to offer a monetary incentive to delinquent and defaulting borrowers (or any of the other segments) for a one hour group – and commensurate with research norms – we would again make a best effort but could not guarantee that anyone will show to the groups. At the very least, the charge for recruiting would increase because it would require more outreach to find people willing to cooperate. And even if someone were to agree to participate, they are far less likely to actually show up at the appointed time when they are not being paid in accordance with the requested time commitment. It is also very possible that our recruiters would not even attempt to recruit a one hour group under circumstances where no monetary incentive is offered.

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response or participation time in minutes (/60) = annual burden hours):

Category of Respondent	No. of Respondents	Participation Time	Burden
Participants completing a screener	60 (10 per group)	5 minutes (average)	5 hours
Screened but disqualified participants*	18 (3 per group)	3 minutes (average)	0.9 hours
Borrowers in repayment	8	60 minutes	8 hours
Borrowers in deferment/forbearance	16	60 minutes	16 hours
Borrowers delinquent/in default	16	60 minutes	16 hours
Borrowers in Grace	8	60 minutes	8 hours
Totals	78		53.9 hours

* An estimate of those who will fall outside of target segment parameters, thus not qualifying to participate in the focus group.

BURDEN COST COMPUTATION

Category of Respondent	No. of Respondents	Hourly Rate	Response Time	Totals
Borrowers in repayment	8	$40	60 minutes	$320
Borrowers in deferment/forbearance	16	$40	60 minutes	$640
Borrowers delinquent/in default	16	$40	60 minutes	$640
Borrowers in Grace	8	$40	60 minutes	$320
Totals	48	$1920	240 minutes	$1920

STATISTICAL INFORMATION

If statistical methods are to be used, fully describe the methodology, sample selection, expected response rates, and any other concepts needed to provide a full understanding of those methods.

Since this is a qualitative study, statistical methods will not be used.

NAME OF CONTACT PERSON: Monica Hill

TELEPHONE NUMBER: 202-377-4080

MAILING LOCATION:

830 First Street, NE Room 32D2Washington, DC 20202

ED DEPARTMENT, OFFICE, DIVISION, BRANCH:

Federal Student Aid

MESSAGE TESTING

ONLINE FOCUS GROUP SCREENER

[Recruit 8, for 5-8 to participate per group]

We are in the process of organizing online discussions on the topic of student financial aid, on behalf of the Department of Education’s office of Federal Student Aid.

The discussions will consist of about 6-8 people like yourself and a Moderator, and will take place online and last no longer than 60 minutes. Please be assured we are not trying to sell you anything; this is strictly for research purposes. Your participation and comments will be anonymous. As our way of thanking you for your time and opinions, you will receive up to $40 for participating fully.

Before confirming your participation, we would like to ask a few questions to determine your availability and the best discussion group for you.

1. Have you received any federal student loans to help finance your college education?

1. YES [CONTINUE]

2. NO [THANK AND TERMINATE]

2. Are you, or will you be, primarily or jointly responsible for making your student loan payments?

1 YES [CONTINUE]

2 NO [THANK AND TERMINATE]

3. Do you or anyone in your household work for any of the following types of organizations?
   [IF “YES” TO ANY, THANK AND TERMINATE]

1 Market research

2 Advertising

3 Higher education administration

4. Higher education finance or lending

5. Consumer banking or lending

6. Media such as TV, radio, or newspaper

4. To what extent are you interested in information, advice or solutions on financial aid for college, or the repayment of financial aid? Are you…

1 Very interested [CONTINUE]

2 Somewhat interested [CONTINUE]

3 Not very interested [THANK AND TERMINATE]

4 Not at all interested [THANK AND TERMINATE]

5. For all of your federal student loans requiring payment either at this time or in the near future, please indicate your current loan status. This information will only be used to determine the best group discussion to join and will not be used for any other purpose.

Select all that apply (rollover definitions are provided)

1. Currently repaying all federal student loans on time [CLASSIFY AS IN REPAYMENT]

2. Have at least one federal student loan in deferment [CLASSIFY AS DEFERMENT/ FORBEARANCE]

3. Have at least one federal student loan in forbearance [CLASSIFY AS DEFERMENT/ FORBEARANCE]

4. Have at least one federal student loan delinquent [CLASSIFY AS DELINQUENT/ DEFAULT]

5. Have at least one federal student loan in default [CLASSIFY AS DELINQUENT/ DEFAULT]

6. Have at least one federal student loan in grace, with repayment not yet required at this time) [CLASSIFY AS IN GRACE]

7 Have paid off all federal student loans [THANK AND TERMINATE]

8. Unsure [THANK AND TERMINATE]

6. The online discussion will take place in the evening on [PROGRAMMER, PIPE IN FOLLOWING DATES BASED ON CLASSIFICATION(S) WITH AN “OR” BETWEEN THE TWO OPTIONS FOR DELINQUENT/DEFAULT AND DEFERMENT/FORBERANCE]. Are you interested and available to take part during this time?

IN REPAYMENT Tuesday, June 10 ^th (6pm EST)

IN GRACE Tuesday, June 10 ^th (8pm EST)

DELINQUENT/DEFAULT Wednesday, June 11 ^th (6pm EST)

DEFERMENT/FORBEARANCE Wednesday, June 11 ^th (8pm EST)

DEFERMENT/FORBEARANCE Thursday, June 12 ^th (6pm EST)

DELINQUENT/DEFAULT Thursday, June 12 ^th (8pm EST)

1. [ALL EXCEPT DEL/DEF AND DEF/FOR] Yes, I’m interested and available to participate during this time.

2. [DEL/DEF] Yes, I’m interested and available to participate Wednesday, June 11 ^th at 6pm EST

3. [DEL/DEF] Yes, I’m interested and available to participate Thursday, June 12 ^th at 8pm

4. [DEF/FOR] Yes, I’m interested and available to participate Wednesday, June 11 ^th at 8pm EST

5. [DEF/FOR] Yes, I’m interested and available to participate Thursday, June 12 ^th at 6pm

6. [ALL] No, I’m interested but unavailable to participate then [THANK AND TERMINATE]

7. [ALL] No, I’m not available nor interested [THANK AND TERMINATE]

[PROGRAMMER: IF PARTICIPANT QUALIFIES FOR MORE THAN ONE SEGMENT, PRIORITIZE HARDEST TO RECRUIT SEGMENTS (DELINQUENT OR DEFAULT, AND DEFERMENT FORBEARANCE) UNTIL THAT QUOTA IS REACHED.]

7. What state do you reside in? [Drop down menu]
   
   

8. What is the highest level of education you have attained? [AIM FOR MIX]

1. High school diploma [THANK AND TERMINATE]

2. Some college [CONTINUE – NO QUOTA]

3. Trade school or proprietary school certificate[CONTINUE – NO QUOTA]

4. Associate degree [CONTINUE – NO QUOTA]

5. Bachelor’s degree [CONTINUE – NO QUOTA]

6. Master’s Degree [CONTINUE – NO QUOTA]

7. PhD or post-doctorate [CONTINUE – NO QUOTA]

9. Which of the following categories best describes your age?

1. 18-24 [CONTINUE]

2. 25-29 [CONTINUE]

3. 30-39 [CONTINUE]

4. 40-49 [CONTINUE]

5. 50-64 [CONTINUE]

6. 65 or older [CONTINUE]

10. What category best describes the area in which you live?
    [AIM FOR MIX. NO HARD QUOTAS.]

1. Urban

2. Suburban

3. Rural

11. Which of the following categories best describes your ethnic background? [AIM FOR MIX. NO HARD QUOTAS.]

Select all that apply

1 American Indian or Alaska Native

2 Asian

3 Black or African American

4 Native Hawaiian or Other Pacific Islander

5 White or Caucasian

6 Hispanic or Latino

7 Not Hispanic or Latino

8. Other (please specify): _______________________________

9. Prefer not to say

12. Are you the first generation in your family to attend college or a trade/proprietary school? [AIM FOR 1-2 FIRST GENERATION PER GROUP. NO HARD QUOTAS]

1. YES

2. NO

3. UNSURE

13. And which of the following categories best describes your annual household income, before taxes? [AIM FOR MIX. NO HARD QUOTAS.]

1 Under $20,000

2 $20,000 to $39,999

3 $40,000 to $59,999

4 $60,000 to $79,999

5 $80,000 to $99,999

6 $100,000 and over

7 Prefer not to say

14. Please indicate your gender: [RECRUIT UP TO 4 PER OPTION]

1 Male

2 Female

15. Do you currently have a computer available with high-speed or broadband connections available for your personal use?

1. YES

2. NO [THANK AND TERMINATE]

16. Have you ever read comments or posted messages on any type of online message board or discussion board, for either business or personal reasons?

1. YES [SKIP TO INVITATION]

2. NO

17. Would you feel comfortable expressing your opinions in writing in an online research forum, if you were given specific instructions on how these types of forums work?

1. YES

2. NO [THANK AND TERMINATE]

INVITATION:

Thanks for answering the qualifying questions! The group will take place [PIPE IN DATE/TIME EST – USE HIDDEN QUESTION IF NEEDED] and will last about 60 minutes. The purpose of the group is to talk about student financial aid and loan repayment information. Most people who take part in these kinds of discussions find them interesting and fun.

As mentioned earlier, we are not trying to sell you anything and you will receive up to $40 in appreciation for your time. If you logon to the discussion at least 15 minutes early, you will receive a $10 bonus on top of $30 for participating fully and answering all questions.

A. Can we count on you to participate?

• YES [CONTINUE]

• NO [THANK AND TERMINATE]

2. Will you agree to be contacted at the phone number you provide to confirm your participation a couple of days before the discussion?

1. YES

2. NO [THANK AND TERMINATE]

2. Great! We will send you logon access, passcode and further information via email by [PIPE IN DATE/TIME – USE HIDDEN QUESTION IF NEEDED]. The email to watch for will come from _______ and the subject line will reference Student Loan Discussion.

As part of participating in the online discussion, we will need your contact information for:

1. Incentive payment purposes

2. Reminder emails

3. Confirmation calls

First name: [TEXT BOX]

Last name: [TEXT BOX]

Email: [TEXT BOX]

Evening phone: [TEXT BOX]

Daytime phone: [TEXT BOX]

Mailing address: [TEXT BOX]

City: [TEXT BOX]

State: [TEXT BOX]

Zip code: [TEXT BOX]

Remember, we are counting on your participation in this group. If something comes up and you find you are not able to participate, please e-mail _______ so that we can schedule a replacement.

DISCUSSION GUIDE

MESSAGE TESTING ONLINE FOCUS GROUP

06/10~12/14

FINALVERSION 01 24 14

INTRODUCTION AND GROUND RULES

NOTE: SOME QUESTIONS & INSTRUCTIONS WILL BE SLIGHTLY ALTERED TO FIT OLFG FORMAT Hello, my name is Dave Glantz and I work for Market Connections. I'm the moderator for today's discussion. We'll be online for about 60 minutes, on behalf of the U.S. Department of Education’s office of Federal Student Aid, to chat about your impressions of information related to student financial aid. I am here to get your ideas, thoughts, and opinions on a range of issues related to this topic.

This session is confidential and will be used only for research purposes. We are conducting this research on behalf of the Department of Education’s Office of Federal Student Aid, and we have no stake in the outcome – my job is to write a report based on your comments. Our report will not identify individuals by name.

A couple things to keep in mind are: Don’t feel you have to tell me what others think, or what you think we want to hear – tell me what you personally think and feel. In that spirit, we are not here to try and come to any consensus, so if you all have different ideas and opinions, that is very important for me to know. In this type of discussion there are no wrong answers – just different opinions.

Also, in the interest of helping the pace of our conversation here are a few more tips:

• Be sure you are in an area with limited distractions. Having your full attention with the group will help us move along as quickly as possible.

• Don't worry about typos or misspellings.

• You can all type at once. There's no need to wait for others to finish typing.

• I will be sure to go slow enough so that you all have time to type and to read others’ comments. And if you have something to add to someone else's comment, please feel free to do so.

• I need to hear from everyone during the course of the conversation. Your opinions are important to me, but don’t feel like you have to answer every question.

• There may be times where I may have to interrupt you or cut you off. Please understand that I’m not being rude. It’s just that I have many questions to cover and want to keep us on track.

Finally, at various points in our conversation I’ll be posting a few written statements for you to evaluate. Each of these will open in a separate window. Please take a second to turn off your pop-up blocker (if you have it on) so that you can see the statements when I post them. Is everyone okay with that? 0:05

I I. WARM-UP, COMPREHENSION AND PERCEPTION OF FEDERAL STUDENT AID COMMUNICATIONS

Start by: 0:05 elapsed time: 0:25

Finish by: 0:30

• I would first like to have everyone introduce themselves. Please type your first name (or what you like to be called) and tell me the single greatest benefit to you personally of a college education.

• Within the past 2-3 years, how many of you recall receiving information on the topic of federal student aid? This type of information can include the types of loans available, repayment options, explanations of loan terms, obligations and penalties, and related topics.

• Generally speaking, do you feel that this information is easy or difficult to understand?

• [IF DIFFICULT] What makes you say that? (PROBE: is it the topic of federal student aid in general, or certain specific areas related to student aid that are difficult to understand?)

• [IF SPECIFIC AREAS] What areas in particular?

• [IF DIFFICULT] I want to ask this question in another way: it is the topic itself that is difficult to understand, or the way it is communicated/explained to you, or is it both?

• Generally speaking, to what extent is it is easy or difficult to act on the information you receive?

• What makes you say that?

• [PROBE IF DIFFICULT] Is any possible inaction on your part more likely due to practical constraints on your time or budget, or because the information does not clearly or persuasively spell out how to act on the information?

• To what extent do you feel the information you receive about federal student aid is written with your personal situation in mind?

• What needs to be emphasized or changed in these types of communications to make you more likely to read or act upon them? (PROBE: tone, length, terminology, timing (specify), mode of communication (e.g., by email, through social media, website, hard copy letter, video, chat, infographic, in-person with college advisors, other (specify))

0:30

I II. MESSAGE EVALUATION

Start by: 0:30 elapsed time: 0:30

Finish by: 1:00

For the remainder of our discussion, I’m going to have you review two written messages, which I’ll post in random order. Then I’ll ask you some questions about each message.

POST MESSAGE 1:

• Please take a minute or two to read the message. Then type the grade you would give this message (“A” means it’s highly useful to you and “F” means it’s not at all useful to you, given any loan-related issues or concerns you may have). Please also type the first thing you said to yourself when looking at this message.

• PROBE IF GRADE OF C, D, F: What needs to change or happen to raise your grade to an A?

• How valuable is this message in general for [borrowers in grace/borrowers in repayment/those deferring or in forbearance/delinquent or defaulted] aiming to gather information, advice or solutions on student aid/financing issues?

• How likely are you to act on the information or instructions communicated in this message?

• What specific action(s) would you take? (PROBE: Would you click on a link?)

• Is there anything about this message that would cause you to stop reading or feel it’s not worthwhile to spend time on? What exactly?

• Briefly, as someone in [repayment; grace; deferment/forbearance; delinquent/default], if you were to receive this message right now, how would you feel about the timing (e.g., is the timing right, too soon, or too late to help you?)

0:45

POST MESSAGE 2:

• Please take a minute or two to read the message. Then type the grade you would give this message (“A” means it’s highly useful to you and “F” means it’s not at all useful to you, given any loan-related issues or concerns you may have). Please also type the first thing you said to yourself when looking at this message.

• PROBE IF GRADE OF C, D, F: What needs to change or happen to raise your grade to an A?

• How valuable is this message in general for [borrowers in grace/borrowers in repayment/those deferring or in forbearance/delinquent or defaulted] aiming to gather information, advice or solutions on student aid/financing issues?

• How likely are you to act on the information or instructions communicated in this message?

• What specific action(s) would you take? (PROBE: Would you click on a link?)

• Is there anything about this message that would cause you to stop reading or feel it’s not worthwhile to spend time on? What exactly?

• Briefly, as someone in [repayment; grace; deferment/forbearance; delinquent/default], if you were to receive this message right now, how would you feel about the timing (e.g., is the timing right, too soon, or too late to help you?)

0:55

Wrap up

• What final advice would you give to FSA to help them provide informative and relevant communications that resonates with you?

Those are all the questions I have for you. Thanks for participating! 1:00

Market Connections, Inc. Proprietary and Company Confidential Page 14 of 14