Appendix E NCER-NPSAS Grant Study-CSFA Focus Group Report

NCER-NPSAS Grant Study

Connecting Students with Financial Aid (CSFA) 2017: Testing the Effectiveness of FAFSA Interventions on College Outcomes

Appendix E

Focus Group Final Report

OMB # 1850-0931 v.2

Submitted by
National Center for Education Statistics
U.S. Department of Education

December 2016

NCER-NPSAS Grant Study

Connecting Students with Financial Aid (CSFA) 2017: Testing the Effectiveness of FAFSA Interventions on College Outcomes

Focus Group Final Report

December 2016

TABLE OF CONTENTS

Section		Page
1.0	Background and Research Methodology	2
2.0	Research Objectives	5
3.0	Conclusions and Implications	5

1.0 BACKGROUND AND RESEARCH METHODOLOGY

Background and Purpose

• In 2010, the National Center for Education Research (NCER) and the National Center for Education Statistics (NCES), both within the U.S. Department of Education’s Institute of Education Sciences (IES), began collaborating on an education grant opportunity related to the cross-sectional National Postsecondary Student Aid Study (NPSAS). Under the NCER-NPSAS grant opportunity, researchers could submit applications to the Postsecondary and Adult Education topic within the Education Research Grants program (CFDA 84.305A) to: 1) explore relationships between malleable factors (e.g. information on benefits of financial aid and FAFSA renewal) and postsecondary persistence and completion, as well as the mediators and moderators of those relationships; and 2) evaluate the efficacy of interventions aimed at improving persistence and completion of postsecondary education. Researchers approved for funding through this program can obtain indirect access to a subsample of the national NPSAS sample (after the study’s student interviews are completed) in order to conduct unique research projects that adhere to the guidelines set forth in the Request for Applications for the Education Research Grants Program, as well as guidelines set forth by NCES and the NPSAS program.

• On July 1, 2016, a grant was awarded to this project: Could Connecting Students with Financial Aid Lead to Better College Outcomes? A Proposal to Test the Effectiveness of FAFSA Interventions Using the NPSAS Sample (referred to as “Connecting Students with Financial Aid (CSFA) 2017”; http://ies.ed.gov/funding/grantsearch/details.asp?ID=1853). The CSFA 2017 study investigates whether an intervention that provides financial aid information increases completion of the Free Application for Federal Student Aid (FAFSA). In addition, information will be provided on how the number of college credits taken can increase the amount of financial aid received to see if this information influences enrollment intensity (full- versus part-time status). The primary grantee is Bridget Long, Harvard University (Grant Award #R305A160388), and the co-principal investigator is Eric Bettinger, Stanford University. Data collection for the study will be led by the contractor, Research Triangle Institute (RTI).

• The research team proposed to recruit for and conduct with postsecondary students two to four focus groups for the CSFA 2017 study to obtain information on how students understand the communications, whether the letters prioritize the most important information to provide, how the framing and formatting of the message influences how the messages are received, and to refine the informational letters/emails to be used in the study’s interventions accordingly. The focus groups were cleared under OMB # 1850-0803 v.184.

• Feedback from this research guided development of the revised intervention materials to be used in the CSFA study.

Research Methodology

• A total of two, 60-minute focus groups were conducted in small conference rooms at the Harvard Graduate School of Education:

Group #	Date	Time
1	Thursday, December 15th	10am-11-am
2	Thursday, December 15th	1:30pm-2:30pm

• Respondents were recruited by Dr. Long from among former students in her course “The Economics of Higher Education.” The course attracts a number of students who have professional experience working with the target population of low-income and first generation undergraduate students and nearly all the students are interested in the issues of college access and success (the main topic of the course). Many are also first-generation students themselves and so can share that perspective from personal experience.

• On December 13, 2016, Dr. Long announced the focus group opportunities using the email listserv of her former class, which ended in October 2016. Interested students responded to an online poll with their availability to participate (three possible times were given). Based on student availability, Dr. Long selected two times that would include a diverse set of focus group participants from among those that responded. One respondent was not included in the focus groups because she was unavailable during the two selected times.

• For documentation purposes, the recruiting disposition is included below:

Focus Groups
Number of responses to the call	10
Number of recruits not available	1
Number assigned to Focus Group 1	5
Number of no shows to Focus Group 1	0
Number assigned to Focus Group 2	4
Number of no shows to Focus Group 2	0
Final number of interviews conductions	2 Focus Groups 9 Participants

Limitations

• A qualitative research methodology seeks to develop direction rather than quantitatively precise or absolute measures. The limited number of respondents involved in this type of research means the results should be regarded as directional in nature and be used to generate hypotheses for future decision making.

• The non-statistical nature of qualitative research means the results cannot be generalized to the population under study with a known level of statistical precision.

• The recruited participants differ from the NPSAS sample that will eventually receive the informational interventions. The focus group participants are students in a master’s program in education, and though they recalled their professional experience with the target population and their own personal experiences, they may not accurately reflect the feedback we would have received from a nationally-represented group of current undergraduates.

• Due to the timing of the focus groups, which coincided with Harvard’s final exams, many students were unavailable to participate due to competing demands or because they were no longer on campus (i.e., they had finished their exams and gone home for the winter break). This affected recruitment slightly, though we were still able to secure a sufficient number of focus group participants to get meaningful feedback on the intervention letters.

2. RESEARCH OBJECTIVES

The specific study objectives are to:

1. Share a sample intervention letter, a shorter version of the letter, possible alternative language, and the supplemental handout.

2. Discuss whether participants understand the communications.

3. Check to see if the letters prioritize the most important information to provide.

4. Explore how the different ways of framing the information (i.e., neutral, positive, or negative) influences how the messages are received.

5. Discuss how the formatting of the letter influences how the messages are received.

While the sample documents were shared via email when confirming the focus group time, we also provided hardcopies of all the materials during our discussion.

2. CONCLUSIONS AND IMPLICATIONS

Information Provided

The focus groups both confirmed that we have prioritized the right information to share about the importance of completing the FAFSA and how to get started. Participants found the information provided to counteract myths about the financial aid process to be very strong. The sample letter included quotes from fictitious students that the group found to accurately reflect the kinds of questions they have heard from students about the financial aid process.

The group cautioned us to be sure to define all terms, such as what a Pell Grant is and the fact that it does not need to be repaid (as opposed to a loan). They also affirmed the importance of emphasizing that the FAFSA is free to complete and free to submit. They encouraged us to mention this at every opportunity.

Letter Organization

While the focus groups confirmed that we are emphasizing the right information, but they suggested that we move up the text about financial aid myths. Their reasoning is that by first addressing students’ misperceptions, we might have a better chance of the student being willing to read the rest of the letter.

When discussing how the number of college credits taken relates to the size of the financial aid award (i.e., students can get a larger Pell Grant award if they take up to 12 credits per term), participants suggested fore fronting the potential benefit of finishing their studies earlier under the assumption that many students want to finish college as fast as possible if they can afford it.

Letter Tone

Both focus groups suggested using more of a conversational tone. For instance, rather than writing “According to our records…” the groups suggested we instead write: “We’ve noticed that…”

Letter Formatting and Graphics

The focus groups like the use of the NCES, U.S. Department of Education letterhead. They also liked the use of a graphic in the middle of the page. However, while a couple of participants liked the chalkboard graphic, most others suggested finding a different graphic to convey the information about aid myths. Some like the idea of using thought bubbles and a few suggested creating a graphic that include pictures of students. They also encouraged the used of different fonts and colors to attract the attention of students. If possible, they affirmed our goal of trying to keep the letter to mostly one page.

Connection to Resources

The participants were happy to see the letter attempt to connect students with resources. They liked having the web address, phone number, referrals to the school financial aid office, and the handout. However, all participants mentioned that the web address should not be a random collection of letters—it should instead be something intuitive (e.g., “/FAFSA” or “/financialaid”). They did like the use of the “.gov” as part of the address to lend credibility to the site being a good source. Several participants also liked the idea of including the QR code on the hard copy letters, though it is unlikely that it will be used by all students.

Information on the Handout

Participants affirmed our goal of having the handout provide additional details that would not fit in a one-page letter. For example, they think it is appropriate for the handout to have such details as what is needed to complete the FAFSA. Several participants also said that it would be a good place to define important terms.

E-8