Passback to OMB questions

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

OMB # 1850-0803 v.191

DATE: March 13, 2017

TO: Robert Sivinski and Ann Carson, OMB

THROUGH: Kashka Kubzdela, OMB Liaison, NCES

FROM: Tracy Hunt-White, Team Lead, Postsecondary Longitudinal and Sample Surveys, NCES

SUBJECT: NCER- NPSAS Grant Study – Connecting Students with Financial Aid (CSFA) 2017 Cognitive Testing - Responses to the OMB Passback (OMB # 1850-0803 v.191)

March 1, 2017 - OMB Passback

On March 1, 2017, OMB provided passback for the CSFA 2017 Cognitive Testing request for testing the CSFA post-intervention survey instrument. The survey questions regarding which OMB provided passback are listed below in bold, with the OMB-questioned content highlighted in yellow (when applicable). The OMB passback and then NCES responses follow. Please note that we added the current survey question numbers for reference.

Below OB passback and NCES responses we also added a section on the additional changes we made to the survey instrument to be cognitively tested. As we worked to clarify survey items, response option, and skip logic, we identified additional areas for possible improvement that we would like to test during the CSFA cognitive interviews.

1. Survey Questions Q4 and Q9: If you have ever applied for government financial aid, how did you learn about the process of applying? Check all that apply.

OMB Question: Are you referring to federal only aid, or state and federal?

NCES Response: The term “government financial aid” is referring to both state and federal aid. We have modified the existing survey instrument to use clarifying language where relevant to indicate “federal and/or state” government financial aid. In the cognitive interviews, we will test respondents understanding of this term and whether this specificity is needed. Also, this question has been divided into two questions (Q4 and Q9):

- Q4: Prior to 2017-18, have you ever applied for state or federal government financial aid to help you with college expenses?

- Q9: How did you learn about the process of applying for government financial aid?

2. Survey Question Q7: Do you know what the FAFSA is?

OMB Question Part 1 (Suggested addendum): and have you applied for it before?

NCES Response: Using skip logic, the only respondents answering this question are those that indicated in the previous question that they did not apply for government aid for the 2017-18 academic year. The primary intention of this question is to determine whether respondents are familiar with the acronym and existence of the “FAFSA” because awareness of the form and the financial aid process in general is thought to be a barrier to getting aid.

OMB Question Part 2, Reference to Response Option: “Yes, I know what the FAFSA is, but I don’t know how to complete the form.” Why is not knowing how to fill it out the only reason listed here for not filling it out?

NCES Response: To simplify the question, we have changed the responses to ask whether the students ever filled out the form. Two additional questions (i.e., Q8a and Q8b) probe students with a variety of possible reasons for not completing the form. We will explore if additional options should be listed in the cognitive interview.

3. Survey Question Q10: (Display Logic: If a student has ever applied for government aid)

We’re interested in your experiences with applying for financial aid. Did you receive assistance from another person to complete the FAFSA the last time you did so? If yes, please indicate from whom. Check all that apply.

OMB Question Part 1, Reference to the use of “government” as a descriptor of financial aid in skip logic programming statement (highlighted above) - Federal and state? Federal only? What do you do with people who only applied for state aid?

NCES Response: The term “government aid” is referring to both state and federal aid. Students who did not apply for government (federal and/or state) aid will not receive this question. State financial aid programs also require completion of the FAFSA so this question is still relevant for those who only receive state aid. We have amended the survey to include “federal and/or state” as a descriptor throughout the survey when discussing government aid.

OMB Question Part 2, Reference to FAFSA: Again, what if they opted to only do state aid?

NCES Response: This question is asking about the application process and completing the FAFSA, which is required to apply for federal and/or state aid. Specifically, state financial aid programs also require completion of the FAFSA so this question is still relevant for those who only applied for state aid.

4. Survey Question Q14: We’re also interested to know your opinions about the financial aid process. Please select one response to each statement.

OMB Question Part 1: Do these [questions] only get asked of people who did apply for financial aid?

NCES Response: That is correct. We have clarified who will be asked this question with the note about the Display Logic preceding Q12. Many of the statements within this block would be irrelevant to a respondent that did not apply for aid.

OMB Question Part 2, Reference to Response Option: “I lost money by not completing the FAFSA this year” I don't understand why this is here. If these questions are being asked of people who DID apply for government aid, why have this statement saying they didn't?

NCES Response: This question will now only be asked of students who did not apply for financial aid for 2017-18. For better logical flow, this possible response has been moved up to Q8b.

5. Survey Question Q18: Are you taking, more, fewer, or the same credits as the last time you enrolled in college courses?

OMB Question: What about people who are in their first semester?

NCES Response: Our sample is limited to students who were in college during 2015-16 so they will be past their first semester of college when the survey is conducted in Fall 2017.

5. Survey Question Q25: We’re also interested to know how financial aid affects your decisions about college. Please select one response to each statement.

OMB Question Part 1, Reference to Response Option 25a: “I’m attending my current school because of the financial aid they were able to give me. They? Who? The school? The feds?

NCES Response: The use of “they” was intended to reference the financial aid award package described in the student’s financial aid award letter from his/her college, which may include a description of federal, state, and institutional sources. Students frequently interpret these offers as coming from the institution because of the structure of the award letters. However, to make what we are referencing absolutely clear, we have changed the text to: “…the financial aid the school and government were able to give me.” We will test this during the cognitive interviews to check for clarity.

OMB Question Part 2, Reference to Response Option 25b: “I’m attending my current school because the available financial aid did not allow me to attend my 1^st-choice school” In total? School+state+fed? Or just fed?

NCES Response: This question is exploring whether financial aid impacted the decision a student made about where to attend. It is referring to total financial aid, which as described in the previous response, is how students tend to interpret the aid package they are offered by a college. The more savvy students will differentiate between grants and loans, but regardless, the goal of the question is to explore the effect of the perceived aid package on college choice.

5. Survey Question Q31: We’re interested to know what you think about your loans. Please select one response to each statement. Statement two: When I took out my loans, I was confident that I would be able to repay them.

OMB Question: Ever? Perhaps put a time limit on this?

NCES Response: There are a series of questions designed to learn more about student perceptions and use of loan options to finance college. Most college students are not very familiar with the terms of student loans and tend to focus on whether they can make the monthly payment instead of thinking about length of time of repayment. With the continual shifting of loan terms and repayment plans, adding more details would likely confuse the respondents. This group of questions is really meant to measure perceptions about ability to pay, not specific loan payment calculations. However, we have responded by expanding the number of related questions to try to understand how students felt about their loans when they originally took them out versus at the time of the survey.

Additional Changes to the Survey Instrument

Based on comments from OMB and requests for clarification of the cognitive interview review process, we have made additional adjustments to the survey instrument.  As discussed above, these adjustments include revisions recommended by OMB, clarifications of the skip and display logic, and other changes designed to improve the flow (e.g., splitting questions into separate parts).  Additionally, we have added several questions designed to solicit information related to the recent policy changes that allow families to use prior-prior-year data on the FAFSA (as opposed to prior-year data).  This is the first year of this new policy, and during the past few months we worked to develop questions designed to inquire about the implementation of the new form. The following are the revisions we have made to the survey and plan to test with participants during the cognitive interviews:

• Survey questions have been re-calibrated to include separate question numbering for the nested inquiries present in the original items. For example, questions utilizing a Likert scale for a series of statements are each identified with a unique question number.

• The survey organization has been adjusted to address some confusion in the skip versus display logic of the online survey tool. This has resulted in some questions being split apart, but retaining the same content and intention.

• We have made changes to word choice where OMB indicated that the questions or response options were not clear or could be misinterpreted. For example, we have included the descriptor "federal and/or state" to clarify what is meant by "government" financial aid.  We will examine word choices in those items during cognitive interviewing to determine how to maximize respondent understanding during survey implementation.

• We have added questions that directly examine the impact of our current information intervention, specifically the information we provide to students about the need for annual renewal of FAFSA applications (Q5) and about the impact of credit/course load on the amount of Pell Grant aid they may be eligible for (Q20).

• Due to the unique circumstances of unpredictable and significant policy changes during the past several months, we have also included a few questions that were not present in our initial submission that was approved by OMB in January 2017 (OMB# 1850-0931 v.1).  These questions are specific to the implementation of using Prior-Prior-Year tax information for FAFSA completion (Q15, 16) and the Early FAFSA schedule for the 2017-2018 school year (Q6a, Q6b).  Both policy shifts have direct implications on the CSFA study and these questions are designed to provide important perspective on the student awareness of available financial aid and their experiences in applying for financial aid.

• In light of the unexpected shutdown of the IRS Data Retrieval Tool for FAFSA completion in March 2017 (anticipated to last several weeks as of 3/10/2017), we have added questions to explore the impact of this development on respondents. The first block (Q11) of questions (for all respondents) examines the potential impact of the IRS DRT shutdown on 1) overall FAFSA submission, 2) potential delay of FAFSA submission, and 3) method of FAFSA submission. The second block of questions are a sub-inquiry of Q12 respondents who filed the FAFSA online as to whether they used the IRS DRT and, if they did not, what were their reasons for that decision. The last sub-question (Q12d) asks FAFSA paper filers why they chose to file on paper vs. online.