Gainful Employment Focus Groups - Students, For Profit and Public Institutions

Attachment D – Institutional Representative GE Moderator Guide

Westat Moderator’s Guide

NCES Gainful Employment Focus Group

For Institutional Representatives From Proprietary Schools and Community Colleges In the DC Metropolitan Area

Revised September 17, 2015

Welcome

Good evening. My name is _______ and I am a researcher at Westat, a social science research company in Rockville, MD. I will moderate tonight’s focus group. Thank you for taking time to participate. First, let me remind you that your participation in this focus group is voluntary.  If you choose to participate, the information you provide will be confidential in that we will not include your name or the name of your school in the study report.

Now, let me tell you about the purpose of this study.

Introduction to the Study

U.S. Department of Education has contracted with Westat to work on a project related to the Gainful Employment regulations that will take effect in 2017.  Under these regulations that were issued in October 2014, postsecondary institutions like yours are required to disclose to current and prospective students information about their institution’s career and training certificate programs through a disclosure template developed by the Department.  Please note that the Department will provide some of the information while institutions are responsible for collecting and providing other information.

This evening’s discussion will focus on two issues. First, we will discuss the template items to solicit your general perspectives about whether you think the items are clear and if you think additional information needs to be provided, such as specific definitions or instructions, to help institutions complete the information. As part of this discussion, we would like you to rank the relative importance of each item by identifying those that you deem most critical to collect and enter into an updated GE template.

Second, we will ask you to respond to the student warning language contained in the regulations that institutions are required to provide if “a GE program could become ineligible based on its debt to earnings rates measure in two out of any three consecutive award years for which the program’s D/E rates are calculated.”

The information you share in this group will be summarized in a report and shared with the Department of Education’s Office of Postsecondary Education. This information will help the Department decide which items should be included in the 2017 GE template. The new disclosure requirements are effective January 1, 2017.

Focus Group Guidelines

1. There are no right or wrong answers. We value a variety of opinions and experiences.

2. Please talk one at a time.

3. Talk in a voice as loud as mine.

4. Avoid side conversations with your neighbors.

5. I need to hear from everyone; however, you do not have to answer every question.

6. It is OK to piggyback on someone else’s comments.

7. Work for equal “air time” so that no one talks too little or too much.

8. Information shared in the group will remain confidential; the report will only summarize the responses and will not identify you by name or school. Help us protect your confidentiality by only stating your first name and NOT mentioning the name of your school.

9. We request that you not share the information discussed with anyone outside of the group until all the data have been collected and the web pages are finalized by the Department of Education, which will not occur until 2017. [Get verbal agreement from everyone.]

10. Staff from both the Department of Education and Westat are observing and taking notes on the discussion behind the one-way mirror. The session will be audio recorded.

11. Please turn off cell phones/Blackberries, etc. or turn them to vibrate.

Any questions before we begin?

Warm-Up and Opening Questions

OK, let’s get started by going around the table and introducing ourselves. Please give your first name only and tell me your official job title and how long you’ve worked at your school. To keep the names of your schools anonymous, please do NOT mention the name of your school or your last name.

First, let’s talk a little about your data-related tasks at your school, and information about your school’s website.

1. Do you work with IPEDS data for your school?

1a. Do you work with compliance issues for participating in federal financial aid programs? What are your primary responsibilities? (Probe on coordinating data to submit to the Department, collecting data from the school, reporting on the data, using the data.)

2. Do you coordinate or submit data on your school to agencies or organizations other than the Department?

2a. If so, what types of organizations or agencies?

3. To the best of your knowledge, what types of GE information are currently available to students on your website? (Probe on description of programs, courses offered, length of program, cost of program, job placement rate)

Thank you. For the remainder of the discussion, we will focus our attention on reviewing the data elements included in the Gainful Employment regulations. As I mentioned earlier, the Gainful Employment regulations issued by the Department of Education in October 2014 require schools to provide specific information to current and prospective students about their certificate programs (and on all programs if you are a for-profit institution). This focus group provides one among several opportunities for the institutions affected by these requirements to provide feedback to the Department about the data elements and student warning language. I will ask you specific questions about each of the items on the screen that relate to the information institutions will collect and provide as well as information that the Department will calculate. On the screen, you’ll see these items are listed in two separate columns.

• [Moderator displays the list of the data elements by category.]

First, let’s talk about the items under Program Information.

4. The first item is “length of the program in calendar time (i.e., weeks, months, years).”

4a. How important is this information to students? [Very important, important, somewhat important, not important.]

5. Now, let’s look at the number of clock or credit hours or equivalent, as applicable, in the program.

5a. How many of you use clock hours? How many use credit hours? How many use something different?

5b. Do you think students understand the difference between clock and credit hours? How important is this information to students?

Now let’s move on to Program Costs, which include tuition and fees, and books, supplies, and equipment.

6. In looking at tuition and fees, the regulations say “the total cost that the student would incur for completing the program within the length of the program.”

6a. Does your institution typically charge tuition to students for the entire program or term-to-term or by academic year?

6b. Do you include books and supplies as part of your tuition? Do you include equipment as part of your supplies or break it out separately?

Now let’s talk about Student Debt.

7. Of students who enrolled in the program during the most recently completed award year, the percentage who received: (a) a Title IV loan for enrollment in the program, (b) a private loan, or (c) either a Title IV or private loan.

7a. Is it clear to you what institutions are being asked to provide here?

7b. How important do you think this information is to students?

7c. Can you estimate the person-hours required to provide this information?

The next set of questions relate to loan repayment rate, median loan debt, and program level cohort default rate—rates that are calculated by the Department.

8. Loan repayment rate for any one or all of the following groups of students who entered repayment on Title IV loans for: (a) all students who enrolled in the program, (b) students who completed the program, and/or (c) students who withdrew from the program.

8a. Which group do you think students would find most helpful: (a) loan repayment rate for all enrolled students, (b) repayment rate for program completers, or (c) the repayment rate for students who withdrew from the program? What makes you say that?

8b. How do you think students would use this information?

9. Median loan debt calculated for any one or all of the following groups: (a) students who completed the program during the most recently completed award year, (b) students who withdrew from the program during the most recently completed award year, and/or (c) students who completed and withdrew from the program during the most recently completed award.

9a. Do you think students would prefer to see median loan debt rate for students who: (a) complete, (b) withdraw, or (c) both? What makes you say this?

9b. Do you think students understand what median is?

9c. How important do you think this information is to students?

10. Now let’s briefly discuss the program level cohort default rate, which is also calculated by ED.

10a. Do you think students understand what this rate means? Compared to the median loan debt and the loan repayment rate, how important do you think the program level cohort default rate is for students?

The next set of questions on program success are also calculated by the Department.

There are several measures of completion and withdrawal rates ranging for 100% of the program length for full time and part time students to 300% of program length for full-time and part-time students. Let’s talk about completion rates first.

11. Do you think some completion rates are more critical than others to report for full time students completing GE programs (i.e., 100%, 150%, 200%, and 300%)?

11a. What makes you say this?

11b. If you could put yourself in the shoes of a student, which one or two of the rates would be most important to you? Why?

11c. Do you think students think some completion rates are more important than others to report for part-time students? Which ones? Why?

12. Regarding withdrawal rates, do you think both rates (i.e., 100% and 150%) are equally important to report for full-time students? Why?

12a. Do you think both are equally important to report for part-time students? Why?

12b. From the student’s perspective, do you think reporting withdrawal rates within 100% or 150% of program length for full time students is easier to understand? What about for part-time students?

12b. Do you think students would have a better understanding of completion rates, withdrawal rates, or both rates? What makes you say that?

Now let’s move to the last category, Program Employment

13. Placement rate, if the institution is required by its accrediting agency or State to calculate a placement rate either for: (a) the program, or (b) the institution, or (c) both, using the required methodology of that accrediting agency or State.

13a. Do you know how the accrediting agency or State defines job placement rates?

13b. Do you know how your institution tracks job placement of your graduates? (Probe on alumni surveys, state data systems, etc.)

13c. How easy or difficult would you say it would be to collect this information? Can you estimate person-hours required to provide this information?

13d. How important is this information to students?

14. Finally, the Department will calculate students’ mean or median earnings for any one or all of the following students groups: (a) those who completed the program during the cohort period used by the Department to calculate the most recent debt to earnings rates for the program, (b) those who were in withdrawn status at the end of the cohort period (c) all who completed the program or were in withdrawn status.

14a. Do you think students have a better understanding of mean or median earnings? What makes you say that?

14b. Which one of the mean or median earnings rates would be easier for students to understand (i.e., rates for completers, those in a withdrawn status, or both)? Please say more about your response.

15. Please comment on the overall institutional burden associated with collecting new items to comply with these GE regulations?

15a. Probe on coordinating with other offices.

Ranking Exercises #1

[Hand out Ranking Exercise #1—ranking sub-elements]: Now, you will complete several ranking exercises based on the items we just discussed. In exercise #1, you will check these “sub-elements” items based on how important or useful you think they are to students. Using the sheet I just passed out please check one sub-element for items 1-2 and 5- 7, and check TWO sub-elements for items 3 and 4, per the instructions. Any questions? I’ll give you a few minutes to complete this. When you’re finished, please turn the exercise over.

Ranking Exercises #2

[Hand out Ranking Exercise #2—ranking within main data elements]: Now, I’m passing out ranking sheet #2. This time, you will rank the main data elements within each GROUP (there are 3 groups) of elements using numbers, with 1 being the most important. Although there are different options (sub-elements) below many of the main data elements, please consider the option(s) you selected in exercise #1 when ranking the “main” element. Any questions?

Let’s spend the final minutes to briefly discuss the student warning language before doing our final ranking exercise. [This information will appear on a slide or we will provide a handout so participants can read along.]

Under the regulations, if a GE program could become ineligible based on its final debt-to-earnings rates calculated for the next award year, the institution must provide warnings to both current and prospective students. )

According to the regulations, the language must state that: [DISPLAY THIS ON A SLIDE]

‘‘This program has not passed standards established by the U.S. Department of Education. The Department based these standards on the amounts students borrow for enrollment in this program and their reported earnings. If in the future the program does not pass the standards, students who are then enrolled may not be able to use federal student grants or loans to pay for the program, and may have to find other ways, such as private loans, to pay for the program.”

16. What is your initial reaction to this language?

16a. What, if any, questions do you have about it?

16b. If you had the option, would you change any of the language? What? How?

16c. How do you think students would react to this language?

Ranking Exercises #3

[Hand out Ranking Exercise #3—Overall ranking of main data elements]:

In this final exercise, please rank the relative importance of each of the 10 data elements with you being the most important. Please rank all 10 of the overall key elements. Please note that we have excluded some data elements (e.g., program length, tuition and fees, enrollment, etc.) because they are already being collected or they will be collected. Any questions?

If time, ask:

17. Are there any other suggestions or topics we have not discussed about the items on the form that you would like to bring up now?

Thank you. This has been a very useful discussion. We appreciate your time and assistance. Excuse me while I go back and get your checks. [As time allows, check to see if anyone has any final questions.] …..

Westat Moderator’s Guide, Institutional Reps Gainful Employment Page 1