Justification Memo - updated

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: April 3, 2014

TO: Shelly Martinez, OMB

THROUGH: Kashka Kubzdela, NCES

FROM: Tracy Hunt-White, NCES

SUBJECT: Focus Groups for NPSAS:16 (OMB No. 1850-0803 v.98)

Submittal-Related Information

The following material is being submitted under the generic clearance agreement (OMB No. 1850-0803) that provides for the National Center for Education Statistics (NCES) to improve survey and assessment instruments by conducting pilot tests, focus groups, and cognitive interviews that lead to improved methodologies, question types, and/or delivery methods. The request for approval described in this memorandum is to conduct focus groups with postsecondary institution staff. The results will guide development of revised data elements and data abstraction instrument for use with student records collections in the 2015-16 National Postsecondary Student Aid Study (NPSAS:16).

Background

Housed in NCES’s Longitudinal Studies Branch of the Sample Surveys Division, NPSAS is a comprehensive study of how students and their families pay for postsecondary education. Key topics addressed by NPSAS include grants and loans from federal, state, institutional, and private sources; need-based and merit aid; work-study and employment while enrolled; tuition and fees; total, net, and out-of-pocket net price of attendance; financial need before and after aid award; cumulative borrowing; and online education.

NPSAS data are collected from multiple sources. Student records at the institutions provide information on student enrollment, budgets, and sources of aid, primarily aid from other than federal sources. Administrative record matching, such as to the Central Processing System and National Student Loan Data Systems, provide demographic data on federal aid applicants and federal aid data awarded and dispersed for aid recipients. Student interviews collect information on enrollment-related topics not otherwise covered, such as parent income for students who did not apply for federal aid, other sources of support, and employment outside of the institution.

NCES, working with its contractor, RTI International, is pursuing a redesign of the student records instrument and data collection procedures for the NPSAS study to allow more frequent collection of institution record data while minimizing the potential burden that such a collection could impose on institutions. In addition to the NPSAS collection, the redesigned methodology is also intended to facilitate longitudinal data collection efforts, and possibly a substantial increase in the number of records abstracted to allow state-representative samples and more precise estimates for subgroups.

We propose to conduct a multistage qualitative evaluation beginning with the focus groups described in this document. The focus groups will bring together various institution staff, a subset of whom will be experienced with the NPSAS data abstractions, for discussion of terminology, formats, data availability, and abstraction methods. The focus groups will also help us develop a standardized set of data elements and definitions to be used in the NPSAS and other NCES postsecondary student records collections, particularly elements which vary across institutions and items used to determine eligibility for longitudinal follow-up studies.

Design and Context

The current request is for approval to conduct several focus groups during May through June 2^nd, 2014, and recruitment starting upon OMB clearance. The focus groups will be conducted by RTI and Shugoll Research and monitored by NCES. This submission includes focus group protocols, covering both open-ended questions and targeted probes, which will explore participants’ opinions, decisions, and understanding of the terminology.

The focus groups will be conducted with staff from postsecondary institutions who were selected by the chief administrator at their institution to provide data for the NPSAS:12 student records collection. These staff members were the actual institution contacts who had provided data for NPSAS:12. A random selection of 400 institutions will be drawn for possible recruitment. The sample of 400 will then be divided into four groups based on institution sector:

Group 1: Public less-than-2-year; Public 2-year institutions

Group 2: Public 4-year non-doctorate-granting; Public 4-year doctorate-granting

Group 3: Private nonprofit less-than-4-year; Private nonprofit 4-year non-doctorate-granting; Private nonprofit 4-year doctorate-granting

Group 4: Private for-profit less-than-2-year; Private for-profit 2-year; Private for-profit 4-year

Within each group, contacting and recruitment will occur until a group of 10 participants is established. It is important to separate the focus groups by type of institution because each of these segments will have different capabilities, resources, and issues related to student records collections. Relatively homogeneous groups with similar issues and needs help ensure that the discussions can be productive.

Focus groups will be conducted remotely by Shugoll Research of Bethesda, MD, using online video/virtual technology so that respondents from institutions across the country can participate. Respondents will participate in the focus groups from their own computers. The virtual remote platform also allows for NCES and other project staff to observe the focus groups via online video streaming.

Respondent Recruitment

Respondents are expected to be administrative staff in the financial aid office or institutional research office at sampled colleges, universities, and trade schools around the country. Respondents will be individuals who have collected data from their institution’s records to complete the NPSAS:12 record abstraction requests. Sampled individuals will be contacted by telephone, and screened to ensure they are responsible for completing or overseeing completion of the student records collection. Once the individuals have verified their responsibility for completing student records in NPSAS:12, they will be invited to participate in one of the focus groups. Respondents will need to have access to a computer with high speed internet access at the location where they will be participating in the focus group. Their computer must have a webcam, or if not, we will send them one to be used for the focus group.

Experienced and trained professional recruiters employed by the contractor will recruit respondents for this study. Recruiters will collect the e-mail addresses of those who agree to participate in the research and a confirmation e-mail will be sent to respondents with information about the study and the date and time of their focus group. A link to the online video interface for the focus group and log-in information will be provided in another e-mail the day before the focus group session when they will be re-confirmed for participation. Right before the focus group begins, all respondents will be re-screened and a tech check will be performed to ensure that they can log into the focus group interface and that all equipment, including the webcam, is functioning properly.

The recruitment screener/invitation is attached as Appendix B. For each focus group, 10 respondents will be recruited to ensure that 6 to 10 participate.

Research Implementation

Respondents will participate in the focus groups virtually/remotely from their own computers. The focus groups will use webcam technology, streaming video, and an audio connection to provide real-time, face-to-face interaction between the moderator and respondents via a shared desktop. The technology will enable participants to view the terms under discussion, visuals of forms to be completed, examples of data entry modes, and proposed definitions. The faces of all respondents appear on the sides of the computer screen so that the moderator and client observers can see them during the entire session and facial expressions can be observed. Each focus group will last about 1-1/2 to 2 hours and will be held at times that work for at least eight of the group’s members.

Each focus group session will be led by a Shugoll Research senior staff person with extensive experience moderating focus groups with administrators and other staff in educational institutions. The moderator will be responsible for keeping the discussion focused and making sure that all respondents take part in the discussion. Each session will be audio and video recorded. The contractor will provide technology checks with all respondents prior to the session and technology support will be available during each focus group. The audio recordings of each focus group will be made available to NCES for review. The moderator’s topic guide/protocol for the focus groups is attached as Appendix C.

At the conclusion of the focus groups, the audio recordings will be transcribed and results analyzed by the moderator. The moderator will summarize the common themes, insights, and ideas emerging from the focus group discussions in a report that will be submitted to NCES through RTI. The report will be prepared in PowerPoint format and will include respondent verbatim quotations to illustrate key findings, and conclusions and recommendations.

Assurance of Confidentiality

Focus group participants will be informed that their participation is voluntary and confidential (see Appendix A). No personally identifiable information from respondents will be released. Respondents will be assigned a unique identifier (ID), which will be created solely for data file management and used to keep all focus group materials together. The ID will not be linked to the respondent’s name in any way. The signed consent forms will be kept separately from the interview files for the duration of the study and will be destroyed after the final report is released.

Estimate of Hour Burden

Despite our positive working relationships with institution staff, the timing of the focus groups – during final exams, the end of the spring term, and graduation – may make it difficult for institutions to participate. Further, some staff may not be able to participate simply because they are unavailable at the time the focus group is scheduled to meet. Consequently, from among the institutions participating in NPSAS:12, we plan to select 100 potential institutions for each focus group from which the 10 participants per group will be recruited.

Each of the four focus groups of 6 to 10 respondents (not to exceed an overall total of 40 participants) is expected to last approximately 1.5 to 2 hours. Total burden is provided in table 1.

Table 1. Estimated respondent burden

Group	Maximum number of contacts for recruitment*	Hours per recruitment	Number participating	Hours per respondent	Total Maximum Burden Hours
1	100	0.07	10	1.5 to 2.0	27
2	100	0.07	10	1.5 to 2.0	27
3	100	0.07	10	1.5 to 2.0	27
4	100	0.07	10	1.5 to 2.0	27
Total	400		40		108

* Contacting will occur within each group of 100 until a group of ten is formed; for a total of 440 responses.

Estimate of Costs for Recruiting and Paying Respondents

To maximize the likelihood of participation, focus groups will be held during the work day. However, taking time in an individual’s work day may necessitate that the time away from work be made up either by an extended day or on a weekend; especially given that the focus groups will take place at the end of the academic year. Given that potential inconvenience, we propose offering an emailed $25 gift certificate from Amazon.com as a token of appreciation for their participation in the focus group. In addition, many staff may not be able to personally accept this incentive and have to turn it over to the institution. This incentive amount may be needed to reimburse the institution for any inconvenience they experience for allowing their employee to participate in the focus group.

Estimate of Cost Burden

There are no direct costs to participants.

Cost to Federal Government

The cost of conducting the focus groups will be $83,858. Table 2 provides the overall cost estimates:

Table 2. Estimated costs to conduct focus groups with institution staff

	Budget
Direct labor	$41,512
Total labor	$41,512
Other direct costs
Computer expenses and materials	$2,456
Respondent recruiting, audio recording, videostreaming	$39,890
Total ODCs	$42,346
Total costs	$83,858

Appendix A

S ample Consent Form

Consent to Participate in Research

Title of Research: National Postsecondary Student Aid Study – Focus Groups

Introduction and Purpose

You, along with participants from other institutions, are being asked to participate in a focus group being conducted by RTI International and Shugoll Research for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. The purpose of the focus group is to bring together various institution staff experienced with the NPSAS student record collection for discussion of terminology, formats, data availability, and collection methods. The results of focus group discussions will be used to simplify study terminology and improve student records collection instruments for future NPSAS and other NCES studies.

Procedures

You are one of approximately 8 individuals in this focus group who will be asked to discuss factors related to providing data for NPSAS and other NCES student records collections. There will be three additional focus groups of approximately 8 participants each from other types of institutions that will be asked similar questions.

The focus group will be audio recorded to make sure we don’t miss anything that you say and to help us write a report summarizing the results of the group discussions. Upon completion of the written report, the recording will be destroyed. Your name will never be used in the report that we write. Although we will ask participants not to reveal anything about others in the group, we cannot guarantee this.

Study Duration

Your participation in the focus group will take about 1½ to 2 hours.

Possible Risks or Discomforts

We do not anticipate that any of the discussion topics will make you uncomfortable or upset. However you may refuse to answer any question or take a break at any time.

Benefits

Your Benefits You will receive a $25 Amazon.com gift certificate, by email, as a token of thanks for your participation in this study.

Benefits for Other People We hope that this focus group will help us develop and improve student records collections for future NCES studies that will aid in understanding students’ college experiences and how they pay for college or trade school.

Confidentiality

Shugoll Research and RTI International are conducting this focus group for the National Center for Education Statistics (NCES) of the U.S. Department of Education. This study is authorized by law under the Education Sciences Reform Act (20 U.S.C., § 9543). Your participation is voluntary. Your responses are protected from disclosure by federal statute (20 U.S.C., § 9573). All responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose, unless otherwise compelled by law.

Future Contacts

We will not contact you in the future as part of this focus group study.

Your Rights

Your decision to take part in this research study is completely voluntary. You can refuse any part of the study and you can stop participating at any time.

Your Questions

If you have any questions about the study, you may call Merrill Shugoll at Shugoll Research (301-656-0310) or Jennifer Wine at RTI International (919-541-6870). If you have any questions about your rights as a study participant, you may call RTI’s Office of Research Protection at 1-866-214-2043 (a toll-free number).

YOU WILL BE GIVEN A COPY OF THIS CONSENT FORM TO KEEP.

Your signature below indicates that you have read the information provided above, have received answers to your questions, and have freely decided to participate in this research. By agreeing to participate in this research, you are not giving up any of your legal rights.

________________ Date _____________________________________ _ Signature of Participant

____________________________________ __

Printed Name of Participant

I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above-named individual.

______________ Date _____________________________________ _ Signature of Person Obtaining Consent

____________________________________ __

Printed Name of Person Obtaining Consent

7