Volume I BPS 2012-2017 Student Records Qualitative Evaluation

National Center for Education Statistics

Volume I

Supporting Statement

Beginning Postsecondary Students Longitudinal Study (BPS:12) Student Records Qualitative Evaluation

OMB# 1850-0803 v.152

Attachments:

Attachment I – Recruitment and Eligibility Screening Materials

Attachment II – Respondent Contacting Materials

Attachment III – Consent to Participate in Research

Attachment IV – Interview Protocol

Attachment V – Student Records Instrument Facsimile

March 11, 2016

Submittal-Related Information

The following material is being submitted under the National Center for Education Statistics (NCES) generic clearance agreement (OMB# 1850-0803), which provides NCES the capability to improve data collection instruments by conducting testing, such as pilot tests, focus groups, and cognitive interviews, to improve methodologies, survey questions, and/or delivery methods.

This request is to conduct qualitative testing in April through June of 2016 in preparation for the Beginning Postsecondary Students Longitudinal Study (BPS:12) Student Records data collection from institutions attended by the 2011-12 BPS cohort during the past six years. The main data collection for BPS:12 Student Records will begin in February 2017. Under contract to the U.S. Department of Education (Contract # ED- IES-09-C-0039), RTI International (RTI) will collect students records data on the BPS:12 cohort covering the period of 2012 through 2017 on behalf of NCES.

This document describes recruiting, screening, and qualitative evaluation procedures for the planned BPS student records instrument, which is adapted for data collection for multiple academic years from the 2015-16 National Postsecondary Student Aid Study (NPSAS:16) full-scale student records instrument that is currently in use through the Postsecondary Data Portal. Usability of the instrument and its multi-year functionality will be evaluated through one-on-one interview sessions with postsecondary institution staff. Testing results will be used to refine the instrument for the BPS Student Record data collection, which is scheduled to begin in February 2017 and which will be conducted concurrently with the HSLS:09 student records collection. As part of NCES’ initiative to align the data elements requested from institutions across studies, the BPS and HSLS:09 student records collection will use the same student records instrument. The instrument facsimile included in this submission (attachment V) has already been approved for HSLS:09 (OMB #1850-0852 v.17).

Study Background

The BPS longitudinal study is designed to follow a cohort of students who enroll in postsecondary education for the first time during a specific academic year, irrespective of their date of high school completion. BPS collects data on students’ persistence in, and completion of, postsecondary education programs; their transition to employment; demographic characteristics; and changes over time in their goals, marital status, income, and debt, among other indicators. Data from BPS are used to help researchers and policymakers better understand how financial aid influences persistence and completion, what percentages of students complete various degree programs, what employment and wage outcomes are for those who have attained certificates and/or degrees, and why students leave school.

The BPS:12 cohort was identified as part of the 2011–12 National Postsecondary Student Aid Study (NPSAS:12), which was based on a sample of students attending postsecondary education in the 2011–12 academic year. The BPS cohort was interviewed in 2012 and 2014 and will be interviewed again in 2017. The Student Records component of BPS will provide institution data on students’ enrollment histories, costs of attendance, and financial aid for all the institutions attended over a six year period. The collection of student records conforms fully to the requirements of the Family Educational Rights and Privacy Act of 1974 (FERPA) (34 CFR Part 99). Under FERPA’s general consent rule, NCES is authorized to obtain student level data from institutions for any study-eligible student, without prior consent, if the disclosure is to authorized representatives of the Secretary of Education [34 CFR § 99.31(a)(3)]. Student data are subject to strict protections that are adhered to by NCES and its contractor organizations. FERPA regulation can be found at http://www2.ed.gov/policy/gen/reg/ferpa.

Design and Context

The proposed BPS Student Records data elements have been previously tested as part of Education Longitudinal Study of 2002 Financial Aid Feasibility Study (ELS:2002 FAFS), NPSAS:16, and two rounds of focus group testing conducted as part of NPSAS in 2014 and 2015. Therefore, the effort for which clearance is requested will instead evaluate the instrument as adapted for collecting multi-year data, with a particular focus on identifying ways to reduce burden for participating institutions. RTI will recruit participants, carry out the evaluation sessions, and summarize the results.

Participants will be recruited from institutions that completed the NPSAS:16 full-scale student records collection and are located in the greater Research Triangle Park, NC area. To the extent possible, participants will be selected such that a range of institution sizes and sectors are represented. If a sufficient number of NPSAS:16 full-scale participants cannot be recruited from this area, the pool will be expanded to include other area NPSAS-eligible institutions (such as NPSAS:16 field test participants, or institutions that participated in prior NPSAS student records focus groups) or institutions from the greater Chicago, IL and Washington, DC areas.

Selected participants will be those types of staff who would be responsible for collecting and providing the requested data for the BPS:12 Student Records collection. Sampled individuals will be contacted by telephone and screened to ensure they are responsible for completing or overseeing completion of the student records collection. Once verified, they will be invited to participate in a one-on-one, in-person interview session, which will take place at the institution. Attachment I presents the screening questions that will be used to determine eligibility for participating in the sessions. Attachment II, submitted with this memorandum, presents the materials that will be used for contacting institution staff. Attachment III contains the Assurance of Confidentiality and consent to participate, which all participants will be asked to sign before the session begins.

Members of the RTI project team will visit each recruited institution, observe the participant as he or she navigates through the instrument for test cases, and solicit feedback on the instrument based on a pre-determined list of discussion topics. Participants will be asked for their feedback concerning the availability and accessibility of the requested data for multiple academic years and the anticipated burden required to complete the request. The session protocol guide, including the list of discussion topics, is presented in Attachment IV. The instrument facsimile is provided in Attachment V. Each session will last for a maximum of 90 minutes. To encourage their participation and to thank participants for their time and help, participants will be offered a $25 Amazon gift card.

Each interview will be audio recorded to facilliatate review of interviews by NCES and RTI project staff. Immediately following the conclusion of each session, methodologists will review the session recordings and notes, highlighting potential themes that may have arisen, and archive the digital audio recording for analysis. The observations will organize and summarize the common themes, insights, and ideas emerging from each of the sessions into a report.

Assurance of Confidentiality

Interview participants will be sent a consent form (attachment II) and will be asked to sign, scan, and return the form prior to the interview session. The consent form will include the following assurance of confidentiality:

“RTI International is carrying out this research for the National Center for Education Statistics (NCES) of the U.S. Department of Education. NCES is authorized to conduct this study under the Education Sciences Reform Act (20 U.S.C., § 9543). Your participation is voluntary. Your responses are protected from disclosure by federal statute (20 U.S.C., § 9573). All responses that relate to or describe identifiable characteristics of individuals may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law.”

Schedule for BPS Qualitative Evaluation Activities

Recruitment will begin in March 2016. Interviews will take place from April through June of 2016.

Estimates of Respondent Burden

Up to 75 institutions will be contacted, with the goal of recuiting 10-12 participants total. Each session is expected to last up to 90 minutes.

Total burden is provided in table 3.

Table 3. Estimated Cognitive/Usability Interview Respondent Burden

	Number of Respondents	Number of Responses	Minutes per respondent	Total Burden (Hours)
Recruitment	75	75	5	7
Interview	12	12	90	18
Total	75	87	-	25

Estimate of Costs for Recruiting and Paying Respondents

To maximize the likelihood of participation by institution staff, sessions will be held during the work day. However, taking time in an individual’s work day may necessitate that the time away from work be made up either by an extended work day or on a weekend. Given this potential inconvenience and to encourage particpation, we will offer a $25 gift certificate from Amazon.com. In addition, some staff may not be able to personally accept this token and have to turn it over to the institution. In such cases, this incentive amount will serve to express gratitude to the institution for any inconvenience they experience in allowing their employee to participate in the 90-minute interview session.

Estimate of Cost Burden

There are no direct costs to respondents.

Cost to Federal Government

The cost to the federal government for conducting these interviews will be $21,000. This cost includes recruitment, conducting interviews, analyses, report writing, and participant incentives.