PartB NPSAS24 FT List Collection

2023–24 NATIONAL POSTSECONDARY STUDENT AID STUDY (NPSAS:24) Field Test

Institution Contacting and List Collection

Supporting Statement Part B

OMB # 1850-0666 v.33

Submitted by

National Center for Education Statistics

U.S. Department of Education

April 2022

Contents

B. Collection of Information Employing Statistical Methods 3

1. Respondent Universe 3

a. Institution Universe 3

b. Student Universe 3

2. Statistical Methodology 4

a. Institution Sample 4

b. Student Sample 6

3. Methods for Maximizing Response Rates 10

a. NPSAS:20 Institutional Contacting 10

b. Matching to Administrative Databases 12

c. Postsecondary Data Portal (PDP) 12

4. Tests of Procedures or Methods 12

5. Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study 13

Tables and Figures

Table 1. Preliminary number of field test institutions in the population and to be sampled, by control and level of instituion 5

Table 2. Preliminary number of full-scale institutions in the population and to be sampled, by control and level of institution 6

Table 3. NPSAS:24 field test preliminary student population counts and smaple sizes, by control and level of instituion and student type 8

Table 4. NPSAS:24 full-scale preliminary student sample sizes, by control and level of institution and student type 9

Figure 1. Institution contacting 13

2. Collection of Information Employing Statistical Methods

This submission requests clearance for the 2023–24 National Postsecondary Student Aid Study (NPSAS:24) field test institution contacting, enrollment list collection, list sampling, and administrative matching activities. The National Center for Education Statistics (NCES) will submit a separate request for the student data collection, including student record data abstraction and student interviews, in the summer of 2022.

1. Respondent Universe

   1. Institution Universe

NPSAS:24 will be nationally representative for both undergraduate and graduate students. NPSAS:24 may also be state-representative for undergraduate students overall, and in public 2-year and 4-year institutions.¹ NPSAS:24 will use a two-stage sampling design. The first stage involves the selection of institutions. In the second stage, students are selected from within sampled institutions. Also, the NPSAS:24 sample is designed to serve as the base year for a potential 2024 cohort of the Baccalaureate and Beyond (B&B) Longitudinal Study and will include a nationally representative sample of baccalaureate recipients. This allows NCES to decide later if there will be a B&B study. To construct the full-scale institution sampling frame for NPSAS:24, we will use institution data collected from various surveys of the Integrated Postsecondary Education Data System (IPEDS). The student sampling frame includes all students who meet eligibility requirements from the participating institutions.

The NPSAS:24 institution (first stage) sampling frame includes all levels (less-than-2-year, 2-year, and 4-year) and control classifications (public, private nonprofit, and private for-profit) of Title IV eligible postsecondary institutions in the 50 states, the District of Columbia, and Puerto Rico. To be eligible for NPSAS:24, an institution must do the following during the 2022–23 academic year for the field test and the 2023-24 academic year for the full-scale:

• offer an educational program designed for persons who have completed secondary education;

• offer at least one academic, occupational, or vocational program of study lasting at least 3 months or 300 clock hours;

• offer courses that are open to more than the employees or members of the company or group (e.g., union) that administer the institution;

• be located in at least one of the 50 states, the District of Columbia, or Puerto Rico;

• be other than a U.S. service academy;² and

• have a signed Title IV participation agreement with the U.S. Department of Education.³

As indicated above, institutions providing only avocational, recreational, or remedial courses or only in-house courses for their own employees will be excluded.

2. Student Universe

The student (second stage) sampling frame is described below. NPSAS-eligible undergraduate and graduate students are those who were enrolled in the NPSAS institution in any term or course of instruction between July 1, 2022 and April 30, 2023 for the field test and between July 1, 2023 and April 30, 2024⁴ for the full-scale and who are:

• enrolled in either (1) an academic program; (2) at least one course for credit that could be applied toward fulfilling the requirements for an academic degree; (3) exclusively noncredit remedial coursework that has been determined by their institution to be eligible for Title IV aid; or (4) an occupational or vocational program that requires at least 3 months or 300 clock hours of instruction to receive a degree, certificate, or other formal award; and

• not concurrently enrolled in high school; and

• not enrolled solely in a General Educational Development (GED^®)⁵

• or other high school completion program.

2. Statistical Methodology

   1. Institution Sample

The NPSAS:24 field test and full-scale institution frame will be constructed from the IPEDS 2021-22 Institutional Characteristics (IC) header, 2021-22 IC, 2020-21 Completions, and 2020-21 Full-year Enrollment files.⁶ The field test and full-scale institution samples will both be selected during the field test, as described below. Then, prior to the start of full-scale data collection, the sample will be freshened using the IPEDS:2022-23 header, 2022-23 IC, 2021-22 Completions, and 2021-22 Full-year Enrollment files. This will ensure that all potentially eligible institutions have a probability of selection. As part of the freshening process, we will use the current IPEDS files to identify full-scale institutions that are no longer eligible for NPSAS:24 due to closure. It is possible that some for-profit institutions and large chains of for-profit institutions may have been closed or sold after the latest IPEDS data collection. Therefore, we will also conduct web searches for articles about closed institutions, to identify these closed for-profit institutions. For the small number of institutions on the frame that have missing enrollment information because they are not imputed as part of IPEDS, we will impute the enrollment.

The institution strata for both the field test and full-scale studies will be the eleven sectors that have been used since the NPSAS:16 full-scale study, which are based on institution level, control, and highest level of offering:

• Public less-than-2-year;

• Public 2-year;

• Public 4-year, non-doctorate-granting, primarily sub-baccalaureate;

• Public 4-year, non-doctorate-granting, primarily baccalaureate;

• Public 4-year, doctorate-granting;

• Private nonprofit, 2-year or less;

• Private nonprofit, 4-year, non-doctorate-granting;

• Private nonprofit, 4-year, doctorate-granting;

• Private for-profit, less-than-2-year;

• Private for-profit, 2-year; and

• Private for-profit, 4-year.

We will select the full-scale sample of 2,000 institutions first using a variation of probability proportional to size (PPS) sampling called sequential probability minimum replacement (PMR) sampling.⁷ This method selects institutions sequentially with probability proportional to size and with minimum replacement. Selection with minimum replacement means that the actual number of hits for an institution can equal the integer part of the expected number of hits for that institution, or the next largest integer, that is, institutions have a chance of being selected more than once.⁸ Instead of the PMR sampling algorithm selecting some institutions multiple times, prior to the PMR sample selection, we will set aside for inclusion with certainty in the sample all institutions with a probability of being selected more than once, that is, adjusting their probability of selection to be one. Then, the probabilities of selection for other institutions are adjusted accordingly prior to PMR selection, so that the total institution sample size target is met. A composite size measure⁹ will be used to help achieve self-weighting samples¹⁰ for student-by-institution strata and to allow flexibility to change sampling rates in selected strata without losing the self-weighting attribute of the sampling method. Institution composite measures of size will be determined using undergraduate and graduate student enrollment counts and baccalaureate recipient counts from the most recent IPEDS 12-Month Enrollment and Completion files, respectively.

The field test sample of 300 institutions will then be selected purposively from the pool of institutions that were not selected for the full-scale study. A purposive sample will allow us to identify institutions that will be less burdened and more agreeable to participate in the field test, based on their past participation and our relationships with them. This will help us receive 150 student enrollment lists by early February,¹¹ which will allow student data collection to start for all sample students by early March. This time schedule gives sufficient time for a field test experiment.¹²

Prior to the start of full-scale data collection, the sample will be freshened using the most recent IPEDS files. This will ensure that all potentially eligible institutions have a probability of selection. The freshened sample size will be determined based on the number and size of new institutions such that the probabilities of selection will be similar to the originally sampled institutions in the same strata. However, this sample size is expected to be small.

The estimated institution sample sizes by the eleven institution strata (described above) for the field test and full-scale are presented in tables 1 and 2, respectively. For the field test, we will select student samples from the first 150 enrolment lists received, or more if needed to have lists from all strata. For the full-scale, we expect to obtain overall 97 percent student eligibility rate and at least an overall 80 percent institution participation (response) rate, which will yield approximately 1,550 institutions providing lists.

Within each institution stratum, additional implicit stratification will be accomplished by sorting the sampling frame by the following classifications, as appropriate:

1. Level of institution;

2. Historically Black Colleges and Universities (HBCUs) indicator;

3. Hispanic-serving institutions (HSIs) indicator;¹³

4. Carnegie classifications of postsecondary institutions;¹⁴ and

5. The institution measure of size.

The objective of this implicit stratification is to approximate proportional representation of institutions on these measures.

Table 1. Preliminary number of field test institutions in the population and to be sampled, by control and level of institution
Control and level of institution	Population estimate	Sample size
Total	6,041	300
Public less-than-2-year	225	6
Public 2-year	919	12
Public 4-year, non-doctorate-granting, primarily sub-baccalaureate	172	41
Public 4-year, non-doctorate-granting, primarily baccalaureate	204	49
Public 4-year, doctorate-granting1	386	0
Private nonprofit 2-year or less	197	7
Private nonprofit 4-year, non-doctorate-granting	892	137
Private nonprofit 4-year, doctorate-granting	731	23
Private for-profit less-than-2-year	1,405	9
Private for-profit 2-year	565	6
Private for-profit 4-year	345	10

¹ There are no public 4-year doctorate-granting institutions in the field test sample to allow the possibility of a census of these institutions to be included in the full-scale sample, which was done in NPSAS:16. Although table 2 doesn’t show a census, we will make a final determination of the full-scale sample sizes after the field test.

SOURCE: Population estimates based on IPEDS 2020-21 data.

Table 2. Preliminary number of full-scale institutions in the population and to be sampled, by control and level of institution
Control and level of institution	Population estimate	Sample size
Total	6,041	2,000
Public less-than-2-year	225	22
Public 2-year	919	377
Public 4-year, non-doctorate-granting, primarily sub-baccalaureate	172	82
Public 4-year, non-doctorate-granting, primarily baccalaureate	204	98
Public 4-year, doctorate-granting	386	338
Private nonprofit 2-year or less	197	20
Private nonprofit 4-year, non-doctorate-granting	892	325
Private nonprofit 4-year, doctorate-granting	731	268
Private for-profit less-than-2-year	1,405	70
Private for-profit 2-year	565	120
Private for-profit 4-year	345	280

SOURCE: Population estimates based on IPEDS 2020-21 data.

2. Student Sample

Although this submission is not for student data collection, the sample design is included here because part of the design is relevant for list collection, and the sampling of students from the enrollment lists will likely have to begin prior to OMB approval of the student data collection.

Student Enrollment List Collection

To begin NPSAS data collection, sampled institutions are asked to provide a list of all their NPSAS-eligible undergraduate and graduate students enrolled in the targeted academic year, covering July 1 through June 30 (methods for contacting the sampled institutions are described below in section B.3, and student list data elements are described in appendix D). Since NPSAS:04, institutions have been asked to limit listed students to only those enrolled through April 30, and starting in NPSAS:20, continuous enrollment institutions were asked to limit listed students to only those enrolled through March 31.¹⁵ These truncated enrollment periods exclude students who first enrolled in May or June (and additionally in April for continuous enrollment institutions), but they allow lists to be collected earlier and, in turn, data collection to be completed in less than 12 months. Any lack of coverage resulting from the truncated enrollment period will be accounted for by the poststratification weight adjustment.

For the field test, to expedite list receipt, as mentioned above, we will ask continuous enrollment institutions to provide enrollment lists including students enrolled July 1, 2022 through December 31, 2022. We expect that students enrolled in these institutions in this time period will be sufficient in their responsiveness and responses to the survey to test the full-scale data collection.

We will request that high school students are included on the enrollment list even though these students are not eligible for NPSAS. While these students will be excluded from sampling, high school student counts are needed later for the weighting poststratification adjustment. We will poststratify the NPSAS students to IPEDS enrollment counts (used as control totals), which include high school students. As dual enrollment becomes more prominent, it is important that we adjust the IPEDS counts downward to account for dual enrollment. Since dual enrollment counts are not currently readily available, using high school student counts from the enrollment lists may be the best source for adjusting IPEDS counts and ensuring accurate control totals.

Student Stratification

The field test student sampling strata will be:

1. potential baccalaureate recipients;

2. other undergraduate students;

3. master’s students;

4. doctoral student; and

5. other graduate students.

For the field test, potential baccalaureate recipients will be oversampled and students in the other strata will be undersampled to have enough baccalaureate recipients if there are any B&B follow-up studies. For the full-scale, final determination of oversampled and undersampled groups will be made in preparation for the full-scale, and then final full-scale strata will be determined.

For the full-scale, we will match the student enrollment lists to two supplemental databases prior to sampling (pre-sampling matching), as has been done since NPSAS:16,. To identify veterans, we will match the student enrollment lists with a list of veterans from the Veterans Benefits Administration (VBA) because the veterans identified by institutions on the lists are incomplete. This veterans information will be used with the veteran status from the enrollment lists for full-scale stratification. We will also match the student lists to the National Student Loan Data System (NSLDS) data and use the financial aid data for student-implicit stratification. Within the student-explicit strata, we will sort the students by federally aided/unaided, and this will allow the sample proportions of aided and unaided students to approximately match the population within institution and student strata. Matching to VBA and NSLDS for these purposes is not necessary for the FT sampling, and we will be able to complete the sampling quicker in the field test without this matching.

Identification of baccalaureate recipients

As mentioned above, NPSAS:24 may serve as the base year for a 2024 cohort of B&B and will include a nationally representative sample of baccalaureate recipients, hence the stratification described above. This allows NCES to decide later if they will conduct a B&B study. We will ask all institutions that award baccalaureate degrees to identify baccalaureate recipients. Instead of waiting until June for institutions to positively identify these students and send in lists, we will request a baccalaureate indicator be included on the enrollment lists to flag whether students have completed requirements for or received a bachelor’s degree between July 1 and the date the enrollment list is provided. In NPSAS:16, we additionally requested that institutions provide a second indicator on the lists to flag students who had not yet received their bachelor’s degree but were expected to receive it by June 30. However, many institutions had difficulty providing this second indicator. Therefore, the NPSAS:24 field test will use other data provided on the list to create a proxy to help identify students who have not yet received their bachelor’s degree in the academic year (2022-23 for the field test and 2023-24 for the full-scale) but are expected to receive it. In NPSAS:20, we used a proxy to identify potential first-time beginning students (FTBs)¹⁶ when institutions did not identify them on the lists, and it was based on a combination of variables: date of first enrollment, degree program, no prior aid receipt, age, high school graduation date, and class level. This approach successfully identified first-time college students for about 75 percent of the students sampled using the proxy. Similarly, in NPSAS:24, we plan to create a proxy by identifying a combination of variables, based on which variables from NPSAS:16 best predicted baccalaureate recipients in the B&B:16/17 cohort. Potential variables include degree program, class level, age, date of first enrollment, high school graduation date, and credits accumulated. For this proxy, we will also consider matching students on the enrollment list to the Central Processing System (CPS) and/or NSLDS prior to sampling to obtain prior loan application and receipt data that could be used for the proxy.

We will use and evaluate the proxy in the field test, and then revise it, as necessary, for the full-scale. In the field test, we will also ask institutions to provide the second indicator for students who are expected to complete requirements for or receive a bachelor’s degree between the date the enrollment list is provided and June 30, 2023, similar to what was requested in NPSAS:16. This indicator will be included in the evaluation of the proxy. Details and results of the evaluation will be included in the full-scale OMB package.

Sample Sizes and Student Sampling

NPSAS:24 will be designed to sample a total of 6,000 students in the field test and 162,000 students in the full-scale. We expect to obtain, minimally, 95 percent eligibility rates, a 65 percent student survey response rate, overall, in the field test, due to a shorter data collection period than in the full-scale, and at least a 70 percent response rate, overall, in the full scale. This will yield approximately 3,710 student surveys for the field test and 107,730 student surveys for the full-scale. The graduate student full-scale sample is 25,000, matching the sample size targets in both NPSAS:18-AC and NPSAS:20. Tables 3 and 4 show the population counts and preliminary sample sizes for the field test and full-scale, respectively, by control and level of institution and student type.

As part of setting the NPSAS:24 sample sizes, we need to determine the sample size of potential baccalaureate recipients for the field test and full-scale, who will be part of both NPSAS and the B&B 2024 cohort, if a B&B study is conducted. The B&B:24/25 field test and full-scale samples would include 2,000 and 30,000 baccalaureate recipients, respectively, who respond to the NPSAS:24 survey and confirm that they have received their bachelor’s degree in the appropriate time frame. The NPSAS:24 potential baccalaureate recipient field test and full-scale sample sizes will be approximately 3,970 and 53,040, respectively, assuming a 95 percent eligibility rate, a 65 percent field test survey response rate, a 70 percent full-scale survey response rates, a 19.7 percent false-positive rate, and a 3.0 percent false-negative rate among other undergraduate students, as in NPSAS:16.¹⁷ and this will be updated after the sample optimization.

Table 3. NPSAS:24 field test preliminary student population counts and sample sizes, by control and level of institution and student type
	Potential baccalaureate recipients		Other undergraduate students		Graduate students
Control and level of institution	Population estimate	Sample size	Population estimate	Sample size	Population estimate	Sample size
Total	2,158,975	3,967	20,248,612	1,767	3,944,578	266
Public less-than-2-year	0	0	84,608	124	0	0
Public 2-year	0	0	7,702,136	449	0	0
Public 4-year, non-doctorate-granting, primarily sub-baccalaureate	23,303	586	2,271,222	49	2,204	1
Public 4-year, non-doctorate-granting, primarily baccalaureate	165,238	587	857,881	126	133,295	27
Public 4-year, doctorate-granting1	1,233,254	0	4,891,309	0	1,711,099	0
Private nonprofit 2-year or less	0	0	77,494	150	0	0
Private nonprofit 4-year, non-doctorate-granting	219,555	1,184	1,030,973	176	211,749	41
Private nonprofit 4-year, doctorate-granting	418,557	1,034	1,748,093	155	1,507,444	169
Private for-profit less-than-2-year	0	0	333,812	251	0	0
Private for-profit 2-year	0	0	329,578	61	0	0
Private for-profit 4-year	99,068	576	921,506	226	378,787	28

¹ There are no public 4-year doctorate-granting students in the field test sample to allow the possibility of a census of public 4-year doctorate-granting institutions to be included in the full-scale sample, which was done in NPSAS:16. Although table 2 doesn’t show a census, we will make a final determination of the full-scale sample sizes after the field test.

SOURCE: Population estimates based on IPEDS 2020-21 data.

Table 4. NPSAS:24 full-scale preliminary student sample sizes, by control and level of institution and student type
	Potential baccalaureate recipients		Other undergraduate students		Graduate students
Control and level of institution	Population estimate	Sample size	Population estimate	Sample size	Population estimate	Sample size
Total	2,158,975	53,043	20,248,612	83,957	3,944,578	25,000
Public less-than-2-year	0	0	84,608	848	0	0
Public 2-year	0	0	7,702,136	26,556	0	0
Public 4-year, non-doctorate-granting, primarily sub-baccalaureate	23,303	3,694	2,271,222	3,702	2,204	97
Public 4-year, non-doctorate-granting, primarily baccalaureate	165,238	3,694	857,881	3,470	133,295	2,015
Public 4-year, doctorate-granting	1,233,254	13,745	4,891,309	13,905	1,711,099	6,074
Private nonprofit 2-year or less	0	0	77,494	1,084	0	0
Private nonprofit 4-year, non-doctorate-granting	219,555	7,050	1,030,973	5,333	211,749	3,358
Private nonprofit 4-year, doctorate-granting	418,557	7,852	1,748,093	4,643	1,507,444	5,878
Private for-profit less-than-2-year	0	0	333,812	4,193	0	0
Private for-profit 2-year	0	0	329,578	8,661	0	0
Private for-profit 4-year	99,068	17,008	921,506	11,562	378,787	7,578

SOURCE: Population estimates based on IPEDS 2020-21 data.

Field test and full-scale institution-level student sampling rates will be set based on frame data and adjusted to account for the overestimation of enrollment counts in IPEDS data that has been found in prior NPSAS list collections. Based on these adjusted rates, students will be sampled on a flow basis as student lists are received. Stratified systematic sampling procedures will be used. As mentioned above, in the full-scale, student strata will be sorted by federally aided/unaided students to maintain proportionality between the sample and frame. Sample yield will be monitored by institution and student sampling strata, and the sampling rates will be adjusted early, if necessary, to achieve the desired sample yields.

Quality Control Checks for Lists and Sampling

The number of enrollees on each institution’s student list will be checked against the latest IPEDS 12-month enrollment and completions data. The comparisons will be made for each student level: baccalaureate recipient, undergraduate, and graduate. As has been done in past rounds of NPSAS, only counts within 50 percent of non-imputed IPEDS counts will pass quality-control (QC) checks and will be moved on to student sampling. Institutions that fail QC checks will be recontacted to resolve the discrepancy and to verify that the institution’s campus coordinator who prepared the student list clearly understood our request and provided a list of the appropriate students and data items. If we determine that the initial list provided by the institution was not satisfactory, we will request a replacement list. We will proceed with selecting sample students when we have either confirmed that the list received is correct or have received a corrected list.

All statistical procedures will undergo thorough QC checks. The data collection contractor has a Quality Management Plan (QMP) in place for sampling and all statistical activities. All statisticians will employ a checklist to ensure that all appropriate QC checks are done for student sampling.

Some specific sampling QC checks include, but are not limited to, checking that the

• institutions and students on the sampling frames all have a known, non-zero probability of selection;

• distribution of implicit stratification for institutions is reasonable; and

• number of institutions and students selected match the target sample sizes.

3. Methods for Maximizing Response Rates

1. NPSAS:24 Institution Contacting

Establishing and maintaining contact with sampled institutions throughout the data collection process is vital to the success of NPSAS:24. Institution participation is required in order to collect enrollment lists and draw the student sample. The process by which institutions will be contacted is depicted in figure 1 and described below.

The data collection contractor will be responsible for contacting institutions on behalf of NCES. Each staff member will be assigned a set of institutions that is their responsibility throughout the data collection process. This allows the contractor's staff members to establish rapport with the institution staff and provides a reliable point of contact for the institution. Staff members are thoroughly trained in basic financial aid concepts and in the purposes and requirements of the study, which helps them establish credibility with the institution staff.

The first step in the process is verification of the chief administrator’s contact information. Web searches and verification calls will be conducted to confirm eligibility and confirm contact information obtained from the IPEDS header files before study information is mailed. The Higher Ed Directory (https://hepinc.com/) may also be used to verify information. Once the contact information is verified, we will prepare and send an information packet to the chief administrator of each sampled institution. A copy of the letter and brochure can be found in appendix D. The materials provide information about the purpose of the study and the nature of subsequent requests. Two versions of the chief administrator letter will be used, tailored to the institution’s situation: (1) one letter for institutions for which we identify and recommend a potential campus coordinator from previous NPSAS participation; (2) another letter for institutions for which we do not identify and recommend a potential campus coordinator. For institutions without a recommended coordinator, institution contactors will conduct follow-up calls to the chief administrator to secure study participation and identify a campus coordinator. If the coordinator is not already a Postsecondary Data Portal user, they will be added as a user.

NCES and its contractor will identify relevant multi-campus systems within the sample because these systems can supply enrollment list data at the system level, minimizing burden on individual campuses. Even when it is not possible for a system to supply data from a centralized office, the system can lend support in other ways, such as by prompting institutions under its jurisdiction to participate. NCES and its contractor will undertake additional outreach activities, such as engaging with higher education organizations and networking within the postsecondary community at conferences and professional meetings. These activities are intended to promote the value of NPSAS both to data providers and data users thereby increasing interest and participation in NPSAS:24.

Figure 1. Institution contacting

Once a campus coordinator has been identified for an institution, the contractor will send the coordinator study materials with a request to complete the online Registration Page as the first step. The materials include a letter, the study brochure, and a quick guide to participation in the study (see appendix D). The primary functions of the Registration Page are to confirm the date the institution will be able to provide the student enrollment list and to determine how they will report student records data, by term or by month. Based on the information provided, a customized timeline for collecting the enrollment list will be created for each institution.

After the Registration Page is completed, the campus coordinator will be sent a letter or email requesting an electronic enrollment list of all students enrolled during the academic year. Enrollment lists will be collected from January 2023 to April 2023. As described above, the lists will serve as the frame from which the student samples will be drawn. Follow-up contacts with institutions include telephone prompts, reminder emails and mailers, typically sent prior to a deadline, and touch-base emails typically sent after a period of no outbound contact from study staff (see appendix D). After enrollment lists are received and validated by the contractor for completeness and quality, the campus coordinator will be sent a “thank you” email acknowledging appreciation for their time and effort.

Alternate Enrollment List Submission Method

For a small number of NPSAS:24 field test institutions (approximately 22), NCES and its contractor will attempt to obtain enrollment list data directly from the National Student Clearinghouse (NSC) in addition to collecting their standard NPSAS enrollment lists. By comparing institutions’ NSC data with their regular NPSAS lists, the feasibility of collecting lists from NSC on a wider scale in the full-scale study can be assessed. Permission from the institution will be required before obtaining enrollment list data directly from NSC. A representative from the contractor’s institution data collection leadership team will contact the institutions identified for this initiative directly by phone to discuss the process.

2. Matching to Administrative Databases

Information about NPSAS:24 sampled students will be matched with their data from several administrative databases. The administrative data sources for NPSAS:24 field test will be NSLDS, CPS including FAFSA, NSC, VBA, and student records obtained directly from postsecondary institutions. Further details about these matches are provided in the Supporting Statement Part A (sections A.1, A.2, A.10, and A.11) and in appendix C. During the field test we will also explore matches to other potential data sources to be added to the full-scale collection, such as Supplemental Nutrition Assistance Program (SNAP) recipients’ data from the U.S. Department of Agriculture.

3. Postsecondary Data Portal (PDP)

The NPSAS:24 field test data collection will utilize NCES’ Postsecondary Data Portal (PDP) website. The PDP is used across NCES postsecondary institution data collections. The flexible design of the website allows it to be used for multiple studies that are in data collection at the same time, even when those studies collect different types of data. Currently, there are no plans for other postsecondary data collections to be underway using the PDP during the NPSAS:24 field test.

There are two types of content on the PDP: general-purpose content and study-specific content. General-purpose pages provide overview information about NCES postsecondary studies and use of the website. These pages are identified in appendix D as the “pre-login” pages. Once a user logs in, they see pages with study-specific content. These pages are identified in appendix D as the “after login” content. The NPSAS:24 study-specific content includes FAQs about NPSAS:24, instructions and resources, and pages for providing data (appendix D). Institutions see study-specific PDP content only for the study or studies for which they have been sampled.

The PDP will be updated for the NPSAS:24 field test to add a new option for institutions to submit the student enrollment list using a pre-formatted Excel template file. This new feature will meet the needs of institutions that have requested an Excel template in prior rounds of NPSAS and will facilitate adding real-time error checking to the enrollment list upload process. Enrollment list error checking will benefit participating institutions by providing them with immediate feedback about potential enrollment list data problems, rather than waiting for NPSAS staff to review the enrollment and follow-up with feedback at a later date. Institutions that prefer to create their own enrollment list files will still be able to do so.

Data Security on the PDP

Because of the risks associated with transmitting confidential data on the internet, the latest technology systems will be incorporated into the web application to ensure strict adherence to NCES confidentiality guidelines. The web server will include a Secure Sockets Layer (SSL) certificate and will be configured to force encrypted data transmission over the Internet. All data-entry modules on this site require the user to log in before accessing confidential data. Logging in requires entering an assigned ID number and two-factor authentication, entering a code that is sent via email and a password. Through the PDP, the campus coordinator at the institution will be able to use a “Manage Users” link to add and delete users, as well as reset passwords and assign roles. Each user will have a unique user name and will be assigned to one e-mail address. Upon account creation, the new user will be sent a temporary password by the PDP. Upon logging in for the first time, the new user will be required to create a new password. The system automatically will log out the user after 20 minutes of inactivity. Files uploaded to the secure website will be stored in a secure project folder that is accessible and visible to authorized project staff only.

4. Tests of Procedures or Methods

The NPSAS:24 field test enrollment list collection will be used to evaluate several new procedures designed to improve data quality and decrease burden on institutions: the bachelor’s degree recipient proxy used for identifying potential B&B cohort members (described above in section 2), the collection of enrollment lists from National Student Clearinghouse (described above in section 3), and the new pre-formatted enrollment list template file (described above in section 3). Tests of procedures or methods related to student data collection will be described in the forthcoming Student Data Collection Package to be submitted in the summer of 2022.

5. Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study

NPSAS:24 is being conducted by NCES. The following statisticians at NCES are responsible for the statistical aspects of the study: Dr. Tracy Hunt-White, Dr. David Richards, Dr. Elise Christopher, and Dr. Gail Mulligan. NCES’s prime contractor for NPSAS:24 is RTI International (Contract# 91990022C0017), and subcontractors include Activate Research; ARSIEM Corporation; EurekaFacts; Forum One Communications; HR Directions; KEN Consulting, Inc.; Leonard Resource Group; Research Support Services; Strategic Communications, Inc.; and The Equity Paradigm. Dr. Anthony Jones, Dr. Vincent Castano, Mr. Eric Atchison, Mr. Richard Reeves, Dr. Sandy Baum, Dr. Matt Springer, and Dr. Shelly Steward are consultants on the study. The following staff members at RTI are working on the statistical aspects of the study design: Mr. Peter Siegel, Dr. Jennifer Wine, Mr. Stephen Black, Ms. Ruby Johnson, Ms. Jennifer Cooney, and Dr. T. Austin Lacy. Principal professional RTI staff, not listed above, who are assigned to the study include: Ms. Kristin Dudley, Ms. Jamie Wescott, Ms. Ashley Wilson, Mr. Austin Caperton, Mr. Jeff Franklin, Dr. Nicole Tate, Mr. Michael Bryan, Dr. Erin Velez, and Dr. Josh Pretlow.

1 A decision regarding the inclusion of state-representative samples is pending. NCES is determining if funds are available for state-representative samples for the full-scale. If the budget does allow for this, then the sampling plan will be revised in the full-scale OMB package planned for 2023.

2 The U.S. service academies (the U.S. Air Force Academy, the U.S. Coast Guard Academy, the U.S. Military Academy, the U.S. Merchant Marine Academy, and the U.S. Naval Academy) are not eligible for this financial aid study because of their unique funding/tuition base.

3 A Title IV eligible institution is an institution that has a written agreement (program participation agreement) with the U.S. Secretary of Education that allows the institution to participate in any of the Title IV federal student financial assistance programs other than the State Student Incentive Grant and the National Early Intervention Scholarship and Partnership programs.

4 The end dates are December 31, 2022 and March 31, 2024 for the field test and full-scale, respectively, for continuous enrollment institutions, as described below.

5 The GED^® credential is a high school equivalency credential earned by passing the GED^® test, which is administered by GED Testing Service. For more information on the GED test and credential, see https://ged.com/about_test/test_subjects/.

6 A preliminary sampling frame has been created using data from the prior year IPEDS files, and population estimates in the sample size tables are based on this preliminary frame. The frame will be recreated with the most up to date IPEDS data prior to sample selection.

7 Chromy, J.R. (1979). Sequential Sample Selection Methods. In Proceedings of the Survey Research Methods Section of the American Statistical Association (pp. 401–406). Alexandria, VA: American Statistical Association.

8 https://support.sas.com/documentation/cdl/en/statug/63347/HTML/default/viewer.htm#statug_surveyselect_a0000000173.htm..

9 Folsom, R.E., Potter, F.J., and Williams, S.R. (1987). Notes on a Composite Size Measure for Self-Weighting Samples in Multiple Domains. In Proceedings of the Section on Survey Research Methods of the American Statistical Association. Alexandria, VA: American Statistical Association, 792–796.

10 Self-weighting samples have equal weights within sampling domains.

11 Many institutions will be able to identify their eligible students by early February.

12 The field test experiment will test the timing and effectiveness of different boost incentives that can then be applied in the full-scale study to target respondent groups. Details of this experiment will be provided in the student data collection OMB package.

13 A Hispanic-serving institutions indicator is no longer available from IPEDS, so we will create an HSI proxy following the definition of HSI as provided by the U.S. Department of Education (https://www2.ed.gov/programs/idueshsi/definition.html) and using IPEDS Hispanic enrollment data.

14 We will use the most recent version of Carnegie classification available on the IPEDS files and decide what, if any, collapsing is needed of the categories for the purposes of implicit stratification.

15 Many institutions know their enrolled students prior to April 30 and provide lists in February, March, or April. However, continuous enrollment institutions, including many of the for-profit institutions, typically cannot provide enrollment lists until mid-May, at the earliest, given that the lists include students enrolled through April 30.

16 In NPSAS:20, FTBs formed the 2020 cohort of the Beginning Postsecondary Students Longitudinal Study.

17 Field test: 2,000 = (3,967 potential baccalaureate recipients * .95 * .65 * .803) + (1,767 other undergraduate students * .95 * .65 * .03), where .803 = 1-.197; and Full-scale: 30,000 = (53,043 potential baccalaureate recipients * .95 * .70 * .803) + (83,957 other undergraduate students * .95 * .70 * .03), where .803 = 1-.197. 

The false-positive and false-negative rates will be updated for the full-scale based on field test results. The false negative rate will also be updated to account for a small percentage (less than two percent) of students who are sampled as graduate students but are baccalaureate recipients. The number of field test graduate students who are baccalaureate recipients will be trivial (less than five), so the graduate student false negative rate will be ignored for the field test.