Part A SLDS Survey 2021-2023 v.10

The National Center for Education Statistics (NCES), of the Institute of Education Sciences (IES), within the U.S. Department of Education, is requesting clearance to continue the Statewide Longitudinal Data System (SLDS) Survey collection, which is intended to provide insight on State and U.S. territory SLDS capacity for automated linking of K-12, teacher, postsecondary, workforce, career and technical education (CTE), adult education, and early childhood data. The SLDS Survey will continue to be collected annually from State Education Agencies (SEAs), and will help inform NCES ongoing evaluation and targeted technical assistance efforts to enhance the quality of the SLDS Program’s support to States regarding systems development, enhancement, and use. The request to conduct all activities related to SLDS 2020-22, including materials and procedures, was approved by OMB in May 2020 (OMB#1859-0933 v.8), with a nonsubstantive change request (OMB#1859-0933 v.9) approved in August 2020. In Summer 2021, in partial response to some comments received during the 60D comment period, Appendices A-E were revised and new copies were submitted as part of 30D review.

The SLDS 2020-22 package included a new data collection tool, a Google Form developed for an electronic data collection. That tool was not as successful as NCES would like (see section A.3 for a richer discussion of this). This new request is to conduct all activities related to SLDS 2021-23. It submits enhancements to the OMB-approved Survey, intended to bring consistency to questions across sectors, provide greater definition and clarity to terminology and questions used within the SLDS Survey, and address pandemic-related response across states. In addition, this request submits screenshots of the new Qualtrics information collection tool that will replace the Google Form introduced for SLDS 2020 and which will be used in the 2021 SLDS Data Collection (for proposed changes, see Appendix E) and is planned for use in subsequent collections. Finally, this request submits screenshots of the updated webinar, as the SLDS Program proposes the option to host one or two SLDS Survey webinars to familiarize respondents with the collection tool and completion process. All proposed changes are captured within these documents, including accompanying appendices.

Legislative Authorization

NCES is authorized to collect this information by the Education Sciences Reform Act of 2002 (ESRA 2002; 20 U.S.C., § 9543), which establishes the SLDS Grant Program.

Prior and Related Studies

The SLDS Survey is the first formal, systematic collection of SLDS system capacity data of its kind. Aspects of these data have been obtained less systematically during regular interactions with SLDS grantees. This effort provides better information to meet NCES’ quarterly progress reporting regarding State capacity to link and use data, to inform future grant rounds, technical assistance efforts, and public knowledge of State capacity to link and use data.

Prior to the development and administration of the SLDS Survey, States provided some indication of data linkages and use to NCES program staff through annual reporting, monthly monitoring updates, and State site visits, these data do not provide a comprehensive look at data capacity. Reasons for incomplete data include, but are not limited to:

Data are collected from States that have active grants, which results in missing data for non-grantee States; and
Grantee States report primarily on their proposed – and funded – projects. For the last three collections, the SLDS Survey has allowed States to discuss the full capacity of their State data systems, which has led to more accurate information and discussion about system capacity across the field.

Prior to the development of the OMB-approved SLDS Survey, the SLDS program office attempted to collect more uniform information about data linkages (for example, asking about early learning program data linkages), but, these efforts have been limited to States with active grants.

External organizations, including the Data Quality Campaign (DQC), have conducted surveys to document data linkage and use capacity at the State level. The DQC data are limited by three factors:

DQC stopped collecting information about data linkages in 2011, with no replacement data source in place.
The DQC survey relied on fairly dichotomous measures of data linking (where a yes response indicated that a State had a link in place, and a no response indicated that a State did not have a link in place). States, however, tend to implement linkages more gradually. For example, a State might conduct a pilot in which Pre-K and K-12 data are linked for one Local Education Agency (LEA), or might link data from a limited set of Pre-K sources, such as Head Start or Early Head Start. The proposed NCES measure allows for States to report on the continuum of data linkage and capacity. For example, the proposed SLDS Survey enables States to rate their own data linkage and use efforts as “Not planned,” “Planned,” “In Progress,” or “Operational.” A State that has established a pilot data linkage process would be deemed “In Progress,” while a universal roll-out would be considered “Operational.” This has proved more effective when attempting to viably gauge a State’s capacity status.
The DQC survey produced less public information about how States were matching data (for example, through a manual process or an automated process) and who was matching the data (for example, a State agency or a vendor such as NSC). Such information has proven to be quite useful when assessing States’ needs and capacity for data linkages.

2. Purposes and Uses of the SLDS Survey

Since 2005, the U.S. Department of Education has awarded approximately $721 million in 97 grants to State Education Agencies to enable them to implement and enhance their SLDS systems. Additional competitive funding was expended prior to the close of FY20 (awarded as FY19 competition and funds). The Department continues to need to maintain a clear and formal means of summarizing and communicating the status of these systems across all States and Territories to: 1) evaluate current but everchanging needs for further systems development; 2) provide targeted technical assistance to States; and 3) accurately reflect progress on the development and use of statewide longitudinal data systems.

Survey results will continue to inform:

Future grant rounds for the SLDS grant program and technical assistance support;
Program offices in the Department of Education, Department of Labor, and Health and Human Services, in addition to external stakeholders;
State development and support efforts; and
Public knowledge of State capacity to link and use longitudinal data.

State information about State capacity for data linkages and use is vital to ensure that program dollars are targeted both for grant funding and for technical assistance development. As federal funding becomes increasingly limited and more competitive (SLDS infrastructure a reemerging priority in FY19 grant competition), we must continue to have a clear sense of SLDS progress across the United States so that federal resources can be utilized and offered most efficiently and effectively. Prior to the development of the SLDS Survey, the SLDS grant program was responsible for providing OMB with up-to-date state capacity indicators on a quarterly basis, with the shortcoming that any changes or updates to these data primarily reflect information from active grantee states only. The report was produced based on continual communication with active grantees that allows the SLDS Program Officers to remain informed of these states’ systems’ capacity, progress, and constraints. Moreover, active grantees are responsible for providing summary reports on at least an annual basis, and this reporting validated assumptions and conversations that took place throughout the year between grantee states and SLDS Program Officers. Reporting for states without active grants had been only ad hoc. For the last three years, the SLDS Survey has not only formalized the data collection processes to respond to these indicators, but also played an integral role in allowing the SLDS program to have more leverage in reaching States without active grants.

There continues to be growing interest in SLDS capacity across the United States both internally within the Department of Education, among States and U.S. territories, and across agencies with common and shared interests (Department of Labor’s Workforce Data Quality Initiative, for example). The SLDS program regularly responds to ad-hoc questions and requests regarding State capacity for data linking and use, including, for example:

How many States can link:
- teacher preparation programs of teachers to student outcomes for students taught by those teachers (Title II);
- K12 and postsecondary data (Performance metric, OPEPD);
- K12, postsecondary, and workforce data (Performance metric, OPEPD, Department of Labor, Workforce Data Quality Campaign, White House Workforce Convening);
- K12 and early learning data (Performance metric, Early Learning Challenge Technical Assistance, Office of Special Education Programs, US Department of Health and Human Services); and
- How are States using data (Performance metric, US Department of Labor).

States and Territories themselves often seek information about which States are linking and using data, and what their processes entail. The SLDS Program facilitates States’ efforts to share promising practices with each other. This enables States to more easily collaborate, learn from each other, share resources with each other, and avoid duplicative work in the presence or absence of SLDS federal funds.

The SLDS program also receives questions about State capacity from the public, which is interested in learning which data are available at the State level and how the data might be accessed. The SLDS program has now developed and codified a recurring process for the first IES-approved Statistics in Brief inclusive of 2017 data, which was released in October 2019. A Statistics in Brief will be released post-analysis of each SLDS Survey collection, and includes descriptive statistics, metrics and use cases showing data-linking and data-use capacity by State. Beginning with the 2018 data Statistics in Brief, interested users will be able to quickly ascertain which States have the capacity to link data across sectors (for example, which States and Territories can link K12, postsecondary, and workforce data) through the accessibility of an accompanying data file. The Statistics in Briefs also include examples of State data use capacity, for example, which States are providing feedback reports so that policymakers at the local level understand how their high school graduates are faring in postsecondary education or the workforce.

3. Appropriate Use of Information Technology

Beginning with the 2018 collection, the SLDS Survey was pre-populated with previous year’s responses to ease the respondent burden. In 2020 we introduced a Google Form as the online data collection tool, and in 2021 we will introduce a replace for that tool built on Qualtrics. Despite the shift away from Google Form as the SLDS Survey instrument, the pre-population will continue with the proposed change to the Qualtrics information collection tool. The link to the Qualtrics information collection tool will be distributed to SEA contacts electronically, as an email, by a Program Analyst. The Program Analyst will act as the primary contact for all questions regarding the Survey, and consult NCES Program staff as needed. All states continue to have a Program Officer and Analyst contact despite their grant status (including states that do not have active grants).

There were unanticipated challenges for states and territories in the utilization of the Google Forms platform, a burden that should be reduced as much as possible. Moving to a survey tool that is built with the capability and functionality to prepopulate prior year’s information and format the SLDS Survey in a clean, neat presentation will make it easier for states and territories to access, navigate, and complete the Survey. Qualtrics allows greater consistency over the coming years with a more robust pre-population option and built-in data quality checks. This tool will support more efficient analysis and reporting from the Survey, as opposed to spending such significant time on formatting, as with the 2020 Survey administration. Some additional key challenges that the Google Form presented that Qualtrics will ultimately alleviate are as follows:

• Potential impacts to data quality

• Limited ability to format the Survey

• State Education Agencies’ inability to access the Google Form links

• Additional burden on SEAs through multiple rounds of QA follow up (multiple contacts that wouldn’t have otherwise needed to occur)

• Additional follow up when there were duplicate or incomplete submissions

• The need for resubmissions (and QC to follow to ensure the correct submission was used)

While the SLDS Survey was originally developed using Microsoft Word, this has since evolved to a tool that intends to ease completion efforts and ultimately reduce respondent burden. The tool has a built-in skip pattern, text boxes, and formatting restrictions that cannot be manipulated.

The information collection tool is expected to facilitate the data collection process, increase the reliability of the data, and reduce error. The Qualtrics information collection tool data can also be downloaded to a .csv or Excel file to ease the analysis process and will be stored on a secure NCES server. NCES Program staff will manage all data and conduct analysis in collaboration with Applied Engineering Management Corporation technical and research staff, made available via the existing Technical Assistance contract (EDTAP contract number ED-IES-C-0015).

In the instance that a State is unable to access Qualtrics or needs a paper copy as a reference copy to use during their consultations with other program offices, a Microsoft Word version of the document (pre-populated with as much information as is available to program staff) can be provided upon request via email (see Appendix E, Section 2 for the updated version of Word instrument; proposed changes will be finalized upon approval of this request). In the highly unlikely case that the Qualtrics platform is inaccessible, States may elect to complete the flat file instrument and return it via email. A paper copy can also be provided for reference.

With a high response rate, the SLDS community is quite familiar with the Survey and anticipate its dissemination on an annual basis at this point. NCES plans to continue to host one to two webinars in 2021 to introduce the new platform to States and their respective respondents. Additionally, with awarding of the newest grant round, FY19 grantees, it will be an ideal opportunity to discuss Survey changes to an already-engaged community.

Respondents will be invited to participate in webinars via listserv email invitation. The listserv is used regularly to communicate with the SLDS community. Webinars will be used as a tool to continue to reiterate the purpose of the OMB-formalized survey, how to navigate the collection tool, and in what manner the data will be used by NCES. In subsequent years, NCES may host one or two webinars per year on an as- needed basis, to answer any questions states may have regarding the Survey.

4. Efforts to Identify Duplication

The information collected through the SLDS Survey does not duplicate information requested or collected by any other federal agency. Further, there is no similar current information available on a consistent national basis that could be used or modified for these purposes. Program offices within the US Department of Education often request and report on similar data. Having a single source of information will decrease redundant data collections and improve ED’s ability to provide valid and reliable data for internal and external users.

In the past, a similar but not equivalent survey has been conducted on an annual basis by DQC, a nonprofit organization participating in national effort to bring quality information to education stakeholders. Between 2005 and 2011, DQC surveyed States in an attempt to report their progress towards the building of longitudinal data systems and implementing effective data use. In 2009, DQC launched the 10 State Actions to Ensure Effective Data Use, which document States’ capacity to use the data in their systems.

While many of the questions that DQC has asked States to report on in the past parallel those set forth in the proposed SLDS Survey, the transition from data linking to data use in 2011 resulted in losing information about fundamental SLDS capacities. Because DQC has taken a new direction, States are no longer asked to report on the types of questions that can assist us in assessing SLDS progress to-date. It is crucial that these data continue to be collected at the national level to guide future efforts in SLDS development and to provide information about State capacity to link and use education data. NCES has used the DQC survey data responses in the past as a resource to help understand changes in State capacity since 2011, and evaluate State SLDS development and data use progress. Nearly a decade later, this resource is now outdated and the SLDS Survey has proven to be a viable solution to better understanding both capacity and use in an ever-evolving field.

5. Minimize Burden

NCES has devised several measures to minimize the response burden for States and Territories participating in the SLDS Survey. On a continual basis, questions are reviewed by the federal SLDS Program Team and the State Support Team (SST) members, a panel of experts who support the Program by offering technical assistance to States. This process led to the majority of the changes proposed within this request. These proposed changes can be found in Appendix E. All SST members have held leadership positions in their respective SEAs. As a result, they are generally aware of the level of burden that the Survey is likely to impose. Each contributor took this into account when providing input, in an attempt to consider conflicting SEA responsibilities and demands and to minimize burden. As with the decision to pre-populate the instrument, SST were advocates for enhancements to the information collection tool, also to reduce the burden on States. Additionally, NCES may offer webinars to provide more information to respondents about the Survey, how to complete the instrument online, and NCES’s planned uses for the data. The SLDS Program Team is consistent in proactive efforts aimed to minimize respondent burden over the long-term.

6. Frequency of Data Collection

The SLDS Survey continues to be an annual survey, scheduled to begin November 2021, and will be administered in November of the following years. Nationwide, SLDS system capacity changes frequently (ex. Infrastructure enhancements, evolving P20W agency collaborations, State legislation impacts, etc.), so collecting data less often would make the information too obsolete to be useful for targeted technical assistance planning.

7. Special Circumstances of Data Collection

There are no additional circumstances that will require special data collection efforts.

8. Consultants Inside and Outside the Agency

Upon development, OMB requested that the survey instrument be reviewed by a methodological expert within NCES. SLDS staff asked Dr. Andy Zukerberg, at NCES, to review the instrument. Dr. Zukerberg suggested revising the skip pattern, providing further (but concise) definition of key concepts mentioned, considering shortening the survey in length, and piloting it with a few SEAs. Per recommendations, the skip pattern was revised and enhancements were made to concept definitions and Survey instructions prior to piloting.

The SLDS survey was initially piloted with the Kentucky, Minnesota, and Washington State Project Teams. Each participating SEA was given approximately two weeks to complete the Survey with notification that completion might require collaboration from other SLDS stakeholders, outside of the immediate project team. Once completed, a debrief teleconference was held to discuss possible improvements, suggestions, and other feedback. In general, pilot participants indicated that they preferred the SLDS Survey over the leading, external survey designed to measure State’s progress towards SLDS development and implementation.

State pilot participants were satisfied with the length of the SLDS survey, stating that while it is somewhat extensive, it is comprehensive in assessing the current state and robustness of SLDS and P20W capacity. The length of the Survey has received recurring feedback as enhancements have been made to add data use questions and bring gradual cohesiveness across sectors, but there seems to be a clear understanding of the need and value across the field. Based on the feedback received during the pilot, changes were made to the overall SLDS Survey structure, content, instructions, concept definitions, and language. As a result of the pilot, a comment box was also added to the end of the SLDS Survey so that State respondents could provide any desired clarifications or explanations. In order to prepare for the renewal of the SLDS Survey, SST convened in two intensive sessions to review the Survey, address inconsistencies, provide clearer definitions and enhance cohesiveness in questions across sectors.

Select states were consulted regarding the impact and feasibility of shifting to an online platform, which was viewed favorably.

In addition to the internal NCES review and SEA piloting, the following individuals from the SLDS State Support Team reviewed the data collection content and plans for the initial Survey release:

from Applied Engineering Management Corporation: Kathy Gosa (SST Lead), Missy Cochenour, Carla Howe, Bill Huennekens, Joyce Popp, Baron Rodriguez, and Jeff Sellers; and
from Chatis Consulting: Corey Chatis.

Since the inception of the SLDS Survey, feedback and suggestions have been solicited periodically and welcomed on an ongoing basis through the following measures:

Point of contact provided on the Survey instrument,
Point of contact provided on SLDS website once,
SST,
Conferences, and
Opportunity for discussion during monthly SLDS teleconference calls.

9. Provision of payments or Gifts to Respondents

No payments or gifts will be offered to Survey respondents.

10. Assurance of Confidentiality

Data collected through the SLDS Survey are public domain data in their respective districts and States. As such, the data collection does not include a pledge of confidentiality. Confidentiality was offered the first collection year, as no data file was released with the Statistics in Brief, but NCES plans to release an accompany data file moving forward, initiated by the Statistics in Brief associated with the 2018 SLDS Survey data collection.

11. Sensitive Questions

None of the questions asked during the SLDS Survey are of a sensitive nature.

12. Estimates of Hour Burden for Information Collection

The response burden will vary by State and U.S. territory, with the expectation that on average it will take 2 hours for each SEA to complete the SLDS Survey. Although the expectation is for the Program Director or past Program Director to complete the Survey on behalf of the State or U.S. territory, staff turnover and/or level of knowledge and expertise varies by State. For example, respondents from a State that has focused on building a K12 SLDS (as opposed to a P20W) might possess limited (if any) knowledge on workforce and postsecondary system capabilities due to the fact that the workforce system might not be housed in their agency. In such cases, cross-agency communication and collaboration may be required to effectively and successfully complete the SLDS Survey. By contrast, a State or U.S. territory with a tenured respondent involved in the implementation of a P20W might be capable to complete the survey independently, with greater ease. The proposed changes found in this request are intended to bring greater consistency and definition to Survey respondents, ultimately intended to reduce burden. Proposed changes are captured in Appendix E.

NCES may host one or two webinars annually, on an as-needed basis, to provide more information to respondents about the Survey, and to answer their questions. The webinar slides have been updated to remain compliant with IES-approved branding, as well as minimize the need for annual updates, by removing screenshots of the platform. Screenshots can be found in Appendix B.

The estimated hours per respondent are based on information directly provided by past state Program Directors who have completed comparable information requests in previous years. Assuming that the respondents (state education agency administrators) earn on average $54.35^¹per hour, the total annualized burden time cost to respondents for the SLDS Survey is estimated to be $7,609.

Table 1. Annual burden estimates for SLDS 2021-2023

Respondent	Number of Respondents	Number of Responses	Estimated hours per respondent	Total Respondent Burden Hours
States / U.S. Territories	56	56	2	112
Webinars	56	56	0.5	28
TOTAL	56	112	-	140

13. Estimates of Costs

SLDS Survey respondents will not incur any costs for this data collection other than their time to respond.

14. Annualized Cost to the Federal Government

Method for estimating costs: The full costs include the projected annual amount of time that Department of Education staff will spend on the survey, separated by labor type. Contracted staff is intended to support pre-collection, collection and analysis, with federal guidance and oversight. Salary costs are based on average FTE salaries and hourly rates for contract subject matter experts and program analysts.

Table 2. Estimated Annual Cost of the SLDS Survey to Federal Government for Fiscal Year 2021

Labor Type	Annual Labor Hours	Cost
Collection and Data Entry	386	$23,259.71
Data Analysis	540	$54,307.69
Product Development and Publishing	375	$31,944.71
Total	1301	$109,512.11

Cost: Department of Education staff assigned to SLDS Survey include one-third (FTE) of one Program Officer’s (GS13) time. Contracted staff also supporting the SLDS Survey (included in SLDS Contract total) consists of 20 percent of one SST and about 10 percent of one full-time Program Analysts employed through Applied Engineering Management Corporation.

15. Reasons for Changes in Response Burden and Costs

There is no change in response burden or annualized burden time costs. The change from the online Google Form to Qualtrics information collection tool does not represent a significant change for respondents in terms of the estimated time of completion, but enhanced technology and pre-population should ultimately ease the respondent process.

Collection and data entry costs have increased greatly to support the pre-population process, dissemination of Surveys, technical support and data transfer/entry in the instance of flat file response. As we are moving into our fifth year of collection, data is more abundant. Therefore, data analysis will begin to consider a more longitudinal study of SLDS capacity over time. This has yielded a dramatic increase in labor hours, as we plan more analysis that will ultimately be shared with the field through technical assistance and recurring Statistics in Brief documents. Product development and publishing experienced an increase in costs as we now know that the formal product resulting from this Survey is an annual, IES-published Statistics in Brief, which requires an extensive IES review and approval process prior to release.

The Federal Government will incur no additional cost for the implementation of this survey beyond the existing cost of managing the SLDS grant program and associated Technical Assistance contract (EDTAP contract number ED-IES-C-0015).

Time Schedule for SLDS Survey

The SLDS Survey is an annual collection. The proposed schedule is for the FY21 survey administration. FY22 and FY23 collections are scheduled as the subsequent and final cycles of this OMB approval and will follow the appropriate 2022 and 2023 versions of the timeline detailed in Table 3 below.

Table 3. Schedule and Timeline

2021 Timeline	SLDS Survey Collection, Processing, and Publication
November 2021	Email instructions to SEA respondents
November 2021	One or two webinars, on an as-needed basis, to provide more information about the Survey, how to complete the instrument on the new online platform (Qualtrics), NCES’s planned use of the data, and to address respondents’ questions about the Survey
November 15–30, 2021	Survey final reminder email
November 30, 2021	SEAs are urged to have finished submitting accurate and complete data
December 15, 2021	Mandatory final submission date
January 2022	Response by SEA’s to requests for clarification, reconciliation, or other inquiries from NCES. All data issues to be resolved. No files are accepted after close-out.
March 15, 2022	NCES review of files, file documentation, and brief analysis completed. Provisional responses available for internal use but not publication
July 15, 2022	Statistics in Brief and accompanying data file become public, NCES website updated. Current year collection data will be available to assess and respond to ad hoc requests
July 15 – October 2022	Respondents have the option to make update, change, or reconciliation requests to adjust state-specific data reflected in the SLDS Survey Statistics in Brief

NCES will release an IES-approved Statistics in Brief (and accompanying data file) associated with each annual SLDS Survey collection. Interested users and stakeholders will be enabled to quickly ascertain which States have capacity to link data across sectors, for example, which can link K12, postsecondary, and workforce data, and how they are using these data to inform policy and practice. Upon IES approval, the Statistics in Brief and data file will be published to the SLDS website. The SLDS grant program is currently responsible for providing updated indicators to OMB on a quarterly basis, so the validity of this reporting will be enhanced as a result of this collection. As data needs evolve and staff capacity permits, the intention is to post more data publicly. The data collected from the SLDS Survey will also be used to respond to questions from internal and external stakeholders regarding SLDS capacity in the States, and to inform future grant rounds and technical assistance planning.

17. Approval to not Display Expiration Date for OMB Approval

No approval is sought to not display the expiration date of OMB approval.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification for Paperwork Reduction Act submission.

1The hourly mean wage for financial managers (SOC code 113031) working in State government is $54.35 per the May 2020 National Occupational and Employment Wage Estimates sponsored by the Bureau of Labor Statistics (BLS). Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ Accessed on May 17, 2021.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Carrie Clarady
File Modified	0000-00-00
File Created	2021-10-09

Part A SLDS Survey 2021-2023 v.10