Part A SLDS Survey 2020-2022 (1)

The National Center for Education Statistics (NCES), of the Institute of Education Sciences (IES), within the U.S. Department of Education, is requesting clearance to continue the Statewide Longitudinal Data System (SLDS) Survey collection, which is intended to provide insight on State and U.S. territory SLDS capacity for automated linking of K-12, teacher, postsecondary, workforce, career and technical education (CTE), adult education, and early childhood data. The SLDS Survey will continue to be collected annually from State Education Agencies (SEAs), and will help inform NCES ongoing evaluation and targeted technical assistance efforts to enhance the quality of the SLDS Program’s support to States regarding systems development, enhancement, and use. This submission is to request OMB clearance to conduct the annual SLDS Survey from 2020 through 2022. Upon clearance of this package, a change request will be submitted with screenshots of the Google Form (see section A.3 for more information) that is still being programmed. While the date of that change request is entirely dependent upon the clearance of this package, we anticipate that the subsequent change request will be submitted in late Spring 2020.

Legislative Authorization

NCES is authorized to collect this information by the Education Sciences Reform Act of 2002 (ESRA 2002; 20 U.S.C., § 9543), which establishes the SLDS Grant Program.

Prior and Related Studies

The SLDS Survey is the first formal, systematic collection of SLDS system capacity data of its kind. Aspects of these data have been obtained less systematically during regular interactions with SLDS grantees. This effort provides better information to meet NCES’ quarterly progress reporting regarding State capacity to link and use data, to inform future grant rounds, technical assistance efforts, and public knowledge of State capacity to link and use data.

Prior to the development and administration of the SLDS Survey, States provided some indication of data linkages and use to NCES program staff through annual reporting, monthly monitoring updates, and State site visits, these data do not provide a comprehensive look at data capacity. Reasons for incomplete data include, but are not limited to:

Data are collected from States that have active grants, which results in missing data for non-grantee States; and
Grantee States report primarily on their proposed – and funded – projects. For the last three collections, the SLDS Survey has allowed States to discuss the full capacity of their State data systems, which has led to more accurate information and discussion about system capacity across the field.

Prior to the development of the OMB-approved SLDS Survey, the SLDS program office attempted to collect more uniform information about data linkages (for example, asking about early learning program data linkages), but, these efforts have been limited to States with active grants.

External organizations, including the Data Quality Campaign (DQC), have conducted surveys to document data linkage and use capacity at the State level. The DQC data are limited by three factors:

DQC stopped collecting information about data linkages in 2011, with no replacement data source in place.
The DQC survey relied on fairly dichotomous measures of data linking (where a yes response indicated that a State had a link in place, and a no response indicated that a State did not have a link in place). States, however, tend to implement linkages more gradually. For example, a State might conduct a pilot in which Pre-K and K-12 data are linked for one Local Education Agency (LEA), or might link data from a limited set of Pre-K sources, such as Head Start or Early Head Start. The proposed NCES measure allows for States to report on the continuum of data linkage and capacity. For example, the proposed SLDS Survey enables States to rate their own data linkage and use efforts as “Not planned,” “Planned,” “In Progress,” or “Operational.” A State that has established a pilot data linkage process would be deemed “In Progress,” while a universal roll-out would be considered “Operational.” This has proved more effective when attempting to viably gauge a State’s capacity status.
The DQC survey produced less public information about how States were matching data (for example, through a manual process or an automated process) and who was matching the data (for example, a State agency or a vendor such as NSC). Such information has proven to be quite useful when assessing States’ needs and capacity for data linkages.

2. Purposes and Uses of the SLDS Survey

Since 2005, the U.S. Department of Education has awarded approximately $721 million in 97 grants to State Education Agencies to enable them to implement and enhance their SLDS systems. Additional competitive funding is projected to be expended prior to the close of FY20 (FY19 competition and funds). The Department continues to need to maintain a clear and formal means of summarizing and communicating the status of these systems across all States and Territories to: 1) evaluate current but everchanging needs for further systems development; 2) provide targeted technical assistance to States; and 3) accurately reflect progress on the development and use of statewide longitudinal data systems.

Survey results will continue to inform:

Future grant rounds for the SLDS grant program and technical assistance support;
Program offices in the Department of Education, Department of Labor, and Health and Human Services, in addition to external stakeholders;
State development and support efforts; and
Public knowledge of State capacity to link and use longitudinal data.

State information about State capacity for data linkages and use is vital to ensure that program dollars are targeted both for grant funding and for technical assistance development. As federal funding becomes increasingly limited and more competitive (SLDS infrastructure a reemerging priority in FY19 grant competition), we must continue to have a clear sense of SLDS progress across the United States so that federal resources can be utilized and offered most efficiently and effectively. Prior to the development of the SLDS Survey, the SLDS grant program was responsible for providing OMB with up-to-date state capacity indicators on a quarterly basis, with the shortcoming that any changes or updates to these data primarily reflect information from active grantee states only. The report was produced based on continual communication with active grantees that allows the SLDS Program Officers to remain informed of these states’ systems’ capacity, progress, and constraints. Moreover, active grantees are responsible for providing summary reports on at least an annual basis, and this reporting validated assumptions and conversations that took place throughout the year between grantee states and SLDS Program Officers. Reporting for states without active grants had been only ad hoc. For the last three years, the SLDS Survey has not only formalized the data collection processes to respond to these indicators, but also played an integral role in allowing the SLDS program to have more leverage in reaching States without active grants.

There continues to be growing interest in SLDS capacity across the United States both internally within the Department of Education, among States and U.S. territories, and across agencies with common and shared interests (Department of Labor’s Workforce Data Quality Initiative, for example). The SLDS program regularly responds to ad-hoc questions and requests regarding State capacity for data linking and use, including, for example:

How many States can link:
- teacher preparation programs of teachers to student outcomes for students taught by those teachers (Title II);
- K12 and postsecondary data (Performance metric, OPEPD);
- K12, postsecondary, and workforce data (Performance metric, OPEPD, Department of Labor, Workforce Data Quality Campaign, White House Workforce Convening);
- K12 and early learning data (Performance metric, Early Learning Challenge Technical Assistance, Office of Special Education Programs, US Department of Health and Human Services); and
- How are States using data (Performance metric, US Department of Labor).

States and Territories themselves often seek information about which States are linking and using data, and what their processes entail. The SLDS Program facilitates States’ efforts to share promising practices with each other. This enables States to more easily collaborate, learn from each other, share resources with each other, and avoid duplicative work in the presence or absence of SLDS federal funds.

The SLDS program also receives questions about State capacity from the public, which is interested in learning which data are available at the State level and how the data might be accessed. The SLDS program has now developed and codified a recurring process for the first IES-approved Statistics in Brief inclusive of 2017 data, which was released in October 2019. A Statistics in Brief will be released post-analysis of each SLDS Survey collection, and includes descriptive statistics, metrics and use cases showing data-linking and data-use capacity by State. Beginning with the 2018 data Statistics in Brief, interested users will be able to quickly ascertain which States have the capacity to link data across sectors (for example, which States and Territories can link K12, postsecondary, and workforce data) through the accessibility of an accompanying data file. The Statistics in Briefs also include examples of State data use capacity, for example, which States are providing feedback reports so that policymakers at the local level understand how their high school graduates are faring in postsecondary education or the workforce.

3. Appropriate Use of Information Technology

Beginning with the 2018 collection, the SLDS Survey was pre-populated with previous year’s responses to ease the respondent burden. The pre-population will continue with the development of the online Google Form and ultimate Survey administration for this purpose; the 2020 online collection will be pre-populated with the 2019 flat file Survey response. The link to the Form will be distributed to SEA contacts electronically, as an email, by a Program Analyst. The Program Analyst will act as the primary contact for all questions regarding the Survey, and consult NCES Program staff as needed. All states continue to have a Program Officer and Analyst contact despite their grant status (including states that do not have active grants).

While the SLDS Survey was originally developed using Microsoft Word, this has since evolved to a PDF version that was developed to ease completion efforts and ultimately reduce respondent burden. The resulting standard form has a built-in skip pattern, text boxes, and formatting restrictions that cannot be manipulated, and this will be replicated within the online Google Form currently under development.

The online platform is expected to facilitate the data collection process, increase the reliability of the data, and reduce error. The Google Form data will also be downloaded to a .csv or Excel file to ease the analysis process, and will be stored on a secure NCES server. NCES Program staff will manage all data and conduct analysis in collaboration with Applied Engineering Management Corporation technical and research staff, made available via the existing Technical Assistance contract (EDTAP contract number ED-IES-C-0015).

In the instance that a State is unable to access the online platform, a Microsoft Word version of the document can be provided upon request.

With a 96% response rate, the SLDS community is quite familiar with the Survey and anticipate its dissemination on an annual basis at this point. NCES plans to continue to host one to two webinars in 2020 to introduce the online platform to States and their respective respondents. Additionally, with the anticipation of new grantees within 2020, it will be an ideal opportunity to discuss Survey changes to an already-engaged community.

Respondents will be invited to participate in webinars via listserv email invitation. The listserv is used regularly to communicate with the SLDS community. Webinars will be used as a tool to continue to reiterate the purpose of the OMB-formalized survey, how to complete the online Google Form, and in what manner the data will be used by NCES. In subsequent years, NCES may host one or two webinars per year on an as- needed basis, to answer any questions states may have regarding the Survey.

4. Efforts to Identify Duplication

The information collected through the SLDS Survey does not duplicate information requested or collected by any other federal agency. Further, there is no similar current information available on a consistent national basis that could be used or modified for these purposes. Program offices within the US Department of Education often request and report on similar data. Having a single source of information will decrease redundant data collections and improve ED’s ability to provide valid and reliable data for internal and external users.

In the past, a similar but not equivalent survey has been conducted on an annual basis by DQC, a nonprofit organization participating in national effort to bring quality information to education stakeholders. Between 2005 and 2011, DQC surveyed States in an attempt to report their progress towards the building of longitudinal data systems and implementing effective data use. In 2009, DQC launched the 10 State Actions to Ensure Effective Data Use, which document States’ capacity to use the data in their systems.

While many of the questions that DQC has asked States to report on in the past parallel those set forth in the proposed SLDS Survey, the transition from data linking to data use in 2011 resulted in losing information about fundamental SLDS capacities. Because DQC has taken a new direction, States are no longer asked to report on the types of questions that can assist us in assessing SLDS progress to-date. It is crucial that these data continue to be collected at the national level to guide future efforts in SLDS development and to provide information about State capacity to link and use education data. NCES has used the DQC survey data responses in the past as a resource to help understand changes in State capacity since 2011, and evaluate State SLDS development and data use progress. Nearly a decade later, this resource is now outdated and the SLDS Survey has proven to be a viable solution to better understanding both capacity and use in an ever-evolving field.

5. Minimize Burden

NCES has devised several measures to minimize the response burden for States and Territories participating in the SLDS Survey. On a continual basis, questions are reviewed by the federal SLDS Program Team and the State Support Team (SST) members, a panel of experts who support the Program by offering technical assistance to States. All SST members have held leadership positions in their respective SEAs. As a result, they are generally aware of the level of burden that the Survey is likely to impose. Each contributor took this into account when providing input, in an attempt to consider conflicting SEA responsibilities and demands and to minimize burden. As with the decision to pre-populate the instrument, SST were advocates of shifting to an online platform for Survey completion, also to reduce the burden on States. Additionally, NCES may offer webinars to provide more information to respondents about the Survey, how to complete the instrument online, and NCES’s planned uses for the data. These proactive efforts are aimed to minimize respondent burden over the long-term.

6. Frequency of Data Collection

The SLDS Survey will continue to be an annual survey that is scheduled to begin October 2020, and will be administered in October of subsequent OMB-approved years. Nationwide, SLDS system capacity changes frequently (ex. Infrastructure enhancements, evolving P20W agency collaborations, State legislation impacts, etc.), so collecting data less often would make the information too obsolete to be useful for targeted technical assistance planning.

7. Special Circumstances of Data Collection

There are no additional circumstances that will require special data collection efforts.

8. Consultants Inside and Outside the Agency

Upon development, OMB requested that the survey instrument be reviewed by a methodological expert within NCES. SLDS staff asked Dr. Andy Zukerberg, at NCES, to review the instrument. Dr. Zukerberg suggested revising the skip pattern, providing further (but concise) definition of key concepts mentioned, considering shortening the survey in length, and piloting it with a few SEAs. Per recommendations, the skip pattern was revised and enhancements were made to concept definitions and Survey instructions prior to piloting.

The SLDS survey was initially piloted with the Kentucky, Minnesota, and Washington State Project Teams. Each participating SEA was given approximately two weeks to complete the Survey with notification that completion might require collaboration from other SLDS stakeholders, outside of the immediate project team. Once completed, a debrief teleconference was held to discuss possible improvements, suggestions, and other feedback. In general, pilot participants indicated that they preferred the SLDS Survey over the leading, external survey designed to measure State’s progress towards SLDS development and implementation.

State pilot participants were satisfied with the length of the SLDS survey, stating that while it is somewhat extensive, it is comprehensive in assessing the current state and robustness of SLDS and P20W capacity. The length of the Survey has received recurring feedback as enhancements have been made to add data use questions and bring gradual cohesiveness across sectors, but there seems to be a clear understanding of the need and value across the field. Based on the feedback received during the pilot, changes were made to the overall SLDS Survey structure, content, instructions, concept definitions, and language. As a result of the pilot, a comment box was also added to the end of the SLDS Survey so that State respondents could provide any desired clarifications or explanations. In order to prepare for the renewal of the SLDS Survey, SST convened in two intensive sessions to review the Survey, address inconsistencies, provide clearer definitions and enhance cohesiveness in questions across sectors.

Select states were consulted regarding the impact and feasibility of shifting to an online platform, which was viewed favorably.

In addition to the internal NCES review and SEA piloting, the following individuals from the SLDS State Support Team reviewed the data collection content and plans for the initial Survey release:

from Applied Engineering Management Corporation: Kathy Gosa (SST Lead), Missy Cochenour, Carla Howe, Bill Huennekens, Joyce Popp, Baron Rodriguez, and Jeff Sellers; and
from Chatis Consulting: Corey Chatis.

For the SLDS Survey renewal, the following individuals from the SLDS State Support Team reviewed the 2019 Survey instrument to propose recommendations:

Carla Howe (SST Lead), Julia Bloom-Weltman (Statistics in Brief author), Corey Chatis, Kathi Gillaspy, Kathy Gosa, Tony Ruggiero, Jeff Sellers and Jeff Watson

Since the inception of the SLDS Survey, feedback and suggestions have been solicited periodically and welcomed on an ongoing basis through the following measures:

Point of contact provided on the Survey instrument,
Point of contact provided on SLDS website once,
SST,
Conferences, and
Opportunity for discussion during monthly SLDS teleconference calls.

9. Provision of payments or Gifts to Respondents

No payments or gifts will be offered to Survey respondents.

10. Assurance of Confidentiality

Data collected through the SLDS Survey are public domain data in their respective districts and States. As such, the data collection does not include a pledge of confidentiality. Confidentiality was offered the first collection year, as no data file was released with the Statistics in Brief, but NCES plans to release an accompany data file moving forward, initiated by the Statistics in Brief associated with the 2018 SLDS Survey data collection.

11. Sensitive Questions

None of the questions asked during the SLDS Survey are of a sensitive nature.

12. Estimates of Hour Burden for Information Collection

The response burden will vary by State and U.S. territory, with the expectation that on average it will take 2 hours for each SEA to complete the SLDS Survey. Although the expectation is for the Program Director or past Program Director to complete the Survey on behalf of the State or U.S. territory, staff turnover and/or level of knowledge and expertise varies by State. For example, respondents from a State that has focused on building a K12 SLDS (as opposed to a P20W) might possess limited (if any) knowledge on workforce and postsecondary system capabilities due to the fact that the workforce system might not be housed in their agency. In such cases, cross-agency communication and collaboration may be required to effectively and successfully complete the SLDS Survey. By contrast, a State or U.S. territory with a tenured respondent involved in the implementation of a P20W might be capable to complete the survey independently, with greater ease.

NCES may host one or two webinars annually, on an as-needed basis, to provide more information to respondents about the Survey, and to answer their questions.

The estimated hours per respondent are based on information directly provided by past state Program Directors who have completed comparable information requests in previous years. Assuming that the respondents (state education agency administrators) earn on average $47.5¹¹ per hour, the total annualized burden time cost to respondents for the SLDS Survey is estimated to be $6,652.

Table 1. Annual burden estimates for SLDS 2020-2022

Respondent	Number of Respondents	Number of Responses	Estimated hours per respondent	Total Respondent Burden Hours
States / U.S. Territories	56	56	2	112
Webinars	56	56	0.5	28
TOTAL	56	112	-	140

13. Estimates of Costs

SLDS Survey respondents will not incur any costs for this data collection other than their time to respond.

14. Annualized Cost to the Federal Government

Method for estimating costs: The full costs include the projected annual amount of time that Department of Education staff will spend on the survey, separated by labor type. Contracted staff is intended to support pre-collection, collection and analysis, with federal guidance and oversight. Salary costs are based on average FTE salaries and hourly rates for contract subject matter experts and program analysts.

Table 2. Estimated Annual Cost of the SLDS Survey to Federal Government for Fiscal Year 2020

Labor Type	Annual Labor Hours	Cost
Collection and Data Entry	386	$23,259.71
Data Analysis	540	$54,307.69
Product Development and Publishing	375	$31,944.71
Total	1301	$109,512.11

Cost: Department of Education staff assigned to SLDS Survey include one-third (FTE) of one Program Officer’s (GS13) time. Contracted staff also supporting the SLDS Survey (included in SLDS Contract total) consists of 20 percent of one SST and about 10 percent of one full-time Program Analysts employed through Applied Engineering Management Corporation.

15. Reasons for Changes in Response Burden and Costs

There is no change in response burden or annualized burden time costs. The change from a fillable PDF to a Google Form does not represent a significant change for respondents in terms of the estimated time of completion.

Collection and data entry costs have increased greatly to support the pre-population process, dissemination of Surveys, technical support and data transfer/entry in the instance of flat file response. As we are moving into our fourth year of collection, data is more abundant. Therefore, data analysis will begin to consider a more longitudinal study of SLDS capacity over time. This has yielded a dramatic increase in labor hours, as we plan more analysis that will ultimately be shared with the field through technical assistance and recurring Statistics in Brief documents. Product development and publishing experienced an increase in costs as we now know that the formal product resulting from this Survey is an annual, IES-published Statistics in Brief, which requires an extensive IES review and approval process prior to release. Finally, the development and implementation of the Google Form represents an additional increase in labor for this project.

The Federal Government will incur no additional cost for the implementation of this survey beyond the existing cost of managing the SLDS grant program and associated Technical Assistance contract (EDTAP contract number ED-IES-C-0015).

Time Schedule for SLDS Survey

The SLDS Survey is an annual collection. The proposed schedule is for the FY20 survey administration. This cycle will repeat in FY21 and FY22.

Table 3. Schedule and Timeline

2020 Timeline	SLDS Survey Collection, Processing, and Publication
October 2020	Email instructions to SEA respondents
October – November 2020	One or two webinars, on an as-needed basis, to provide more information about the Survey, how to complete the instrument on the new online platform (Google Form), NCES’s planned use of the data, and to address respondents’ questions about the Survey
November 15–30, 2020	Survey final reminder email
November 30, 2020	SEAs are urged to have finished submitting accurate and complete data
December 15, 2020	Mandatory final submission date
January 2021	Response by SEA’s to requests for clarification, reconciliation, or other inquiries from NCES. All data issues to be resolved. No files are accepted after close-out.
March 15, 2021	NCES review of files, file documentation, and brief analysis completed. Provisional responses available for internal use but not publication
July 15, 2021	Statistics in Brief and accompanying data file become public, NCES website updated. Current year collection data will be available to assess and respond to ad hoc requests
July 15 – October 2021	Respondents have the option to make update, change, or reconciliation requests to adjust state-specific data reflected in the SLDS Survey Statistics in Brief

NCES will release an IES-approved Statistics in Brief (and accompanying data file) associated with each annual SLDS Survey collection. Interested users and stakeholders will be enabled to quickly ascertain which States have capacity to link data across sectors, for example, which can link K12, postsecondary, and workforce data, and how they are using these data to inform policy and practice. Upon IES approval, the Statistics in Brief and data file will be published to the SLDS website. The SLDS grant program is currently responsible for providing updated indicators to OMB on a quarterly basis, so the validity of this reporting will be enhanced as a result of this collection. As data needs evolve and staff capacity permits, the intention is to post more data publicly. The data collected from the SLDS Survey will also be used to respond to questions from internal and external stakeholders regarding SLDS capacity in the States, and to inform future grant rounds and technical assistance planning.

17. Approval to not Display Expiration Date for OMB Approval

No approval is sought to not display the expiration date of OMB approval.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification for Paperwork Reduction Act submission.

1 The mean salary for financial managers (SOC code 113031) working in State government is $47.51 per hour. The Occupation and Employment Statistics at the U.S. Department of Labor, Bureau of Labor Statistics (BLS) were accessed on June 9, 2015 SOC code: Standard Occupational Classification code -- see http://www.bls.gov/soc/home.htm.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Carrie Clarady
File Modified	0000-00-00
File Created	2021-01-14

Part A SLDS Survey 2020-2022 (1)