National Center for Education Statistics
Volume I
Supporting Statement
2013-14 Civil Rights Data Collection (CRDC)
Improvement Study
Cognitive Interviews
OMB# 1850-0803 v.99
April 10, 2014
1. Submittal Related Information 3
2. Background and Study Rationale 3
5. Consultations Outside the Agency 5
7. Assurance of Confidentiality 5
Attachment A: Recruitment Materials 7
Attachment B: Advance Letter 9
This material is being submitted under the generic National Center for Education Statistics (NCES) clearance agreement (OMB #1850-0803), which provides for NCES to conduct various procedures (e.g. exploratory, cognitive, usability, or follow-up interviews; focus groups; feasibility studies; pilot tests) to develop, test, and improve its data collection instruments and methodologies.
The purpose of the U.S. Department of Education (ED) Civil Rights Data Collection (CRDC) is to obtain data related to the obligation of the nation's public school districts and elementary and secondary schools to provide equal education opportunity to their students. To fulfill this goal, the CRDC collects a variety of information, including student enrollment and education programs and services data that are disaggregated by race/ethnicity, sex, limited English proficiency, and disability. The CRDC is a longstanding and critical aspect of the overall enforcement and monitoring strategy used by the Department’s Office for Civil Rights (OCR) to ensure that recipients of the U.S. Department of Education’s federal financial assistance do not discriminate on the basis of race, color, national origin, sex, and disability. The CRDC data are also used by other ED offices as well as by policymakers and researchers outside of ED for research and statistical purposes.
The U.S. Department of Education has collected CRDC data on school characteristics, programs, services, and student outcomes directly from local education agencies (LEAs) on a biennial basis since 1968. For many years, the collection operated as the Elementary and Secondary School Civil Rights Compliance Report (OMB# 1870-0500). Since 2004, the CRDC has been conducted primarily online (with flat file submissions and paper surveys also allowed). With the exception of two SEAs, each participating LEA submits an LEA summary survey and a survey about each school in the LEA.
Feedback from prior CRDC administrations indicates that some LEAs experience difficulty responding to the CRDC. The issues documented fall into two categories: content and data collection tool. In the category of content, one of the issues is reporting burden. For example, respondents report that some of the requested data are already submitted to their SEAs (although these data items are not currently submitted by their SEAs to ED) and that other requested data are not maintained by schools or LEAs at the level of granularity required by the CRDC. Other content issues include a lack of clarity around definitions of key terms. Respondents also report that the current CRDC data collection tool has performance issues; in particular, respondents mention that there are not enough built-in edits and that some of the edit messages they receive are unclear. Further, analysts have raised concerns about the quality of some questions or items collected as part of the CRDC, especially the items collected for the first time by the 2009-10 CRDC.
In light of these and other concerns, the National Center for Education Statistics (NCES), within the U.S. Department of Education, has partnered with OCR to improve the CRDC data collection process. This is a continuation of the successful partnership within ED. Beginning with the 2004 CRDC, the CRDC has been a part of the ED initiative to better coordinate and consolidate data reporting from SEAs and LEAs across ED through a data collection known as EDFacts. Under EDFacts, the CRDC transitioned from being primarily a paper form-based data collection system to a primarily web-based collection system designed and operated by EDFacts as part of their data collection contract. The partnership between the CRDC and EDFacts has been a successful partnership that continues with the transition of EDFacts to the Administrative Data Division of NCES that took place in the fall of 2013.
Numerous NCES studies, including the Schools and Staffing Survey and the School Crime Survey, are planning to supplement their data collections with data from the CRDC. Therefore, in partnership with OCR, NCES is committed to redesigning all aspects of the CRDC data submission process and improve alignment of CRDC data with other surveys. In the initial stage, NCES is now working to develop a new data collection tool for the 2013-14 CRDC that will improve data quality, reduce burden on respondents, and improve the usability of the data collection tool and data feedbacks reports.
As part of the design of the new tool, we first sought to understand more about how LEAs access and house the data they need to report during the CRDC data collection period, the procedures they have in place for reporting data, and problems they face in meeting the CRDC request for information. Understanding of these areas will help us develop tools and procedures that better facilitate reporting and improve data quality. To research these issues, we conducted a process improvement and feasibility study that consisted of in-person site visits with schools, LEAs, SEAs, and OCR regional offices.
While site-visits allowed us to focus on issues of data access and reporting, there is a further need to identify specific data elements and definitions that are confusing for CRDC reporters when they are responding to the data request. To research CRDC responders’ understanding and interpretation of the CRDC data elements and definitions, we will conduct cognitive interviews with staff at 20 LEAs who have responded to or may respond in the future to the CRDC request.
The cognitive interviews will gather information on which data elements in the CRDC data request (e.g. athletics participation, non-personnel expenditures, security staff numbers) are confusing to respondents and what information can be added to instructions, definitions, tables, and questions to make the data request easier for respondents. The cognitive interview protocol will focus on specific data elements that NCES, OCR, and the site-visit research have found to be problematic for respondents.
Members of the study team will conduct a telephone interview with LEA staff who are involved in the reporting of CRDC data elements. The interview will focus on respondents who use the online submission tool. All interviews will be audio-recorded with respondents’ permission.
Recruitment
Sites will be recruited by NCES contractors, Sanametrix and AIR, in consultation with NCES. Recruitment will be conducted by phone calls or emails, depending on the information available, to superintendents, CRDC points-of-contact, or other key personnel. Since not all sites recruited will be able to participate in the cognitive interviews due to scheduling conflicts or time constraints, NCES anticipates contacting approximately 26 LEAs in order to recruit the target 20 LEAs. The recruitment process is expected to take 5-20 minutes per entity.
Individuals responsible for data collection and entry will be recruited from within each site. Depending on the LEA, one person or several individuals within the selected site may fulfill all of these responsibilities. We expect all participants to be on the call at the same time and each respondent will answer as needed depending on the questions being asked. If an LEA prefers to schedule separate interviews, we will accommodate this request. All recruitment communication will contain the following key information:
Purpose of the study
Study sponsorship
Importance of participation
Voluntary nature of participation
Estimated burden
Training
All interview staff will participate in interviewer training to ensure familiarity with and understanding of the purpose of the study, content of the protocols, and interview procedures. The study team will be trained in effective probing using objective and non-biased interview techniques and how to motivate respondents to provide accurate and complete information during interviews. The team will also be trained in key procedural issues such as guidelines for ensuring privacy during interviews, taking high-quality interview notes, and providing required follow-up communications.
Materials
The advance letter to be sent to participants, the CRDC draft tables and definitions, and the protocol to guide the interviews are provided in the attachments. The CRDC draft tables and definitions will be sent to respondents in advance of the interview.
The CRDC 2013-14 and 2015-16 proposed design and content was sent out for public comment, which was received in August 2013. Additionally, OCR maintains a CRDC working group, which currently consists of 12 LEA and SEA contacts from 12 different states: California, Louisiana, Massachusetts, New Jersey, Kansas, Missouri, North Carolina, Virginia, Ohio, Washington, and Wisconsin. Florida has also been a long-standing member of the working group, but is currently not part of the group due to a recent change in staffing. The working group provides feedback on design and content.
Additionally, in early 2014, NCES conducted 15 site visits to LEAs, SEAs, and schools to gather information about the CRDC submission process and data collection procedures. During these visits, problems related to the clarity of specific data elements were noted. These problems will be explored in detail in the cognitive interviews.
Respondents will not receive payments.
The study will ask about data holdings and processes of public institutions and will not collect personal data about respondents, therefore no confidentiality will be promised. Participation in the study is voluntary and the study recruitment phone calls and emails will notify the LEA of the study sponsorship and voluntary nature. Interviewers will also remind participants of these prior to the start of the interview, as noted in the interview protocol.
We expect to spend a total of 90-minutes interviewing staff at each LEA. We expect an LEA to also spend approximately 90-minutes reviewing the CRDC draft tables and definitions prior to the interview. Many LEAs use multiple staff to respond to the different CRDC modules. We will interview up to three LEA staff about different parts of the CRDC. For LEAs where three staff are interviewed, we expect the interview and prep time to last 30-minutes each. For LEAs where two staff are interviewed we expect the interview and prep time to last 45-minutes each. For LEAs where a single person is interviewed, we expect the interview and prep time to last 90-minutes each. For purposes of the burden estimation, we assumed about equal numbers of LEAs would have one, two, or three staff responding.
Entity Interviewed |
Estimated LEAs Contacted |
Estimated Recruitment Time (min) |
Estimated Number of LEAs Participating |
Estimated Number of Respondent Individuals |
Estimated Prep Time (min) |
Estimated Interview Time (min) |
Total Estimated Burden Time (hrs) |
LEA three respondents |
9 |
20 |
7 |
21 |
30 |
30 |
24 |
LEA two respondents |
8 |
20 |
6 |
12 |
45 |
45 |
21 |
LEA single respondent |
9 |
20 |
7 |
7 |
90 |
90 |
24 |
Total* |
26 |
n/a |
20 |
40 |
n/a |
n/a |
69 |
*The total number of responses in this study is 66.
There is no direct cost to respondents.
The researchers will prepare a summary that includes problems identified and suggested improvements that could be made to reduce burden and improve data quality.
Recruitment will begin upon OMB’s approval. The interviews will begin as soon as possible thereafter, and are expected to be completed by May 31, 2014.
The U.S. Department of Education is preparing for the next Civil Rights Data Collection (CRDC), which will collect data for the 2013-14 school year. As you know, the CRDC is a valuable source of information about access to education opportunities in our nation’s public schools.
For the upcoming 2013-14 CRDC, the National Center for Education Statistics (NCES) and the Office for Civil Rights (OCR) are collaborating to design a new submission tool to reduce the reporting burden on school districts and provide actionable reports to improve the quality of CRDC data. To accomplish this goal, we need active participation and feedback from school districts to guide our development efforts.
We hope your LEA will be willing to participate in a telephone interview to guide the Department’s improvements to the CRDC data collection process. Your input into the design of the new collection tool will ensure that future data collections are less burdensome and the resulting data are more helpful to you and other districts, states, schools, and data users.
During the interview, staff from the NCES contractors, Sanametrix and AIR, will ask you to go through the new proposed tables and data elements for the upcoming CRDC. They will ask you to let them know which areas are difficult or problematic to report and what definitions or terms are unclear. To gather input, we can speak to up to three individuals responsible for gathering data for the CRDC submission.
We expect it will take about 90-minutes per LEA to cover the information we need to ask about. If we speak to different individuals about different sections of the CRDC, the interview time per-person will be about 30 to 45 minutes. We will also send materials to review in advance of the interview. We expect it will take 90-minutes to review all of the materials, or 30-45 minutes per-person if multiple people review different parts of the materials. Participation in these interviews is voluntary.
If you agree to participate, someone from NCES CRDC team will be in touch with you to discuss scheduling an interview and to go over what to expect. Please feel free to contact, Abby Potts, at [email protected] or (202) 502-7566 with any questions or concerns.
Phone Script
Hello, my name is ____________. I work at the U.S. Department of Education and am calling in regards to the upcoming Civil Rights Data Collection, or CRDC, for the 2013-14 School Year.
[Alternative intro for contractors: I am calling from _____ on behalf of the National Center for Education Statistics in regards to the upcoming Civil Rights Data Collection for the 2013-14 school year.]
In preparation for the next CRDC, the National Center for Education Statistics (NCES) and the Office for Civil Rights (OCR) are collaborating to design a new submission tool to reduce the reporting burden on school districts.
In order for this new tool to be successful in improving the CRDC process, we need thoughtful and honest feedback from LEAs to serve as the foundation for the new system. We are recruiting a diverse group of LEAs to provide feedback on their experience reporting data for the CRDC.
I am calling to ask your LEA to participate in a telephone interview about the CRDC. Let me tell you about what’s involved and answer any questions you may have.
During the interview, staff from the NCES contractors, Sanametrix and AIR, will ask you to go through the new proposed tables and data elements for an upcoming CRDC, which we will provide to you in advance. They will ask you to let them know which areas are difficult or problematic to report and what definitions or terms are unclear. We can speak to up to three individuals responsible for gathering data for the CRDC submission. Your input into the design of the new collection tool will ensure that future data collections are less burdensome and the resulting data are more helpful to you and other districts, states, schools, and data users.
We expect it will take about 90-minutes per LEA to cover the information we need to ask about. If we speak to different individuals about different sections of the CRDC, the interview time per-person will be about 30 to 45 minutes. We will also send materials to review in advance of the interview. We expect it will take 90-minutes to review all of the materials, or 30-45 minutes per-person if multiple people review different parts of the materials. Participation in these interviews is voluntary.
Do you have any questions?
[Pause]
Would you be willing to participate in this interview?
[If yes] Fantastic. Thank you for agreeing. Your input will help ensure an improved reporting system for the 2013-14 CRDC. As the next step, I’ll have someone from the NCES CRDC team reach out to you to discuss scheduling options.
[If no] We know that there are many demands on your time. I completely understand. As always, you can reach out to any member of the CRDC team if you have questions about the upcoming collection or want to provide feedback to the team.
[on NCES letterhead]
[Recipient Name]
[Title]
[Company Name]
[Street Address]
[City, ST ZIP Code]
Dear [Recipient Name]:
Thank you for agreeing to assist us in the U.S. Department of Education’s efforts to improve the Civil Rights Data Collection (CRDC).For the upcoming 2013-14 CRDC, the National Center for Education Statistics (NCES) and the Office for Civil Rights (OCR) are collaborating to design a new submission tool to reduce the reporting burden on school districts and provide actionable reports to improve the quality of CRDC data
Your input into the design of the new collection tool will ensure that future data collections are less burdensome and the resulting data are more helpful to you and other districts, states, schools, and data users. We greatly appreciate your help.
Your interview is scheduled for [MONTH DAY, YEAR at TIME]. During the interview, staff from the NCES contractors, Sanametrix and AIR, will ask you to go through the new proposed tables and data elements for the upcoming CRDC. They will ask you to let them know which areas are difficult or problematic to report and what definitions or terms are unclear. To gather input, we can speak to up to three individuals responsible for gathering data for the CRDC submission.
The interview will take about 90-minutes per LEA to cover the information we need to ask about. If we speak to different individuals about different sections of the CRDC, the interview time per-person will be about 30 to 45 minutes. Accompanying this letter are materials to review in advance of the interview. We expect it will take 90-minutes to review all of the materials, or 30-45 minutes per-person if multiple people review different parts of the materials. Participation in these interviews is voluntary.
Someone from NCES CRDC team will be in touch with you to confirm your appointment and go over what to expect. Please feel free to contact, Abby Potts, at [email protected] or (202) 502-7566 with any questions or concerns.
Sincerely,
Ross Santy
Associate Commissioner
Administrative Data Division
U.S. Department of Education
Attachments: CRDC Draft Table Layout
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | andy |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |