Response to Public Comment Summary

1820-0624 Part B SPP/APR Comments and Discussion May 24, 2017

The information 1820-0624 Part B SPP/APR Comments and Discussion supersedes the information provided in the Explanation and Rationale document that was posted at the beginning of the 60 day comment period.

General

Comment: Several commenters expressed concerns that stakeholders are not meaningfully involved in the development of, and revisions to, the State Performance Plan/Annual Performance Report (SPP/APR), and requested that OSEP provide clear direction on stakeholder involvement.

Discussion: We believe that broad stakeholder involvement in

the development of the SPPs/APRs is very important and is a key component of developing and revising the SPP/APR. States are required to include information in the SPP/APR that describes mechanisms for soliciting broad stakeholder input on the State’s targets in the SPP/APR and any subsequent revisions that the State has made to those targets, and the development and implementation of Indicator 17, the State’s Systemic Improvement Plan (SSIP). The description is intended to be an actual reflection of the activities that have occurred, or will occur. We will continue to provide technical assistance on this topic to support States in meaningful stakeholder engagement.

Changes: None.

Comment: Several commenters expressed concerns with the SPP/APR module in GRADS360°, the online tool that States use to submit the SPP/APR. Commenters highlighted technical issues with data fields and the timing of the data prepopulation. Additionally, one commenter requested that the SPP/APR module be available for the State to enter data for the next year’s report much sooner than when it is currently available.

Discussion: OSEP appreciates ongoing feedback regarding the SPP/APR module and the willingness of States to work with OSEP as the SPP/APR module continues to evolve. We catalog all of the fields that are not functioning as intended and deploy fixes as soon as possible after the defect is identified. Similarly, we maintain a list of enhancements that we will deploy to better meet the needs and expectations of system users. Currently, data are populated at the moment the data are transferred to OSEP. We continue to work with our partners in the Department to explore options for transferring the data earlier. OSEP will also continue to work with its contractor partner to determine the earliest date by which the SPP/APR module could be made available for future year’s reporting.

Changes: None.

Comment: One commenter expressed support for maintaining the requirement in the current instructions, approved by OMB in 2012, that States must include an explanation of slippage in indicators where the State did not meet its target, noting that a State analyzing and understanding why there is slippage in meeting its targets is the first step in addressing the slippage, and reporting this information holds the State accountable for that first step in addressing the slippage.

Discussion: OSEP appreciates the comment and proposes slightly revising the current language in the General Instructions document to reflect the language in 2 CFR §200.328(b)(2)(ii) of the Uniform Administrative Requirements, Cost Principles), and Audit Requirements for Federal Awards (Uniform Guidance). This section requires, unless other collections are approved by OMB, that the State’s SPP/APR include brief information on the reasons why the State did not meet its established goals, i.e., targets, if appropriate. We believe that it is appropriate for the State to include the reasons for slippage in indicators where the State did not meet its target so the State can analyze and address the slippage.

Changes: The General Instructions document language has been revised to require that States’ SPPs/APRs include “the reasons for slippage” as we believe this more accurately captures the intent to conduct an initial analysis of the data for possible reasons for slippage.

General- Indicators 4, 9, 10, 11, 12, 13

Comment: Several commenters noted that the proposed instructions indicate that if a State's data for these indicators for the previous reporting period reflected less than 100% compliance, and the State did not identify any findings of noncompliance during the previous reporting period, States must explain why the State did not identify any findings of noncompliance during the previous reporting period. The commenters recommend that States be allowed to provide this information through a specific text field in GRADS360°.

Discussion: OSEP appreciates this feedback and will propose this recommendation as an enhancement to the SPP/APR module in GRADS360°.

Changes: None.

Comment: None.

Discussion: OSEP received a comment in the Part C SPP/APR information collection expressing concern about the level of clarity regarding the expectation that States report on the status of findings of noncompliance when the previous year’s data is below 100%. In response OSEP made edits to the measurement table language to provide additional clarification.

Changes: The measurement table instructions are revised to align with the Part C Measurement Table, reflecting the following language: “If the State reported less than 100% compliance for the previous reporting period (e.g. for the FFY 2016 SPP/APR, the data for FFY 2015), and the State did not identify any findings of noncompliance, provide an explanation of why the State did not identify any findings of noncompliance.”

Indicator 1

Comment: Several commenters expressed support for revising the SPP/APR measurement table to allow States the flexibility to report graduation data for children with disabilities using either the four-year adjusted cohort graduation rate (ACGR) or an extended-year ACGR under the ESEA, if the State has established one, and specifically supported the extended-year graduation rate because of its alignment with the ESEA, as amended by the ESSA. However, several other commenters argued that revising the SPP/APR measurement table to allow States this flexibility is inconsistent with ESEA, as amended by ESSA because the ESEA requires, at minimum, that States report the four-year ACGR. Under the ESEA, as amended by ESSA, States may also report the extended-year ACGR in addition to their four-year ACGR, if the State has established one. The commenters suggested that OSEP require that States report the four-year ACGR and permit States to report an extended-year ACGR.

Discussion: The requirement that States submit the four-year AGCR to the Department has not changed. This data is reported annually and made publicly available for all students and disaggregated for subgroups, including children with disabilities. However, in the SPP/APR, OSEP has given States the flexibility to reflect an extended rate that would capture additional students who graduate after four years of high school but who are within the age range for which the State makes a free appropriate public education (FAPE) available to children with disabilities. We believe that this flexibility is warranted in order to accurately reflect the total number of graduating youth with disabilities for States that elect to report an extended-year ACGR.

Changes: We appreciate the comments about aligning IDEA with the requirements related to graduation rate reporting in the ESEA, as amended by the ESSA and agree that it is important to make clear that annually reporting to the Department the four-year ACGR remains a requirement for States. As a result, we added a note under the SPP/APR measurement table instructions that emphasize this reporting requirement. However, the measurement table continues to allow States the flexibility to report either the four-year or extended year ACGR.

Indicator 2

Comment: Many commenters expressed significant concern that States will only be allowed to use a single measurement for this indicator – the same measurement that the Department uses in considering drop-out data as part of the annual determinations under section 616 of the IDEA (i.e., Option 1). Specifically, commenters noted that a majority of States and territories have reported data using the same data source and measurement that the State used to report in its FFY 2010 SPP/APR submitted on February 1, 2012 (Option 2), which would no longer be available, and therefore States will have to re-establish baselines, set new targets with stakeholder input and spend considerable time educating stakeholders about the change. Furthermore, Option 2 allows for the comparison of dropout rates for students with disabilities to their peers without disabilities and aligns more closely to other public reporting by States. One commenter supported the change but expressed concern with the timeline to report using a new methodology in time for the February 2018 SPP/APR submission. Two commenters supported the change with no additional comment.

Discussion: OSEP appreciates the detailed arguments submitted by States in support of keeping the flexibility of reporting using either Option 1 or Option 2 under Indicator 2. We understand the benefit of being able to compare dropout rates for children with disabilities to children without disabilities and other subgroups as well as being able to publicly report this information using one consistent measure. As such, OSEP proposes to continue to allow States to report using either option. Under Option 2, we note that it is possible that States have made small adjustments to their approach since FFY 2010 as they work to ensure that they are reporting more valid and reliable data. As such, if States have made or propose to make changes to the data source or measurement under Option 2, when compared to the information reported in its FFY 2010 SPP/APR submitted on February 1, 2012, the State should submit a justification as to why such changes are warranted.

Changes: We have added the language allowing States to use Option 2 back to the measurement table and also added language in the instructions noting that if the State has made or proposes to make changes to the data source or measurement under Option 2, when compared to the information reported in its FFY 2010 SPP/APR submitted on February 1, 2012, the State should include a justification as to why such changes are warranted.

Indicator 3

Comment: Several commenters recommended that we continue to require States to report proficiency rates for children with IEPs scoring at or above proficient against grade level and alternate academic achievement standards.

Discussion: We appreciate the commenters’ recommendations. We want to make clear that the reporting requirements related to annual assessments are not new. Under Indicator 3C, States are required to report the percentage of children scoring at or above proficient on the regular assessment with and without accommodations, and on the alternate assessment aligned with alternate academic achievement standards. OSEP will continue to require these data in the SPP/APR.

Changes: None.

Comment: Two commenters agreed with OSEP’s decision to eliminate the requirement to report on the percent of districts that meet the State’s adequate yearly progress (AYP)/annual measurable objective (AMO) target for the disability subgroup under SPP/APR Indicator 3A, emphasizing that the proposed changes are consistent with the ESEA, as amended by ESSA, which no longer requires States to demonstrate AYP or establish AMO targets. Several commenters suggested that OSEP maintain the Indicator 3A requirement, but align it with the requirement under Title I, Part A of the ESEA, as amended by the ESSA, that State and local report cards include information on the progress of all students and each subgroup, including children with disabilities, toward meeting State-defined long-term goals and measurements of interim progress. Rather than eliminating this requirement, the commenters recommended revising the requirement to have States report on the percent of districts that are meeting interim goals for performance on assessments for children with disabilities. Two commenters recommended that we require States and districts to report on the number and percentages of students with the most significant cognitive disabilities who take the alternate assessment aligned with alternate academic achievement standards by grade and subject, respectively, so that the SPP/APR reporting is aligned with State and local report card requirements under Title I, Part A of the ESEA, as amended by the ESSA. Finally, the commenters explicitly stated that these proposed changes would increase transparency in the SPP/APR.

Discussion: We appreciate the commenters’ suggestion to maintain indicator 3A. We also recognize the importance of analyzing multiple layers of data (including at the district and State level) and ensuring that the performance of students with the most significant cognitive disabilities taking the alternate assessment aligned with alternate academic achievement standards is included in the data reporting requirements. We note that under section 612(a)(16)(D) of the IDEA and 34 CFR §300.160(f), a State is required to report to the public with the same frequency and in the same detail as it reports on the assessment of nondisabled children the number of children with disabilities, if any, participating in alternate assessments based on alternate academic achievement standards and compared with the achievement of all children, including children with disabilities, the performance results of children with disabilities on regular assessments and alternate assessments based on grade-level alternate academic achievement standards. The measurement table instructions require States to include in their SPPs/APRs where to find public reports of assessment participation and performance results, as required by 34 CFR §300.160(f), i.e., a link to the Web site where these data are reported. Section 1111(h)(1)(C)(ii) and (2)(C) of the ESEA, as amended by ESSA, requires State and district report cards to include information on student achievement on required annual State assessments for all students, including children with disabilities. We note that under these ESEA requirements, while State and district report cards must include information on the performance of all children with disabilities, including children with the most significant cognitive disabilities who take the alternate assessment aligned with alternate academic achievement standards, States and districts are not required to report separately on the performance of children who take the alternate assessment aligned with alternate academic achievement standards.

Therefore, in an attempt to minimize burden on the State and avoid unnecessary duplication of effort, we do not believe that requiring States to report on the percent of districts that are meeting interim goals for performance on assessments for children with disabilities in the SPP/APR is necessary at this time. However, OSEP will continue to analyze the impact of ESSA implementation when considering future changes.

Changes: None.

Comment: None.

Discussion: As a part of its review of documents in response to comments, OSEP moved the requirement in the measurement to calculate 3B and 3C separately for reading and math outside the bracket of the numerator and denominator, for additional clarity. This change does not impact the reporting requirements in any way.

Changes: The measurement language for Indicator 3B now reads, “Participation rate percent = [(# of children with IEPs participating in an assessment) divided by the (total # of children with IEPs enrolled during the testing window)]. Calculate separately for reading and math. The participation rate is based on all children with IEPs, including both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year.” The measurement language for Indicator 3C now reads, “Proficiency rate percent = [(# of children with IEPs scoring at or above proficient against grade level and alternate academic achievement standards) divided by the (total # of children with IEPs who received a valid score and for whom a proficiency level was assigned)]. Calculate separately for reading and math. The proficiency rate includes both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year.”

Comment: None.

Discussion: As a part of its review of documents in response to comments, OSEP revised the instructions to clarify that proficiency calculations in the SPP/APR must result in proficiency rates for reading/language arts and mathematics assessments, rather than each content area across all ESEA assessments. This does not change any of the reporting requirements already in place.

Changes: The instructions specific to Indicator 3C in the measurement table now read, “Indicator 3C: Proficiency calculations in this SPP/APR must result in proficiency rates for reading/language arts and mathematics assessments (combining regular and alternate) for children with IEPs, in all grades assessed (3-8 and high school), including both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year. Only include children with disabilities who had an IEP at the time of testing.”

Indicator 4

Comment: Several commenters agree with the proposal to require States that choose to use a minimum n size for this indicator include only districts that meet the State’s minimum n size in both the numerator and denominator. Some commenters, however, disagreed with the proposal to require States that choose to use a minimum n size for this indicator include only districts that meet the State’s minimum n size in both the numerator and denominator. Although the commenters noted that the proposed change could be beneficial in establishing consistency across States, they asserted that using “all districts in the State” as the denominator when calculating significant discrepancy for this indicator is a better option for many States and they recommended that States continue to be allowed the flexibility to choose which districts are included in the denominator. One commenter that opposes the proposed change cited the fact that districts meeting the State-established n size can vary from year to year and the proposed requirement would result in States not being able to compare the rate of significant discrepancy over time. Additionally, the same commenter expressed specific concern that for some States, the proposed requirement could result in high number of districts being excluded from the denominator and affect the overall calculation for this indicator.

Discussion: We do not agree that allowing States to use “all districts in the State” as the denominator for this calculation is the better option. We believe that the requirement for States that choose to use a minimum n size for this indicator to include only districts that meet the State’s minimum n size in both the numerator and the denominator provides a more accurate reflection of the percentage of districts with a significant discrepancy in the rate of suspensions and expulsion for 4A, and a more accurate reflection of the percentage of districts with a significant discrepancy, by race or ethnicity, in the rate of suspensions and expulsions, which have policies, procedures, or practices in the specified areas that contribute to that discrepancy for 4B. Regarding one commenters concern regarding comparability of data over time, OSEP acknowledges that there may be variance from year to year in the number of districts that meet the State’s minimum n size. However, we do not believe that this variance would affect the calculation any more than the calculation may be affected by a yearly increase or decrease in the total number of districts in a State that is using “all districts in the State” as the denominator. Finally, we agree that for some States the proposed requirement could result in a high number of districts being excluded from the denominator. Therefore, when selecting a minimum n size requirement, States should consider the impact such a decision will have on their analyses of the rates of suspensions and expulsions of children with disabilities, including by race or ethnicity, by selecting an n size that maximizes accountability, minimizes privacy issues, and does not result in a high number of districts being excluded from the calculation.

Changes: None.

Comment: None.

Discussion: As part of its review of documents in response to comments, OSEP added language to Indicator 4B to clarify that, while States must ensure timely correction of noncompliance, States are only required to report on the correction of findings of noncompliance identified in the current reporting year in the next year’s SPP/APR. The language added is consistent with the language for other compliance indicators and does not change the requirements already in place.

Changes: Language for Indicator 4B correction of noncompliance now reads, “Provide detailed information about the timely correction of noncompliance as noted in OSEP’s response for the previous SPP/APR. If discrepancies occurred and the district with discrepancies had policies, procedures or practices that contributed to the significant discrepancy and that do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards, describe how the State ensured that such policies, procedures, and practices were revised to comply with applicable requirements consistent with the Office of Special Education Programs (OSEP) Memorandum 09-02, dated October 17, 2008. If the State did not ensure timely correction of the previous noncompliance, provide information on the extent to which noncompliance was subsequently corrected (more than one year after identification). In addition, provide information regarding the nature of any continuing noncompliance, improvement activities completed (e.g., review of policies and procedures, technical assistance, training, etc.) and any enforcement actions that were taken. If the State reported less than 100% compliance for the previous reporting period (e.g., for the FFY 2016 SPP/APR, the data for FFY 2015), and the State did not identify any findings of noncompliance, provide an explanation of why the State did not identify any findings of noncompliance.”

Indicator 5

Comment: While acknowledging that the data used in Indicator 5 correspond to data submitted to EDFacts according to file specifications, several commenters requested flexibility to split the count of five-year-olds between Indicator 5 and Indicator 6, depending on whether the five-year-old was enrolled in a preschool program or in kindergarten.

Discussion: The data source for this indicator is data collected under section 618 of the IDEA through EDFacts. The EDFacts Submission System technical guide for C089 – Children with Disabilities (IDEA) Early Childhood File Specifications (technical guide) provides the answers that are responsive to the comments. The technical guide states that five-year-olds who attend kindergarten are counted as attending a regular early childhood program. Therefore, OSEP declines to make the change as requested.

Changes: None.

Comment: None.

Discussion: As a part of its review of documents in response to comments, OSEP realized that we had inadvertently omitted the phrase “aged 6 through 21” in the description of the numerator in measures for Indicators 5A, 5B, and 5C.

Changes: The description of the numerators in the measures for Indicators 5A, 5B, and 5C have been revised to add “aged 6 through 21.”

Indicators 5 and 6

Comment: Several membership organizations expressed concerns that Indicators 5 and 6 are not appropriate measures for least restrictive environment (LRE). Commenters stated that accounting for the time spent in general education settings does not provide the information needed to ensure that students with disabilities are receiving a free appropriate public education in the LRE.

Discussion: IDEA section 616(a)(3)(A) specifically requires that the Department monitor SEAs, and SEAs monitor LEAs, using quantifiable indicators, and qualitative indicators as necessary, in the priority area of the provision of a FAPE in the LRE. Subsequent to the reauthorization of IDEA in 2004, the Department, with broad stakeholder input, developed Indicators 5 and 6 regarding FAPE in the LRE to meet this requirement. Indicator 5 measures the percent of children aged 6 through 21 who receive FAPE in the LRE. Indicator 6 measures the percent of children aged 3 through 5 who receive FAPE in the LRE. Under the LRE requirement in section 612(a)(5) of the IDEA, to the maximum extent appropriate, children with disabilities, including children in public or private institutions or other care facilities, are educated with children who are not disabled, and special classes, separate schooling, or other removal of children with disabilities from the regular educational environment occurs only when the nature or severity of the disability of a child is such that education in regular classes with the use of supplementary aids and services cannot be achieved satisfactorily. Therefore, we believe that time spent in general education settings is critical to measure whether a child is receiving FAPE in the LRE. Furthermore, these indicators are not the sole method OSEP uses in its monitoring. Results Driven Accountability, or RDA, OSEP’s accountability system, provides differentiated monitoring and support to States based on an analysis of risk factors (including data on compliance and results or outcome indicators, including data on least restrictive environment, as well as other factors). Therefore, OSEP will focus on FAPE and LRE in States where it has been determined that there is an increased risk that all students with disabilities are not receiving FAPE in the LRE.

Changes: None.

Indicator 6

Comment: None.

Discussion: As a part of its review of documents in response to comments, OSEP realized that we had inadvertently omitted the language in the instructions, “Describe the results of the calculations and compare the results to the target” that is present for other indicators. This is not a new requirement; it simply lays out the need to report data and compare actual performance against targets and (if needed) slippage.

Changes: The sentence, “Describe the results of the calculations and compare the results to the target” was added to ensure consistency with other indicators.

Indicator 7

Comment: One commenter appreciated the instruction that the measurement include, in the numerator and denominator, only children who received special education and related services for at least six months during the age span of three through five years. The commenter requested that OSEP define when the six month period would begin, and suggested that the six months begin on the date of the IEP since States already collect this data point and stated that this date signals the beginning of service delivery to preschoolers with disabilities.

Discussion: OSEP declines to define the six month start date because this is a decision best left to States. While the date of the IEP may be an appropriate start date in some States, other States may collect data on the actual start date of service delivery, which could differ from the date on which the IEP was signed.

Changes: None.

Indicator 8

Comment: While one advocacy organization expressed support, most commenters representing States did not support the inclusion of expanded instructions related to the State’s analysis of the extent to which the demographics of the parents responding are representative of the demographics of children receiving special education services. States were concerned that the reporting requirement imposes unnecessary burden, particularly if the State has demonstrated sound methodology in survey practices.

Discussion: The current measurement table requires States to report on the extent to which the response data (i.e., data regarding the demographics of the respondents) for this indicator are representative. As part of its response to a State’s SPP/APR, where the State has not addressed representativeness or has reported that the response data were not representative, the Department has required the State to report, as part of the next year’s SPP/APR, whether the following year’s data are from a response group representative of the population, and, if not, the actions the State is taking to address this issue. The instructions are revised to suggest, not require, that States consider providing more detailed information about the State’s effectiveness in collecting data that are representative of the demographics of children who receive special education services, including disaggregation by race and ethnicity, age of the student, disability category, and geographic location in the State.

In addition, the Department proposes requiring States that have not addressed representativeness or have reported that the response data were not representative to include strategies for improving the representativeness of the data in the current year’s SPP/APR, rather than next year’s SPP/APR, to increase the timeliness of the State’s development and implementation of such strategies.

Changes: None.

Indicator 9

Comment: None.

Discussion: As a part of its review of documents in response to comments, we determined that the instructions required revisions to make it clear that data for Indicator 9 should be reported for children aged 6 through 21, aggregated across all disability categories.

Changes: Language in instructions revised to read, “Provide racial/ethnic disproportionality data for all children aged 6 through 21 served under IDEA, aggregated. Provide these data for all children with disabilities across all disability categories.”

Indicators 9 and 10

Comment: Several commenters stated that the proposed measurement language removes the choice of using all districts in the State as the denominator. They further argued that while there may be some benefit in establishing consistency across States, the better option would be to use all districts in the State as the denominator rather than the number of districts that met the State-established minimum n and/or cell size.

Discussion: We do not agree that allowing States to use “all districts in State” as the denominator for this calculation is the better option. OSEP believes that the requirement for States that choose to use a minimum n and/or cell size for this indicator to include only districts that meet the State’s minimum n and/or cell size in both the numerator and the denominator provides a more accurate reflection of the percentage of districts with disproportionate representation of racial and ethnic groups in special education and related services that is the result of inappropriate identification for Indicator 9, and a more accurate reflection the percentage of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification for Indicator 10. Regarding one commenter’s concern regarding comparability of data over time, we acknowledge that there may be variance from year to year in the number of districts that meet the State’s minimum n and/or cell size. However, we do not believe that this variance would affect the calculation any more than the calculation may be affected by a yearly increase or decrease in the total number of districts in a State that is using “all districts in the State” as the denominator. Finally, we agree that for some States the proposed requirement could result in high number of districts being excluded from the denominator. Therefore, when selecting a minimum n and/or cell size requirement, States should consider the impact such a decision will have on their analyses of disproportionate representation of racial and ethnic groups in special education and in specific disability categories by selecting a minimum n and/or cell size that that maximizes accountability, minimizes privacy issues, and does not result in a high number of districts being excluded from the calculation.

Changes: None.

Comments: One commenter supported the proposed changes to Indicators 9 and 10, but encouraged the Department to work with States to ensure minimum n and/or cell sizes are not too large.

Discussion: We appreciate the commenters’ support for the proposed changes to Indicators 9 and 10. We recognize the need to work with States to ensure that minimum n and/or cell size, if established, are selected with consideration for maximizing accountability and minimizing privacy issues. We will continue to provide technical assistance and guidance, and encourage States to work with OSEP-funded TA providers as they select and implement any minimum n and/or cell sizes.

Changes: None.

Comment: None.

Discussion: As a part of its review of documents in response to comments, we determined that the following sentence should be removed from the Data Source and Measurement column: “If inappropriate identification is identified, report on corrective actions taken.” While States are required to address noncompliance, they are only required to report on actions taken to address noncompliance in the next year’s SPP/APR. This requirement is already included in the “Instructions for Indicators/Measurement” column.

Changes: The sentence, “If inappropriate identification is identified, report on corrective actions taken” was deleted from the Data Source and Measurement column.

Indicator 12

Comment: Two commenters expressed support for OSEP’s proposal to add a category for the number of children whose parents chose to continue early intervention services beyond the child’s third birthday through a State’s policy under 34 CFR §303.211 or a similar State option, which would then be subtracted from the denominator. One commenter from a State with an extended option policy disagreed with the proposed method to account for children continuing services under an extended option, explaining that both the current and proposed calculation do not accurately reflect the State’s outcomes for students with disabilities and is misleading to the public, as significantly more children are receiving services than are included in the numerator of the calculation as part of reporting on effective transition.  The commenter expressed that allowing children to continue under such an extended option protects the rights of parents and students, and that the State is being penalized for protecting these rights and choices. Furthermore, the commenter felt that excluding this group of students seems inconsistent with OSEP’s results-driven accountability system and does not accurately reflect the State’s outcomes for students with disabilities.  For these reasons, the commenter strongly encouraged OSEP to revise the proposed calculation to include the students in measurement f with those students in measurement c for those states with laws that allow parents to keep their children in the Part C program beyond their third birthday.  Another commenter expressed concern that additional data field requirement increases burden for States, and further stated that States would not be able to collect the information until the 2018-2019 school year. 

Discussion:  Regarding the concern for how States with an extended option policy under 34 CFR §303.211 or a similar State option are able to account for data under this indicator, OSEP appreciates the concerns expressed by the commenter. However, Indicator 12 measures a State’s compliance with section 612(a)(9) of the IDEA and 34 CFR §300.124(b), which require that a State have policies and procedures in place to ensure that by the third birthday of a child participating in an early intervention program under Part C, and who will participate in preschool programs under Part B, an IEP, or if consistent with §300.323(b) and section 636(d) of the Act, an IFSP has been developed and is being implemented for the child. Therefore, making Indicator 12 specific to implementation of the IEP by the child’s third birthday is a legal requirement, and only children for whom IEP implementation has occurred by this age can be counted in the numerator (measurement c).  Including children who continue to receive early intervention services beyond their third birthdays as children who “have an IEP developed and implemented by their third birthdays” is not consistent with required measurement in 12c.  While OSEP agrees that the numerator and denominator for this calculation will be smaller for States that back out children served under an extended option in measurement f, therefore increasing the impact of instances of noncompliance on the percentage the State reports under Indicator 12, States should still be able to ensure a high level of compliance for the children that are not continuing in the extended option, especially as there are fewer numbers of children for whom the State must ensure that an IEP is developed and implemented by their third birthdays.  OSEP supports States who implement an extended option through a State’s policy under 34 CFR §303.211 or a similar State option as well as parents’ right to exercise choice in such States.  It is OSEP’s position that requiring States to back out children who continue to receive early intervention services beyond a child’s third birthday through a State’s policy under 34 CFR §303.211 or a similar State option from their Indicator 12 calculation does not in any way limit a parent’s option to select to continue to have their child receive EI services in States where that option is available.  OSEP does not believe that this measurement is inconsistent with its Results-Driven Accountability System and we will continue to work collaboratively with States in implementing the different components of our system. Just as for other indicators, States are welcome to report additional information regarding the potential impact of State requirements on the data reported as part of their SPP/APR submission. 

With respect to burden and timelines for collecting data under measurement f, this is not a required reporting category. Rather, OSEP would only expect data to be reported in that category by States that have an established State policy under 34 CFR §303.211 or a similar State option.

Changes: None.

Indicator 14

Comment: Many commenters responded to OSEP’s request to receive feedback on whether the term “competitive employment” and its definition, as used in Indicator 14, should be revised to reflect the term “competitive integrated employment” and its definition, as set forth in Section 7(5) of the Rehabilitation Act, as amended by the Workforce Innovation and Opportunity Act, (WIOA) and its implementing regulation in 34 CFR §361.5(c)(9). Comments submitted by advocacy organizations generally supported adopting the WIOA definition of “competitive integrated employment” as the definition used in Indicator 14. Additionally, the commenters noted the importance of implementing a common definition and understanding of employment outcomes across Federal agencies and programs to enhance the coordination of services and supports for young adults with disabilities. Commenters representing States generally did not support aligning the Indicator 14 definition with the WIOA definition. Some commented that it would create a data collection burden, while others were concerned that adopting the new definition would actually lead to a decrease in an already low response rate because it would be complicated to succinctly explain the terms “competitive” and “integrated” to respondents. Another State wondered how States would verify if a job was truly integrated. However, one State indicated that aligning the definition would increase alignment with vocational rehabilitation programs and the State's goals to provide every student with the supports and services necessary to attain employment consistent with the definition in WIOA. Several commenters did not support aligning the Indicator 14 definition in WIOA, but made several requests if OSEP does align the definition. One commenter requested that OSEP remove the “20 hours a week” standard from the definition because the definition in WIOA does not impose such a standard. Another commenter requested that OSEP further study the issue and engage with its technical assistance centers to revise the measure to include the “competitive integrated employment definition.”

Discussion: OSEP is fully supportive of individuals with disabilities being employed in competitive, integrated positions and believes that ultimately adopting the WIOA definition of “competitive integrated employment” in Indicator 14 is critical to facilitate the cooperation and collaboration between the Vocational Rehabilitation program and State and local educational agencies in serving students with disabilities transitioning from school to post-school activities, including employment in the community. However, OSEP is sensitive to the increased burden, logistical challenges, and potential data quality issues associated with adopting the WIOA definition of “competitive integrated employment” for the FFY 2016 SPP/APR, due February 1, 2018. Therefore, OSEP will phase in the WIOA definition of “competitive integrated employment” in Indicator 14. Beginning with the FFY 2016 SPP/APR, due February 1, 2018, States will have two options for reporting “competitive employment” in Indicator 14. Option 1 is to use the same definition as used to report in the FFY 2015 SPP/APR, i.e., competitive employment means that youth have worked for pay at or above the minimum wage in a setting with others who are nondisabled for a period of 20 hours a week for at least 90 days at any time in the year since leaving high school. This includes military employment. Option 2 is to report in alignment with the term “competitive integrated employment” and its definition, in section 7(5) of the Rehabilitation Act, as amended by WIOA, and 34 CFR §361.5(c)(9). For the purpose of defining the rate of compensation for students working on a “part-time basis” under this category, OSEP maintains the standard of 20 hours a week for at least 90 days at any time in the year since leaving high school, in order to maintain consistency and rigor in the standard for what work can be considered part-time work. This definition applies to military employment as well. All States must report using the WIOA definition of “competitive integrated employment,” i.e., Option 2, beginning with the FFY 2018 SPP/APR, due February 3, 2020.

Changes: Indicator 14 has been revised to provide States with two options for reporting “competitive employment” in the FFY 2016 SPP/APR, due February 1, 2018, with the requirement that all States report using the WIOA definition of “competitive integrated employment” beginning with the FFY 2018 SPP/APR, due February 3, 2020.

Comment: One commenter provided several recommendations for revising Indicator 14, including requiring the addition of simple, specific data on hours worked and wages for individuals participating in competitive integrated employment, and ensuring that Indicator 14 results are addressed in the development of school improvement plans at the LEA level for underperforming districts, and that SEAs develop policies and resources for technical assistance to underperforming schools.

Discussion: OSEP appreciates the commenter’s recommendations for revising Indicator 14 and will consider these suggestions, as appropriate, when developing technical assistance related to this indicator.

Changes: None.

Comment: While one advocacy organization expressed support, most commenters representing States did not support the inclusion of expanded instructions related to the State’s analysis of the extent to which the response data are representative of the demographics of youth who are no longer in secondary school and had IEPs in effect at the time they left school. States were concerned that the reporting requirement imposed unnecessary burden, particularly if the State has demonstrated a sound data collection methodology.

Discussion: The current measurement table requires States to report on the extent to which the response data (i.e., data regarding the demographics of the respondents) for this indicator are representative. As part of its response to a State’s SPP/APR, where the State has not addressed representativeness or has reported that the response data were not representative, the Department has required the State to report, as part of the next year’s SPP/APR, whether the following year’s data are from a response group representative of the population, and, if not, the actions the State is taking to address this issue. OSEP proposes requiring States to include strategies for improving the representativeness of the data in the current year’s SPP/APR, rather than next year’s SPP/APR, to increase the timeliness of the State’s development and implementation of such strategies.

The instructions are revised to suggest, not require, that States consider providing more detailed information about the State’s effectiveness in collecting data that are representative of youth who are no longer in secondary school and had IEPs in effect at the time they left school, including disaggregation by race and ethnicity, disability category, and geographic location in the State.

Changes: None.

Indicator 17

Comment: One commenter requested that Indicator 17, the State Systemic Improvement Plan (SSIP), not be considered an “indicator.” The commenter stated that the SSIP is significantly different than the other indicators in structure and reporting requirements, and requires more resources to complete than the other indicators.

Discussion: We do not agree that the SSIP should not be included as an indicator because the structure and reporting requirements are not the same as the other SPP/APR indicators. Consistent with IDEA section 616(a)(3), the Secretary may use qualitative indicators, as necessary, to adequately measure performance. The SSIP, a key component of Results Driven Accountability, is a qualitative indicator that measures a State’s progress towards achieving improved results for children with disabilities.

Changes: None.

Comment: None.

Discussion: OSEP received several comments in the Part C SPP/APR information collection related to the SSIP (Indicator C11). In response, OSEP made edits to the Measurement Table that were also added to Indicator B17 in order to ensure consistency across the Part B and Part C SSIP requirements for Phase III.

Changes: The following sentence was added to the Phase III description: “Phase III should only include information from Phase I or Phase II if changes or revisions are being made by the State and/or if information previously required in Phase I or Phase II was not reported.” Additionally, under Section C of this indicator, OSEP is proposing that the language be revised to read: “The State must describe the specific strategies implemented to engage stakeholders in key improvement efforts and how the State addressed concerns, if any, raised by stakeholders through its engagement activities.”

Page 8 of 8