Designing and Implementing Results-Based Accountability and Support Systems

March 1, 2022

This post is adapted from the brief, Fast Five: Five Lessons Learned from States in Designing and Implementing Results-Based Accountability and Support Systems, produced by the National Center for Systemic Improvement (NCSI) and posted here with permission.

State special education general supervision systems—also known as systems of accountability and support—consist of the mechanisms by which state education agencies ensure district compliance with the Individuals with Disabilities Education Act (IDEA) and that students are provided a free and appropriate public education (FAPE). While state general supervision systems have resulted in high rates of compliance with IDEA requirements for a number of years, there has not been commensurate improvement in student outcomes, including academic achievement, graduation rates, or post-school outcomes. Many states are therefore working to reframe their accountability and support systems to focus increased energy, resources, and attention on the goal of improving student outcomes.

States are taking a variety of approaches to amplifying student results or outcomes in their accountability and support systems. For example, some states are seeking to hold local education agencies (LEAs) accountable for improving outcomes for students with disabilities in addition to maintaining compliance with the procedural requirements of IDEA, and they have chosen to incorporate results data (e.g., academic achievement, graduation rates) into their LEA determinations. Some states are including results or outcomes data in their risk assessments to select LEAs for monitoring and/or to determine the nature and focus of that monitoring.

Some states are incorporating results or outcomes data points into district risk assessments to differentiate monitoring and/or support. Several states are using results or outcomes data to target technical assistance (TA) activities focused on multitiered systems of support, effective data-driven decision-making, Universal Design for Learning, or specially designed instruction (as compared to trainings focused more on procedural requirements). Still other states are reimagining the onsite monitoring process to be more results focused.

As states have made these types of shifts to their general supervision systems and embraced a results-driven orientation, they have learned lessons along the way about what has worked well and what has been challenging. In the fall of 2020, the NCSI Results-Based Accountability Systems (RBAS) team interviewed five states that have implemented results-based accountability and support systems to learn more about their experiences and to ask what advice they would offer other states interested in creating similarly results-focused general supervision systems. What follows is a list of their five “lessons learned.”

We express our sincere gratitude to Colorado, Indiana, Nebraska, Oklahoma, and Vermont for sharing their insights with us.

1. Clearly message that the goal of the system is to improve outcomes.

The states we interviewed indicated that it was very important to the launch of their new, results-based accountability and support systems to message internally and externally that, first and foremost, the goal of the shift was to improve student outcomes. Nebraska described “constantly discussing and refocusing the conversations on improvement” and the need to reaffirm repeatedly to internal and external stakeholders that “improvement is our focus.” Another state simply stated that “the old system was not measuring the right things” and that districts had learned how to “get a good grade for compliance” but were not supporting students with disabilities to succeed and thrive.

In addition to emphasizing a focus on improved student outcomes, Nebraska also messaged internally and externally that the role of the state education agency was to partner with districts and support them in the improvement process and emphasized that the state would not be punitive in its approach. They characterized this as “implementing a mind shift” about the state role, which was historically focused on compliance. The state felt that a more partnership-oriented relationship with districts would afford them the “opportunity to better understand” a district’s story and therefore provide better, more responsive supports.

2. Engage stakeholders internal and external to the state agency to ensure a strong system design and increase buy-in.

The five states interviewed all expressed an unwavering commitment to meaningfully engaging stakeholders internal and external to the state agency in the development and implementation of their results-based accountability and support system. For example, Colorado convened a stakeholder group to help craft a new, results-driven approach to calculating LEA determinations. The state described this stakeholder group as having “great power in decision-making in the process.” The stakeholders created a “wish list” that included alignment with the state’s general education accountability system as well as growth measures.

The state incorporated both into their LEA determinations based on stakeholder input. Colorado also convenes their “RDA stakeholder group” on an ongoing basis to provide feedback on the system and make tweaks as needed. Indiana similarly reached out to stakeholders to help guide the overhaul of their LEA determinations process and named authentically engaging stakeholders as the most important piece of advice they would offer other states embarking on this work. Indiana began with a large-scale survey and asked respondents to rank results APR indicators by “importance and utility.”

They then convened their state chapter of CASE (Council of Administrators of Special Education) in a large stakeholder group and invited parent/ family representatives, educators, and regional TA providers to join as well. Vermont also engaged in a comprehensive stakeholder involvement effort, meeting with their State Advisory Panel (SAP) as well as stakeholder groups specifically convened to contribute to the design of their results-based accountability and support system.

Inviting stakeholders to the table ensures a diversity of perspectives will inform the development process, thereby strengthening the ultimate design. It also helps those in the field (e.g., district special education directors, State Advisory Panel members, family representatives, state-level TA providers) feel some ownership over the process and will, as a result, build buy-in when the system is rolled out. As Indiana offered, “You don’t want your constituency to feel like the state is coming down from on high. Rather, you want them to feel like they have a say in the system.”

3. Differentiate LEA accountability/monitoring and support activities.

Creating results-driven accountability and support systems requires differentiation to target limited state resources to the LEAs and issues within LEAs with the greatest need. All five states we interviewed described establishing a differentiated approach to either monitoring, technical assistance, or both. Colorado has incorporated a wide variety of results data points into their LEA determinations and are now considering how to differentiate support to districts based on the results.

Nebraska has created a results-driven risk assessment to target specific support to districts based on needs revealed in the data as well as to identify pervasive challenges that point to the need for statewide professional development. They continually evaluate and tweak the components of their risk assessment in order to accurately assess district need, ensuring that the risk assessment serves to differentiate between districts.

As they explained, “If data doesn’t differentiate districts, maybe it doesn’t need to be included.” Similarly, Indiana introduced a variety of results data points to a “technical support level” calculation for districts that helps the state differentiate the level of need among districts as well as the specific areas.

4. Intentionally deepen capacity at the state level to engage in results-based work.

The states interviewed acknowledged that results-focused work potentially requires new and different mindsets and skill sets among state-level staff. Indiana is working to build their staff capacity to help districts improve and has crafted protocols and resources to structure conversations staff have with districts around improving results. States shared the perception that supporting districts to improve student outcomes is a less straightforward undertaking and more time-consuming than training on how to meet compliance requirements.

One state said that helping districts improve results is less a “scripted” or “written-down” process than holding them accountable for compliance. Another state acknowledged that committing to a focus on results can lead to some “tough conversations” among state staff because “letting go of the status quo” (the state’s historical, singular focus on compliance) can be challenging. For that reason, states emphasized that part of their results-based accountability and support journey has involved building a strong culture of communication and collaboration among their team members in order to navigate the changes and expectations introduced by the new system.

5. Be flexible and adjust as needed… and expect this kind of practice and culture change to take time.

States emphasized that flexibility and adaptability are critical to this type of change process. Continuous improvement means constantly collecting data and making tweaks as needed. Indiana explained, “[We are] always evolving and thinking about what we can do better.” One state said, “You don’t have to have all the answers” when you begin. But they encouraged states interested in making a change to “be clear” in their goals and purpose and then just dive in, recognizing that the “first draft” of a new system will not be perfect.

States also suggested avoiding taking on too much at once. Vermont recommended “approaching the process in smaller chunks” and piloting the process with LEAs to gather feedback and make tweaks as necessary. States also pointed out that change takes time. Vermont is launching their new LEA determinations and monitoring process two years after initiating the change process. All the states interviewed indicated that it has taken a significant amount of time to go from the decision to make a change to designing what changes to make to their system, and then to implementing those changes.

To focus state accountability and support systems on improving results requires a culture and practice change at both the state and local levels. This kind of significant systems change requires trust, deliberation, intentionality, commitment, and teamwork — all of which require time to cultivate. As one state suggested, “It is important to plan for resources, including human resources, necessary to do work. Can we make this vision become a reality?”

The National Center for Systemic Improvement provides differentiated support through Universal, Targeted, and Intensive technical assistance to support SEAs to best use their general supervision and professional development systems to establish and meet high expectations for every student with a disability. Learn more.