Creating state accountability systems that help schools improve 

The learning process for every school, like every child, is a personalized journey of continuous improvement.  

 

The nation can establish a system of educational accountability that helps lift the performance of every learner, teacher, leader, and community. But this will only happen if states choose to shed approaches common to the No Child Left Behind (NCLB) era that created a compliance culture of blame and an inequitable system of winners and losers.  

The Every Student Succeeds Act (ESSA) provides an unprecedented opportunity for states to create a new framework for accountability that has continuous improvement at its core and local context as its foundation. Unlike NCLB, ESSA focuses more on improvement than punishment. It empowers policy makers to look critically at all aspects of school that influence learning. ESSA requires reporting to the public on outcomes and opportunities to learn for all students, including per-pupil expenditures, access to rigorous coursework, and measures of school climate. The law returns power to the states to determine what accountability should look like. Each state must establish its own statewide accountability system and related school support and improvement activities by the 2017-18 school year. Because ESSA offers states flexibility without imposing set structures, absent deliberate and intentional action, we still could wind up where we began: overemphasizing test scores and choosing indicators based on what is easiest to measure.   

This country needs an accountability system that examines all aspects of what schools do, reveals root causes of underperformance, and reflects the relationships between the strategies or actions that are implemented and the results they achieve (or fail to achieve). By developing information systems and feedback structures that identify strengths and weaknesses within schools and districts, states can set the stage not only for identifying what is working but also for changing educator practice where it matters most — at the school and classroom levels. These goals are at the heart of the continuous improvement approach. 

Accountability for continuous improvement 

Organizational leaders from nearly every sector have been using continuous improvement models and improvement science for years to improve products, services, and processes. These efforts gain power — and greater efficiencies and improved performance — through the ongoing and continuous examination of performance, problem identification, design change, and ongoing review that are all key components of the continuous improvement cycle.  

Though continuous improvement processes are not new in education, they are relatively new in the state policy arena. In the past, federal and state requirements drove state-level strategies for improving educational outcomes at the school and system levels centered largely on developing one or more annual “improvement plans” to chart action and investment. Statewide systems of school improvement and support were focused largely on compliance and sanctions based on bald, end-of-year data that provided little evidence of how results were achieved; moreover, they were not useful to school and system leaders charged with making decisions. Plans were typically long documents with a focus on compliance and a goal structure with unrealistically short timelines. 

This country needs an accountability system that examines all aspects of what schools do, reveals root causes of underperformance, and reflects the relationships between the strategies or actions that are implemented and the results they achieve (or fail to achieve). 

In a continuous improvement system, educators use data, test scores, and outcomes as evidence of performance but not as goals for the system or the main drivers of accountability. That is because the information is not actionable as it does not provide adequate evidence required to make improvements. Moreover, when benchmarks become goals, states can too easily either change the targets or cut scores for success and then declare victory without any meaningful change in the system.  

Such a system changes reporting from a compliance activity to a process that enables positive change at a local level. All districts and schools should receive comprehensive feedback, which lets them act on what they learn. Positive results reaffirm for schools what they are doing well and enable educators to build on their good work and do better. Negative feedback based on real evidence makes it easier for superintendents and principals to justify the need for improvement in those areas and push leadership to provide more consistency, clarity, and coherence around school improvement.  

Reporting on and analyzing a broad range of indicators can help decide where to reallocate resources, such as support for new teachers and professional development, in ways that make a difference. These actions enable districts to update their processes and procedures to ensure better coherence across the system, help formalize baseline expectations around key processes, and force districts and their schools to pay more attention to every aspect of what they do. 

The continuous improvement process not only shows educators where their schools stand but also empowers them to move forward. All schools can improve, including those whose high test scores mask complacency. Continuous improvement also addresses what happens to a school or district that is performing at a high level but begins to drift. In these cases, we often discover that student engagement has plummeted, achievement has tailed off, teacher morale has declined, or leadership has faltered.  

Continuous improvement does not just begin and end once a year. It helps schools take continuous action as enrollments grow and shrink, teachers come and go, curricula change, technology evolves, and educational standards rise. Any or all of those factors may demand that a school or district adjust and rethink policies and practices that worked so well in the past. 

In a continuously improving system, schools have a consistent way of measuring success that provides the ability to cross-reference and cross-validate evidence to draw a realistic picture — a 360-degree view or CT scan to operate on the system. With that information, leaders can more clearly see what needs to happen to obtain goals. 

Creating a next-generation accountability system 

To create an accountability system that sparks student and school improvement, states need to move beyond limited or arbitrary measures of student learning to rate schools and impose punitive measures. To ensure that accountability doesn’t impede improvement, each state needs to find the right mix of measures that come together to tell a holistic story about how schools and their students are performing — and those measures must provide enough meaningful information to help states plan and implement appropriate and targeted supports.  

In a continuously improving system, key success criteria help propel progress. A system is considered effective when:   

• Various processes and components of the system are connected and aligned so they work together as part of a complex whole in support of a common purpose.
• System improvements are driven by a process of continuous measurement and feedback with a focus on collecting and sharing data that informs and transforms.
• System actors understand and engage each other and the system successfully.
• System outputs are of the desired quality and produced within the desired time frame.

When these conditions are in place, the educational system can address its core challenges and:

1. Acknowledge and adapt to the realities, complexities, and uniqueness of schooling.  
2. Employ a systemic approach to actively assess, monitor, and improve at all levels with regard to key education factors and high-quality standards. 
3. Ensure a holistic understanding of education quality through reliable data and myriad pieces of information,and use these data to continuously drive and evaluate improvement actions and support services. 
4. Provide transparency and accountability through a valid multimetric, nonpunitive representation of data and information.  
5. Identify, acknowledge, and engage all stakeholders.  
6. Provide constant and consistent reinforcement, guidance, and accountability of all stakeholders and factors toward a shared vision. 

Going beyond ‘NCLB-Lite’ 

In the new ESSA policy environment, some state education agencies will be tempted to do the bare minimum to meet the new requirements, continuing the same testing regime and adding a survey or two here and there to gauge what’s happening in schools. Some states will add a few extra measures but still not go beyond a compliance checklist with some additional accountability. These state education agencies  may (or may not) earn federal approval, but they are unlikely to improve results. Instead of modifying or improving existing systems, states should take this opportunity to acknowledge lessons learned, wipe the slate clean, and envision a forward-thinking education system.  

States must focus their attention on four key areas: 

#1. Establish a clear vision of the purpose and direction of schooling. 

This is a crucial point of reference for innovation and decision making as the demands and complexities of the system evolve. 

#2. Identify appropriate measures and broaden the kinds of information being gathered. 

In considering school quality factors, the so-called nonacademic factors — such as school culture, resource allocation, and student engagement — are as important as proficiency on tests and graduation rates. In fact, we need information about a multitude of internal and external factors and how they have changed over time. This information can be compared with industry standards or other similar entities as a means of making judgments. 

#3. Identify new ways to support low-performing schools. 

States need to find more effective ways to identify what’s happening in these schools before they can develop more effective solutions. In recent years, Kentucky has instituted a diagnostic review process that considers all aspects of school performance that influence learning. Every low-performing school undergoes external evaluation with teams of experts who visit the school and monitor performance and develop improvement plans based on actual needs identified in the evaluation.  

#4. Introduce formative assessments to allow for corrective student and school-level actions over time.

 State education agencies should look at testing not as a one-time event but as a means to look at student achievement over a multiyear period. Interim assessments offer real-time data that can drive supports and ultimately affect results and growth. 

Additional benefits 

Encouraging and supporting districts and schools to own the evidence for their success creates an opportunity for states to create a self-innovating system that helps schools become what we aspire for them to be — dynamic learning centers that help all talent rise regardless of where learners start. Changing this process also can shift the role of the state education agency. As the state education agency builds an informative feedback system that helps educators and local leaders act on what they learn to improve their schools, the state will no longer be relegated to the role of good cop or bad cop (and parole officer) when working with districts but instead will become a true agent for improvement, providing needed technical assistance, coaching, and monitoring.  

While ESSA gives greater authority to states, schools will improve only  if states pass some of their power to districts and schools to make needed changes in their buildings. The change begins with loosening the grip that single measures have on our education systems. Just as we encourage students to provide multiple points of evidence to back up their conclusions, we must ensure that schools have access to multiple sources of relevant data needed to make decisions and sustain progress.  

For more than 15 years, America has tried rigid systems of high-stakes testing to encourage and demand improvements in school quality, teacher effectiveness, and student learning. Such systems have fallen woefully short of achieving desired improvements in these areas. If we are committed to every child’s success, we must create accountability systems that guide, support, and ensure continuous improvement. The learning process for every school, like every child, is a personalized journey of continuous improvement.  

References 

Darling-Hammond, L., Wilhoit, G., & Pittenger, L. (2014). Accountability for college- and career-readiness: Developing a new paradigm. Stanford, CA: Stanford Center for Opportunity Policy in Education. https://edpolicy.stanford.edu/publications/pubs/1257 

O’Day, J.A. & Smith, M.S. (2016). Quality and equity in American education: Systemic problems, systemic solutions. In H. Braun & I. Kirsch (Eds.), The dynamics of opportunity in America.  New York, NY: Springer. 

 

Citation: Elgart, M.A. (2016). Creating state accountability systems that help schools improve.  Phi Delta Kappan, 98 (1), 26-30. 

---

How does continuous improvement differ from typical approaches to accountability? 

Continuous improvement differs in four crucial ways from typical approaches to accountability used by state education systems. According to Jennifer O’Day of the American Institutes of Research and Marshall Smith, a senior fellow at the Carnegie Foundation for the Advancement of Teaching (2016), continuous improvement: 

#1. Focuses on root causes not just outcomes.

Rather than focus exclusively on collecting and analyzing data on student outcomes without information about what happens in the system to produce those outcomes, continuous improvement provides detailed information about particular practices to identify important connections between actions and results. By understanding root causes, educators can assess the true effect of their actions in order to change what they do to alter outcomes. 

#2. Sees failure as a means to improve not a reason to assign blame or sanctions.

Rather than seeing failure as an opportunity for blame and negative consequences, continuous improvement uses failure as a means to identify needed assistance and learning. In continuous improvement, Smith and O’Day note, “Mistakes and failures are expected; they are both the basis for identifying the focal problem of practice and are opportunities for collective learning about how to make things better” (2016, p. 317).  

#3. Enables informed decision making based on rich context and evidence. 

Rather than mandate solutions about what should be done when something fails without considering what caused the problem or the strength of the evidence, continuous improvement approaches enable educators to make decisions based on context so participants understand which solutions are likely to work for whom and under what conditions. 

#4. Places the source of accountability and decisions about action for improvement within the system. 

Rather than placing the source of accountability far from the district and school and removing local actors from setting goals and identifying solutions to problems, the main source of accountability in a continuous improvement approach resides within the system — with key players within the organization focused on the practices and feedback loops they have put in place. 

---

California’s accountability system shows the way 

California’s accountability system is built around many of the key principles described in this article. California combines multiple measures of performance and a comprehensive framework for technical assistance/intervention with a broad use of data to identify the successes of its demographic subgroups. California allocates all funds based on pupil needs and weights funding according to poverty, language status, and foster care status. 

The state requires districts to provide, develop, and adopt — with parent and community involvement — an accountability plan that identifies goals and measures of progress across indicators of opportunities and outcomes. Local districts can add indicators to those required by the state. Indicators range from student achievement, persistence, and graduation to student inclusion, college- and career-readiness, the availability of quality teachers and adequate facilities and evidence of parent participation and opportunities for input (Darling-Hammond, Wilhoit, & Pittenger, 2014). 

In addition, 10 California school districts that together educate more than 1 million students, or one-fifth of California’s entire student population, have added additional measures of social and emotional learning and school climate that include chronic absenteeism, disproportionate special education identification, and student climate surveys. 

---

Accountability evidence required for continuous improvement 

An accountability system designed to have sustained effect begins by ensuring it can gather information about the factors that lead to success or are impeding improvement efforts. Such a system ensures that educators and policy makers can answer crucial questions about performance so school leaders can scale up successful strategies and modify or abandon failing approaches.  

For example, how engaged are students in their learning? What is the informal culture and climate of the school, and how does it affect student learning and engagement? What are school leaders doing to build the morale of students, teachers, and staff? How well are school initiatives being implemented? Are system resources equitably distributed? How active are parents in school governance? What is the school’s reputation in the larger community? 

Achievement tests and graduation rates cannot answer such questions. However, those outcomes speak volumes about a school’s performance and its capacity to empower and engage the entire school community, and they invite the analysis and discussion that is absolutely critical to the success of any school improvement effort.  

To deepen understanding of what is happening in schools, districts, state education agencies, and schools need to gather and be conversant about the crucial information across four areas:  

• School culture, which helps ascertain the school’s underlying expectations, norms, and values; the depth of student engagement and efficacy; teacher and stakeholder engagement; school leadership; the quality of the learning environment; and school safety and student well-being.
• Talent, which helps determine the school’s ability to recruit, induct, support, and retain staff and provide opportunities for professional learning and growth. These are often the first things cut by districts.
• Execution, which identifies how well schools are implementing plans and strategies; the fidelity to a course of action; what school leaders, superintendents, and state education agencies do to ensure plans and strategies are producing results, including monitoring and adapting strategies to consistently improve; andhow effectively districts and schools allocate resources to meet identified needs and achieve desired results.
• Knowledge, which determines what schools and districts do with the wealth of data that they have and whether they actually are using it to improve results; whether the information is used broadly and shared; and the extent to which it is useful for improvement. Schools also need to determine when information can be made available, as continuous formative information can ensure that educators can improve their practice, tailor learning for individual students and address gaps in student skills and knowledge in real time.

Understanding success in each of these areas requires a rich store of local data — the kind of context that does not show up on test scores. Schools, states, and districts that do this well use simple tools, surveys, and observation protocols, and they implement basic information-gathering systems that they find are well worth the investment of time and resources.