Massachusetts uses its authorizing process and a system of regular site visits to ensure that charter schools are giving all students access to a high-quality education.

Many proponents of charter schools draw attention to their potential benefits for educationally disadvantaged students. As schools of choice, one argument goes, charter schools increase the educational opportunities available to students who might otherwise be confined to low-performing neighborhood schools. Moreover, given that charter schools are not subject to many of the regulations that apply to traditional public schools, advocates believe that charter schools are able to experiment and find new ways to offer higher-quality education to all the students they serve, including students from economically distressed families, students with disabilities, English learners, and students of color. But how can policy makers ensure that charter schools are doing right by these students?

One approach is to turn to the research on whether charter schools improve student outcomes on standardized tests (e.g., Center for Research on Educational Outcomes [CREDO], 2013, Gleason et al., 2010). That research generally finds that while some charter schools appear to succeed in raising student test scores above what they otherwise would be, many do not — and some have negative effects. The most positive findings are that charter schools in several urban areas, most notably Boston and New York City, appear to have strong positive effects on students’ academic outcomes (e.g., Angrist, Pathak, & Walters, 2013; Dobbie & Fryer, 2013; CREDO, 2015). Given that many such charter schools serve large proportions of students who have historically faced educational disadvantages, the findings imply that charter schools can, at least in some situations, generate better academic outcomes than traditional public schools for such students.

At the same time, studies that focus primarily on the academic outcomes of disadvantaged students already enrolled in charter schools miss other key issues, starting with the extent to which such students have fair access to individual charter schools in the first place. Fair access would require, at minimum, that charter schools recruit and retain student populations that are representative of the students in their local area, including those who have greater needs.

A second issue is the quality of the overall educational experiences charter schools provide to disadvantaged students once they enroll. Evaluating how well these students are treated requires information that goes beyond test scores to include such factors as student suspension and attrition rates, as well as the availability and quality of school programs specifically designed to support vulnerable students. While publicly reported data can shed light on some of these issues, such information is rarely enough to provide a full understanding of how the schools’ internal policies and practices affect these students.

One way for states to determine how well charter schools are serving disadvantaged students — and to nudge those schools to improve — is to implement an inspection-based public accountability system for charter schools that pays explicit attention to such students.

Public accountability and charter schools

As is the case for all publicly funded schools, public accountability is important for charter schools because it ensures taxpayer dollars are being used responsibly to promote student outcomes targeted by state or federal policy directives (such as the test scores required under No Child Left Behind) and to support the public interest in education, not merely the private interests of those being served. For several reasons, however, the special characteristics of charter schools demand that accountability for these schools be more expansive than for traditional public schools.

First, charters are schools of choice, which means that families need full and reliable information to make good decisions about them. Charter schools have a responsibility not only to provide general information for all families but also to address the specific needs of some  groups, such as non-English speakers who require information in their own language. Further, given that many parents, especially white parents, make decisions based in part on the composition of students in the school, external accountability may be needed to offset the pressures parental preferences can create for charter schools to be racially or economically segregated (Bifulco & Ladd, 2007; Ladd & Turaeva, 2020).

Second, charter schools are typically operated by non-governmental organizations, some of which may have motives and priorities that run counter to the public interest and, in particular, to the needs of certain kinds of students. For example, some charters may have financial incentives to minimize the enrollment of children with disabilities, who often require relatively costly services. Others, prioritizing an orderly school environment, may choose to suspend or expel misbehaving students, rather than making an effort to figure out why those students are misbehaving or to provide the kinds of support and instruction that would help them succeed. In addition, financial pressures may lead charter schools to hire inexperienced and less costly teachers, some of whom may be underprepared to work with students who face significant challenges.

Third, charter schools enjoy greater autonomy than regular public schools. Released from the requirement that all teachers be certified, for example, some charter schools may hire teachers who are not well trained to work with English learners or those with special needs. And in the absence of policies mandating that transportation or subsidized meals are available, some schools may not be  accessible to students who are economically disadvantaged.

Fourth, many charter schools have unique missions and are experimental by design. Their unique missions can enrich the offerings of public school systems as a whole, as well as of students whose needs are not currently met by traditional schools, but accountability is needed to ensure that the benefits these schools offer are balanced against students’ specific needs. While experimentation can lead to good outcomes, both for the individual charter school and for other public schools, it can also be harmful to students, especially to vulnerable students, if the experiment fails and schools need to close.

Finally, charter schools do not operate in a vacuum. In some situations, their presence can undermine the local community’s ability to serve students in other schools. This occurs whenever public funding for charter schools significantly reduces funding for traditional public schools or when charters enroll less than their fair shares of expensive-to-educate students, thereby leaving greater concentrations of such students in the traditional public schools (e.g., Bifulco & Reback, 2014; Ladd & Singleton, 2020).

The special characteristics of charter schools demand that accountability for these schools be more expansive than for traditional public schools.

Primary responsibility for public accountability of charter schools falls to the charter school authorizers. These are the state-sanctioned entities that license schools to operate, monitor their performance, and make periodic decisions about charter renewal (Finn, Manno, & Wright, 2017). In some states, the authorizers are part of traditional K-12 systems, such as state boards of education or local education authorities, but in many states they also include charter school boards, institutions of higher education, or nonprofit organizations. While six states, including Massachusetts, have a single statewide authorizer, a few states, such as California, Texas, and Wisconsin, have 100 or more individual authorizers (Ladd, 2019).

Although some authorizers do a good job and follow principles laid down by the National Association of Charter School Authorizers (NACSA), most are not affiliated with NACSA, and many do a poor job of overseeing the schools they authorize (Finn, Manno, & Wright, 2017). Poor authorizing and monitoring practices can lead to widespread fraud and abuse, badly managed and failing schools, and poor academic quality. Even strong advocates of charter schools have long understood that high-quality oversight and evaluation practices are needed to promote financially viable, well-managed, and effective schools (Finn, Manno, & Vanourek, 2000).

Massachusetts offers an excellent example of how a state can use its authorizing process, along with a system of site visits, to hold its charter schools accountable not only for basic educational aims but also for the additional goals of equitable access and fair treatment of students who face various disadvantages.

Charter authorization in Massachusetts

The Massachusetts State Board of Elementary and Secondary Education (or State Board) is the sole authorizer of two types of charter schools: a large number of Commonwealth Charter schools and a few Horace Mann schools. The Commonwealth schools are fully independent of local school boards and teacher unions, while Horace Mann schools operate under a Memorandum of Understanding with the district in which they are located. Both types of schools are subject to renewal every five years. Since 1994, when charters were first allowed in Massachusetts, the State Board has approved 112 charter schools, 31 of which either never opened or subsequently closed. As of 2019, there were 81 charter schools in the state, most of which are Commonwealth schools that operate independently of local districts, but with per-pupil funding based on a district’s overall budget for education.

The percent of education spending that can flow out of a district for the support of students in charter schools — and hence the number of charters that the state can approve within a local area — is strictly limited. Significantly, in 2010, with the goal of promoting charters that would serve needy students, the state raised this limit for districts with low-performing public schools and required charter operators to develop and gain approval for plans to recruit and retain students with high needs.

Once a charter school is established, it is subject to ongoing review in the form of periodic site visits carried out by teams from the State Department of Education, often supplemented by other outside educators or specialists. The visits, which last from half a day to two days, vary in purpose but generally involve gathering information about a school’s performance on some or all of the 10 Charter School Performance Criteria, depending on the type of visit. These criteria fall in three main categories: Faithfulness to the Charter Mission, Academic Program Success, and Organizational Viability.  Three of the criteria are particularly relevant to the goal of promoting equity: Criterion 2 (Access and Equity), Criterion 6 (Program Delivery) and Criterion 7 (School Climate and Family Engagement). The site reviews are modeled on the well-known inspection systems used for all schools in a number of other countries, including England, the Netherlands, and New Zealand (Ladd, 2010).

In preparation for the visits, schools provide a range of documents and records related to the purpose of the visit. During the visit, team members conduct focus groups and, during longer visits, observe classrooms, guided by the relevant site visit protocols. After the visit, the Department of Education prepares a written report that the school may formally respond to if it wishes. The process and reports are intended to draw attention to strong and weak aspects of the school but do not include specific advice about what the school must do moving forward or recommendations about renewal. The records from these visits also provide valuable insight into how well charter schools are providing equitable educational opportunities to all students.

Fair access to charter schools

A careful review of all the Massachusetts site visit reports for a few past years, supplemented with other statewide data, indicates that, in most counties, the average proportion of students of color in charter schools exceeds the average proportion of such students across the school districts within the county. Thus, if fair access for students of color is defined relative to enrollment patterns in traditional public schools, the state’s charter schools do very well. However, it turns out that many charter school students of color are being educated in more racially segregated school environments than they would be in traditional public schools.

An analysis of the 63 site visit reports during the past few years indicates that many, but not all, Massachusetts charter schools enroll their fair shares of specific groups of needy students (meaning that the proportion of such students in charters is at least 90% of the proportion of such students in comparable district schools). Specifically, two-thirds of the charter schools in Massachusetts enrolled their fair shares of economically disadvantaged students, and a somewhat higher proportion enrolled their fair shares of students with disabilities. However, in contrast, slightly less than half the charter schools enrolled their fair shares of English learners. Importantly, the site visit reports draw attention to factors that may limit a school’s enrollment of students with certain needs, such as limited translation of promotional and other materials into the various languages that are common in the local area, which would support the enrollment of English learners.

Equitable treatment of students

Enrolling students in charter schools is of little value if those schools are not able to provide an effective education. Determining how well any charter school serves its most vulnerable students requires attention to its internal school policies and practices. Site visitors rated schools on the quality of supports for students with additional needs, the quality of instruction, the safety of the school environment, and suspension rates. The site reports also provide information on the academic performance of these students, but careful interpretation of test scores requires more detailed causal analysis (e.g., Angrist et al., 2013; CREDO, 2015; and Dobbie & Fryer, 2013) than what was included in these reports.

Quality of supports and instruction

Of the 73 ratings related to school supports for diverse learners for fiscal years 2016 to 2019, 31 charter schools (43%) fell short of the state standard. The visitors’ reports often noted that schools lacked capacity to provide adequate supports or interventions for students who needed them. For example, schools lacked sufficient staff certified to teach English learners or students requiring special education services. Other reports noted the absence of systems, such as an English-as-a-second-language curriculum, well-designed tutorial programs, or appropriate accommodations for struggling students.

During that same period, reviewers rated 31 schools on the quality of their instruction. Of those schools, two-thirds fell short of the state’s standards. In more than half of the observed classes, the teachers failed to maintain high expectations for all students and to keep the students engaged. In some cases, the observers concluded that the classroom environments were not conducive to learning, perhaps because of poor classroom management or general disruptive behavior of the students.

Safety of the school environment

This indicator focuses on whether the school and classroom environments are safe, supportive, and culturally responsive; whether they support all students’ sense of belonging; whether they help students develop social and emotional skills, and whether the school has formal procedures to support students in crisis or at risk of dropping out. The specific qualities being evaluated under this indicator evolved over time, with ratings available for every school visit only in fiscal year 2019.

Three-quarters of the 20 schools subject to site visits in that year received favorable reviews on this standard, while only five were classified as not meeting the standard. Many of the problems noted among schools that did not meet the standard were related to school discipline. Students in these schools typically complained about excessive punishments for minor infractions, inconsistent application of rules, and more emphasis on handing out “demerits” than “merits.” The visitors found that students in these schools tended to feel that teachers did not respect them, especially when they came from very different backgrounds than their teachers. Several of these schools indicated that they were moving away from harsh disciplinary approaches and relying less on suspensions.

By contrast, schools that met the standard tended to have found ways to celebrate students of diverse identities through awards and support for affinity groups. They also tended to have formal structures in place to identify struggling students and to have policies favoring restorative justice.

Suspension rates

Many charter schools, including most of the charter schools in Boston that have received praise for their students’ strong academic performance, use a “no-excuses” approach to education. This approach features high expectations and strict disciplinary codes that often lead to high suspension rates. Although strict disciplinary codes have often been linked to high academic performance, a recent review of the research questions that relationship and also emphasizes that strict discipline may undermine non-academic outcomes, such as students’ social and behavioral skills (Golann & Torres, 2020).

In 2014, a new state law required the State Department of Elementary and Secondary Education to collect and report suspension rates for all public school districts. Not until the fall of 2018, however, did the state require that charter school site visit reports draw attention to high suspension rates. As a result of that change, starting in 2019, many charter schools were called out for having very high out-of-school suspension rates, especially for many of their disadvantaged students.

In sum, the site visit reports and other data reveal that, as a group, Massachusetts charter schools are doing quite well at providing fair access to all students. But, at the same time, there is substantial room for improvement with respect to ensuring students receive fair treatment once they enroll, particularly when it comes to providing high-quality support and instruction and avoiding punitive discipline.

The power of intentionality and firsthand observation

Equitable treatment of all students in charter schools does not happen automatically. Even if empirical studies show that some charter schools successfully raise student achievement, equity-oriented accountability policies are needed to focus attention on the broader issues of fair access and treatment. Thus, for charter schools as a group to benefit  students with the most needs, states will need intentional charter school policies that not only promote fair access for such students but also provide high-quality environments once they enroll. The Massachusetts approach, including its authorization policies and its system of periodic site visits that include a focus on equity, illustrates a promising approach to accomplishing this goal.

The inspectors do not tell the schools what they must do. Instead, the process is designed to make sure school officials understand the ways in which they are falling short and encourage them to make changes. The recent attention to high suspension rates, for example, is now putting pressure on charter school leaders to acknowledge the problem and find ways to reduce those rates.

In addition to providing feedback to charter schools themselves, the ongoing inspection reports provide state policy makers with useful qualitative information that is far more detailed than would be available without this type of accountability system. The site visit reports remain part of each school’s permanent record, and when charter schools come up for renewal, a summary report based on the renewal visit as well as previous site visits are used to determine whether the charter should be renewed, whether conditions should be placed on the renewal, or whether the school should be placed on probation.

We are not in a position to state definitively how much the equity-oriented accountability standards and the site visits themselves have promoted more equity and access among Massachusetts charter schools than would have emerged without them. But the mere fact that some charters do not meet the standards highlights the need for such procedures. With the insights provided in the site visit reports — information gleaned from a school’s annual report, school data, interviews, focus groups, and classroom observations — no school is able to ignore the fact that it is falling short of state expectations with respect to its most vulnerable students. Given the autonomy granted to charter schools, however, how a school responds to any shortfalls is ultimately up to the school itself.

Note: For a longer and more detailed look at the Massachusetts system, see our paper Equity-oriented accountability for charter schools: Lessons from Massachusetts, available as an Edworking paper of the Annenberg Institute of Brown University. Alison Bagg and her team at the Massachusetts Department of Elementary and Secondary Education provided information about the goals and details of charter school accountability in Massachusetts and access to site visit reports.

References

Angrist, J.D., Pathak, P.A., and Walters, C.R. (2013). Explaining charter school effectiveness. American Economic Journal: Applied Economics, 5, 1–27.

Bifulco, R. & Ladd, H. (2007). School choice, racial segregation, and test-score gaps: Evidence from North Carolina’s charter school program. Journal of Policy Analysis and Management, 26 (1), 31-56.

Bifulco, R. & Reback, R. (2014). Fiscal impacts of charter schools: Lessons from New York. Education Finance and Policy, 9 (1), 86-107.

Center for Research on Educational Outcomes. (2013). National charter school study. Stanford University.

Center for Research on Educational Outcomes. (2015). Urban charter school study. Stanford University.

Dobbie, W. & Fryer, R.G. (2013). Getting beneath the veil of effective schools: Evidence from New York City. American Economic Journal: Applied Economics, 5 (4), 28-60.

Finn, C.E., Jr., Manno, B.B., & Vanourek, G. (2000). Charter schools in action: Renewing public education. Princeton University Press.

Finn, C.E., Jr., Manno B.B., & Wright, B.I. (2017). Improve governance for charters. Phi Delta Kappan, 98 (6), 63-69.

Gleason, P., Clark, M., Tuttle, C.C., & Dwoyer, E. (2010). The evaluation of charter school impacts: Final report. National Center on Education and the Economy.

Golann, J.W. & Torres, A.C. (2020). Do no-excuses disciplinary practices promote success? Journal of Urban Affairs, 42 (4), 617-633.

Ladd, H.F (2010). Education inspectorate systems in New Zealand and the Netherlands., Education Finance and Policy, 5 (3), 378-392.

Ladd, H.F. (2019). How charter schools threaten the public interest, Journal of Policy Analysis and Management, 38, 1063-1071.

Ladd, H.F. & Singleton, J.D. (2020). The fiscal externalities of charter schools: Evidence from North Carolina. Education Finance and Policy, 15 (1), 191-208.

Ladd, H.F. & Turaeva, M. (2020). Charter schools and segregation in North Carolina (Working paper). National Center for the Study of Privatization in Education, Teachers College, Columbia University.