How can education research better inform policy debates?
Education policy research, as its name suggests, is intended to inform public policy decisions. In the 60 years since the field began, it has been highly successful in providing data and analysis on a wide range of issues — no small accomplishment. It has been far less successful, however, in actually contributing to enlightened policy choices.
At the core of the problem is the value-laden nature of the issues policy researchers address — school choice, standardized testing, integration and segregation, bilingual education, school funding, student discipline, tracking, affirmative action, and the list goes on. Indeed, virtually every one of these issues evokes strongly held value judgments that are much more compelling than any research findings. That basic psychological fact, and the political rhetoric fueling it, will not change.
What we can change, however, are practices that further limit the policy relevance of the research. First, we can challenge the academic rewards system that gives researchers strong incentives to choose topics and write for journals that are read almost exclusively by other researchers rather than by a broad audience of policy makers and educators.
Second, we can heed the familiar warning, “Not everything that counts can be counted,” and avoid topics that cannot be addressed by methodologically sound research. Consider, for instance, how much time has been spent comparing student test scores on international assessments of reading, mathematics, and science. The two major sponsors of these assessments have conducted such comparisons 14 times over the last 26 years (always at great expense and with great fanfare), despite the reality that it has never been feasible to obtain representative samples across dozens of countries with very different economies, education systems, and student enrollment rates. And without representative samples, the test-score comparisons are meaningless. Yet, the comparisons continue, as do policy makers’ endless promises that they will raise test scores (somehow, as if by magic) and boost their country’s rankings.
In calling for more research, we often appear to be stalling for time.
Finally, the most important thing we can do (when valid research findings are available) is to state our conclusions and describe the policy implications of our work. All too often, we insist that the research remains inconclusive and we are not yet ready to weigh in on the policy question at hand — even when the preponderance of evidence is clear. Consider the number of research reports that wrap up by noting the “need for more research.” Is more research truly necessary? Will a more sophisticated statistical technique, a larger sample size, or yet another comparison of student test scores resolve any inconsistencies in our findings? Or are inconsistencies inevitable in policy research, given the enormous variation in educational, social, and economic conditions among the schools, districts, and countries we study? In calling for more research, we often appear to be stalling for time, in hopes of reaching an illusory milestone where ambiguity disappears and it becomes crystal clear which kind of instruction, or which school model, or which reform is most effective.
Take, for example, research on charter schools. We have learned from numerous attempts to compare the performance of charter and traditional schools that in some comparisons, charter schools have the edge and, in others, traditional schools come out ahead. In many cases, there is no significant difference and, on average, the academic outcomes, as measured by standardized test scores, turn out to be very similar. Nor, as Mark Berends suggests in the September 2021 Kappan, can such “horse race” research ever be conclusive, given that “schools of choice vary in their quality, design, and many other factors.” We cannot declare charter or traditional schools the winner when it is impossible to say what, exactly, we are comparing.
Value judgments will continue to compete with research findings (and researchers are not immune).
But research does tell us that charter schools often contribute to increased segregation, increased competition for scarce resources, the weakening of traditional public schools, decreased public participation in setting education policies, and increased differences between school districts serving large proportions of low-income and minority students and those serving a more diverse, and often more affluent, student population. These studies, like the test-score comparisons, have not been entirely consistent, given the significant differences among schools and priorities within both the charter and traditional school sectors. The existence of some inconsistency, however, should not detract from the bulk of the evidence on the impact of charter schools on a whole range of important matters in communities throughout the country.
Like research on value-laden issues generally, these findings will be interpreted through the lens of beliefs — in this case, beliefs about the merits of school choice, the utility of standardized test scores, the risks of segregation, the optimum allocation of resources, the benefits of traditional public schools and school boards, and the impact of privatization. Value judgments will continue to compete with research findings (and researchers are not immune).
Researchers, nonetheless, can play an important role by focusing on issues that are relevant to the problems faced by educators and policy makers, by conducting research that has a realistic chance of producing valid results, and by clearly communicating conclusions and policy implications that are supported by the research. A wealth of available findings has not yet been communicated to the audiences policy research was intended to serve. On many topics, we do need more research. But before initiating yet another comparison that will provide another inconclusive answer to a question that may never be settled, let’s take a careful look at the findings we do have and determine where the preponderance of evidence supports a conclusion that a specific policy does, or does not, merit our support.
This article appears in the October 2021 issue of Kappan, Vol, 103, No. 2.
ABOUT THE AUTHOR
Iris C. Rotberg
Iris C. Rotberg is a research professor of education policy at the Graduate School of Education and Human Development at The George Washington University, Washington, DC. She is the editor, with Joshua L. Glazer, of Choosing Charters: Better Schools or More Segregation?
