When examining data on student performance, be careful of ways it can be used to reinforce bias rather than improve instruction. 

After reviewing disappointing assessment results for 7th graders at his middle school, Dennis, an English language arts teacher, explained the scores in some of the classes by pointing to emergent bilingual students and students with disabilities: “[Those classes] have literal learners [who] have trouble discerning between two answers. . . .  It’s just hard for those kinds of kids.” (All names used in this article are pseudonyms.) Dennis’ deficit-laden explanation of the scores didn’t include an explanation of what he meant by “literal learners,” but we can surmise that he viewed “those kinds of kids” as having difficulty with abstract and metaphorical thought. In essence, he suggested that it was natural for emergent bilingual students and students with disabilities to get low scores. 

From 2011 to 2012, we conducted a study of professional practices in six middle schools (including the one where Dennis taught) across four districts (three medium-size suburban and urban districts in one western state and a fourth larger district serving families in rural, suburban, and urban areas in a southern state) that were engaged in efforts to get teachers to increase their use of data to inform their instructional decisions (Bertrand & Marsh, 2015). During our observations, we often heard comments that were similar to the one Dennis made. Time and again, when reviewing test scores and other data, teachers would refer to the supposedly inherent deficits of emergent bilingual students, students with disabilities, and other student populations. Instead of reflecting on their own classroom instruction or asking what the school could do to support those children more effectively, many teachers were quick to conclude that the children were themselves to blame for their performance, by virtue of their categorization.  

Data in the context of systemic racism 

In this era of high-stakes testing and accountability, school and district leaders have touted data-driven decision making as a powerful means of improving teaching and learning and achieving more equitable outcomes for all students. In theory, the more teachers know about their students’ test scores, grades, attendance rates, and other aspects of their performance, the better they can adapt their instruction to meet those students’ needs (Dodman et al., 2019; Marsh & Farrell, 2015).  

Indeed, evidence suggests that such data use does sometimes promote better and more equitable outcomes, especially when educators start from the premise that all students can succeed and that the purpose of analyzing data is to find ways to help them do so (Datnow & Park, 2018; Park, 2018). However, data-driven decision making can also perpetuate inequities, especially when educators use data in color-neutral ways (Roegman et al., 2018) that confirm assumptions about students or their families (Datnow & Park, 2018; Marsh & Kennedy, 2020), disregard students’ cultural identities (Garner, Thorne, & Horn, 2017), or reinforce harmful tracking practices (Park & Datnow, 2017). 

The failure of data-driven decision making to live up to its promise is not surprising. Like many high-stakes accountability policies, data-use initiatives ignore the unjust context that shapes student outcomes (Dodman et al., 2019; Milner, 2012). Students live in a world where systemic racism and white supremacy fuel inequities in access to educational opportunities (Carter & Welner, 2013) and lead to disproportionately harsh discipline for Black students, Indigenous students, and other minoritized students in comparison to white students (Carter et al., 2016; Losen et al., 2015; Whitford, 2017). In addition, white supremacy and systemic racism are at play when educators espouse deficit thinking about Black, Indigenous, and other students of color, placing the blame for inequitable outcomes on the cultures of the students and their families and communities instead of on structural inequities (Valencia, 2010). Some educators believe that, if students of color are earning low grades, it must be due to their culture, language background, or the influence of their families and communities. And, in turn, this deficit thinking — which is especially prevalent among white teachers (Fox, 2016; Ouazad, 2014) — results in low expectations along race lines (Liou & Rotheram-Fuller, 2019). This leads to a self-fulfilling prophecy: Teachers have low expectations for students of color, which has negative effects on those students’ outcomes (Jussim & Harber, 2005). 

Also, this racist and inequitable context is tied to both special education and language classifications. Emergent bilingual students, the majority of whom are Latinx (National Center for Education Statistics, 2019), are framed as lacking English rather than proficient in one or more languages (Aldana & Martinez, 2019). Likewise, students with disabilities are racialized (Artiles, 2011; Tefera & Voulgarides, 2016), reflecting “the historical intersections between racial oppression and the marginalization of people with disabilities” (Aronson & Boveda, 2017, p. 1). Indeed, students of color, especially Black and Indigenous students, are overidentified for special education, a fact that is connected to educator bias (Connor, 2017; González, Tefera, & Artiles, 2015; Tefera et al., 2019). In other words, when educators use students’ classifications for special education or English learner services as simple explanations for test scores, they are actually feeding racism in education. 

Deficit thinking and data  

To understand how teachers were making sense of and using data in the six schools we studied, our research team observed teachers, coaches, and administrators (representing a range of racial/ethnic identities) over the course of a year. We found that their conversations often followed the pattern described above, with teachers explaining away students’ low test scores by pointing to some aspect of their identity, such as English language or special education designations. This was especially concerning because all six schools were attended mainly by students of color (Black students in one school and Latinx students in the five others). The percentage of students designated for special education ranged from 8% to 20%; in the five predominately Latinx schools, 25-35% of students were identified as emergent bilingual students.  

Over the course of the year, we documented 62 instances in which educators attributed students’ performance to one or more specific causes. In 25 (or 40%) of those instances, the educator explained that performance by pointing to student designations (instead of or in addition to the instruction they had received, the design of the test, or other factors).  

In other words, they attributed students’ performance to their categorization, asserting that if emergent bilingual students, students with disabilities, and other groups of students get low test scores, it is because they are uniformly and unalterably deficient. Echoing the comment made by Dennis, above, a teacher named Paula said, “I believe my biggest challenge is making curriculum accessible to [special education] and far-below-basic students. Language arts is very inferential, and this provides difficulties for the population that I teach.” She used a broad brush to paint a wide range of students, implying that all students in special education, regardless of their disability, have difficulty with inferential thinking. Similarly, Renée explained low scores in her class this way: “It’s not surprising because I have some low boys in there, and I have some resource kids.” Students who belong to certain categories, she suggested, cannot be expected to improve their performance.  

By extension, such comments also seem to absolve teachers from responsibility. After all, the thinking goes, if those students are destined to achieve at low levels, then their performance has nothing to do with the quality of the instruction they’ve received. So, for example, when discussing the outcomes of emergent bilingual students and students with disabilities in her school, a teacher named Rebecca asked her colleagues, “[If] these are their deficiencies, [then] what can we use to help them get to that point?” In her view, perhaps she could help the students reach a certain level, but she could not change what she saw as the students’ essential nature as poor in academics.   

Like Rebecca, many of the teachers acknowledged that the quality of their instruction had some effect on students’ performance. In fact, in 14 of the 25 instances when teachers attributed test results to student categorizations, they said that their teaching had an influence, as well. For instance, a 6th-grade English teacher named Katie said:  

I ask myself questions about my teaching. It’s like, is it these type of kids? . . . What’s working with these kids that’s not working with these other kids? . . . And I was asking myself whether it was my teaching, or whether it was certain kids. 

Still, while Katie wondered whether her teaching might have been partly to blame for her students’ low scores, she quickly pivoted back to the idea that “these type of kids” don’t do well in school.  

The problem with categories 

We don’t mean to suggest that teachers should never mention students’ designations or the services they receive when discussing data. We understand that emergent bilingual students may have difficulty in a class or on a test, simply because they have not yet mastered English. Likewise, we recognize that some disabilities do pose real challenges for learning. And we know that, at times, it is perfectly reasonable for educators to point out students’ language status or disabilities as part of a discussion about how best to tailor instruction.  

However, it becomes a serious problem when, as we saw many times in our study, educators reduce students to the labels they’ve been given, whether those labels attach to their identity (e.g., emergent bilingual students), the services they receive (e.g., “resource” students), or their test scores (e.g., students who score far-below-basic). In the data-team meetings that we observed, some teachers tended to forget that the students behind the data are whole, vibrant, and ever-changing young people whose potential can’t be summed up by naming the category to which they’ve been assigned.  

It is no surprise that many educators would conflate individual students with the labels they’ve been given, considering how test-based accountability policies have shaped our schools. For two decades, teachers have been pressured to boost scores by, for instance, moving students from “far below basic” to the “basic” level, while administrators have had to keep a close eye on the test scores of various subgroups, such as emergent bilingual students or students with disabilities. In short, our state and federal policies have conditioned educators to see students not as individuals but as populations to be measured and monitored in categories. Moreover, “looking at the data” tends to mean looking for ways to boost the scores of certain subgroups, lest the whole school be sanctioned. 

Due to the student populations at the schools in our study, when the educators looked at the data and attributed test scores to students’ status as English learners (ELs), resource students, or struggling students, they were blaming students of color. Consider this in light of the larger context, in which Black, Indigenous, and other students of color continually face lower expectations and harsher punishment than their white peers and in which emergent bilingual students and students with disabilities are racialized and Black and Indigenous students are overidentified for special education. Deficit perspectives characterize these racialized student groups — largely defined by education polices — as monolithic and persistently “low.” This is especially troubling considering the harmful effects of low expectations and the ways that some educators and schools use data to track students (Park & Datnow, 2017), a practice that ultimately re-entrenches and legitimizes racial inequity (Oakes, 2005). Data-driven deficit thinking, then, works to entrench systemic racism and white supremacy. 

What should we do? 

Given the dangers of data-driven deficit thinking, we offer two overarching suggestions.  

First, our research makes clear that any efforts to promote data use in schools should be grounded in an understanding of systemic racism, educator biases, and how to use data with intentionality. Schools and districts should promote “normative” change, or shifts in core beliefs and values (Holme, Dien, & Welton, 2014; Oakes et al., 2005) as opposed to surface-level, technical change. In other words, rather than considering the work complete after simply creating disaggregated data displays and setting up meetings to review them, leaders should invest time in changing cultures and challenging deficit mindsets that greatly affect the interpretation of those data. If they ask teachers to analyze students’ data, they must, at the same time, provide coaching, professional development, and other resources designed to challenge their assumptions about specific student populations (Christman, Ebby, & Edmonds, 2016), confront their biases (Katz & Dack, 2014), and give them the historical context they need to make sense of why some student groups might perform better than others on the usual measures of academic performance. And when educators go on to analyze data, their school leaders should constantly remind them that the goal is to improve instruction, not to blame students or talk about their supposed deficits. 

We are not suggesting that educators should avoid disaggregating data by student subgroups but, instead, that they should approach these data with intentionality. Lesli Myers and Kara Finnigan (2018) offer a framework to facilitate conversations that directly address racial biases in data use and assist with learning. Central to the framework is the use of disaggregated data on both outcomes and opportunities along racial lines to help educators notice patterns and discuss root causes of different student outcomes. In these discussions, Myers and Finnigan (2018) suggest that educators should be asked to consider how systemic racism affects youth and to question “deeply held beliefs or assumptions” (p. 41). In this way, educators can explore outcome data for subgroups of students alongside opportunity data so that they are better equipped to “focus on solutions rather than on blaming particular people.”  

Thus, to gain a more nuanced understanding of the data they study, educators should consider not just their students’ outcome data but also the perspectives of youth of color (Bertrand, 2018) and data on racial opportunity gaps (Milner, 2012). Such information allows for what has been called “critical data-driven decision making” (Dodman et al., 2019), which calls on educators to take a careful look at the ways in which their schools have linked students’ identities with particular labels, such as when students of color are disproportionately designated for  special education (Artiles, 2011; González , Tefera, & Artiles, 2015; Tefera et al., 2019).  

Second, we argue that tracking has no place in critical data-driven decision making or in any district that wants to interrupt systemic racism. When educators use data, they sometimes default to what Vicki Park and Amanda Datnow (2017) call a “logic of tracking and ability grouping,” as opposed to a “logic of differentiated instruction.” The educators in our study relied on a logic of tracking when they blamed test scores on students’ institutional labels and the tracks defined by these labels. This kind of data use results in “homogeneous” groupings that do not often change and that severely limit opportunities for students in the “low” tracks (Oakes, 2005; Park & Datnow, 2017). On the other hand, a logic of differentiation involves the occasional use of flexible grouping based on students’ understanding at a given time. We saw this idea at work in our study with a teacher who regularly asked students to privately analyze their own interim assessment results to determine improvement goals. On one occasion, the teacher offered five different activity groups focused on different content areas and allowed students to choose the group that would offer them the most help. 

These two suggestions are especially salient now, as we contend with an extended pandemic that forced schools across the country to move to online learning at the end of the last school year. We wonder if some educators’ worries about learning “loss” and “gaps” due to some students’ more limited access to formal schooling during the school closures could lead them to disproportionately target students of color for special interventions, whether they need them or not. This could engender or reinforce deficit thinking. With this in mind, critical data-driven decision making — involving reflexive, race-conscious, and holistic approaches to data use — is more important than ever.    

References 

Aldana, U.S. & Martinez, D.C. (2019). Reimagining the educational environment in California for emergent bilinguals: The implications of the passage of Proposition 58. In M. Pacheco, P.Z. Morales, & C. Hamilton (Eds.), Transforming schooling for second language learners: Theoretical insights, policies, pedagogies, and practices (pp. 81-100). Information Age Publishing Inc. 

Aronson, B. & Boveda, M. (2017). The intersection of white supremacy and the education industrial complex: An analysis of #BlackLivesMatter and the criminalization of people with disabilities. Journal of Educational Controversy, 12 (1), 1-20. 

Artiles, A.J. (2011). Toward an interdisciplinary understanding of educational equity and difference: The case of the racialization of ability. Educational Researcher, 40 (9), 431-445. 

Bertrand, M. (2018). Youth participatory action research and possibilities for students of color in educational leadership. Educational Administration Quarterly, 54 (3), 366-395.  

Bertrand, M. & Marsh, J.A. (2015). Teachers’ sensemaking of data and implications for equity. American Educational Research Journal, 52 (5), 861-893. 

Carter, P.L., Skiba, R., Arredondo, M.I., & Pollock, M. (2016). You can’t fix what you don’t look at: Acknowledging race in addressing racial discipline disparities. Urban Education, 52 (2), 207-235. 

Carter, P.L. & Welner, K.G. (2013). Closing the opportunity gap: What America must do to give every child an even chance. Oxford University Press. 

Christman, J.B., Ebby, C.B., & Edmunds, K.A. (2016). Data use practices for improved mathematics teaching and learning: The importance of productive dissonance and recurring feedback cycles. Teachers College Record, 118 (11), 1-32. 

Connor, D.J. (2017). Who is responsible for the racialized practices evident within (special) education and what can be done to change them? Theory Into Practice, 56 (3), 226-233.  

Datnow, A. & Park, V. (2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19 (2), 131-152. 

Dodman, S.L., DeMulder, E.K., View, J.L., Swalwell, K., Stribling, S., Ra, S., & Dallman, L. (2019). Equity audits as a tool of critical data-driven decision making: Preparing teachers to see beyond achievement gaps and bubbles. Action in Teacher Education, 41 (1), 4-22.  

Fox, L. (2016). Seeing potential: The effects of student-teacher demographic congruence on teacher expectations and recommendations. AERA Open, 2 (1). 

Garner, B., Thorne, J.K., & Horn, I.S. (2017). Teachers interpreting data for instructional decisions: Where does equity come in? Journal of Educational Administration, 55 (4), 407-426.  

González, T., Tefera, A., & Artiles, A.J. (2015). The intersections of language differences and learning disabilities: Narratives in action. In M. Bigelow & J. Ennser-Kananen (Eds.), The Routledge handbook of educational linguistics (pp. 145-157). Routledge. 

Holme, J.J., Diem, S., & Welton, A. (2014). Suburban school districts and demographic change: The technical, normative, and political dimensions of response. Educational Administration Quarterly, 50 (1), 34-66. 

Jussim, L. & Harber, K. (2005). Teacher expectations and self-fulfilling prophecies: Knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review, 9 (2), 131-155. 

Katz, S. & Dack, L.A. (2014). Towards a culture of inquiry for data use in schools: Breaking down professional learning barriers through intentional interruption. In Studies in Educational Evaluation, 42, 35-40. 

Liou, D.D. & Rotheram-Fuller, E. (2019). Where is the real reform? African American students and their school’s expectations for academic performance. Urban Education, 54 (3), 397-429. 

Losen, D., Hodson, C., Keith II, M.A., Morrison, K., & Belway, S. (2015). Are we closing the school discipline gap? The Center for Civil Rights Remedies at The Civil Rights Project, University of California, Los Angeles. 

Marsh, J.A. & Farrell, C.C. (2015). How leaders can support teachers with data-driven decision making: A framework for understanding capacity building. Educational Management Administration & Leadership, 43 (2), 269-289. 

Marsh, J.A. & Kennedy, K. (2020). Chapter 9: Possibilities and challenges: Conditions shaping educators’ use of social–emotional learning indicators. Teachers College Record, 122 (14), 1-28. 

Milner IV, H.R. (2012). Beyond a test score: Explaining opportunity gaps in educational practice. Journal of Black Studies, 46 (6), 693-718. 

Myers, L.C. & Finnigan, K.S. (2018). Using data to guide difficult conversations around structural racism. Voices in Urban Education, 48, 38-45. 

National Center for Education Statistics. (2019). Indicator 8: English Language Learners in Public Schools. U.S. Department of Education. 

Oakes, J. (2005). Keeping track: How schools structure inequality (2nd ed.). Yale University Press. 

Oakes, J., Welner, K., Yonezawa, S., & Allen, R. (2005). Norms and politics of equity-minded change: Researching the “zone of mediation.” In M. Fullan (Ed.), Fundamental change: International handbook on educational change (pp. 282-305). Springer. 

Ouazad, A. (2014). Assessed by a teacher like me: Race and teacher assessments. Education Finance and Policy, 9 (3), 334. 

Park, V. (2018). Leading data conversation moves: Toward data-informed leadership for equity and learning. Educational Administration Quarterly, 54 (4), 617-647.  

Park, V. & Datnow, A. (2017). Ability grouping and differentiated instruction in an era of data-driven decision making. American Journal of Education, 123 (2). 

Roegman, R., Samarapungavan, A., Maeda, Y., & Johns, G. (2018). Color-neutral disaggregation? Principals’ practices around disaggregating data from three school districts. Educational Administration Quarterly, 54 (4), 559-588. 

Tefera, A.A., Artiles, A.J., Lester, A., & Cuba, M. (2019). Grappling with the paradoxes of inclusive education in the U.S.: Intersectional considerations in policy and practice. In M. Hartmann, M. Hummel, M. Lichtblau, J. Löser, & S. Thoms (Eds.), Facetten inklusiver Bildung (pp. 117-125). Klinkhardt. 

Tefera, A.A. & Voulgarides, C.K. (2016). Is educational policy alleviating or perpetuating the racialization of disabilities? An examination of “big-P” and “little-p” policies. Teachers College Record, 118. 

Valencia, R.R. (2010). Dismantling contemporary deficit thinking: Educational thought and practice. Routledge. 

Whitford, D.K. (2017). School discipline disproportionality: American Indian students in special education. The Urban Review, 49 (5), 693-706.