The harm in assessment-backward practices

My five-year-old recently brought home her math assessment results and an item covering content she understood was marked incorrect. As someone who has years of both practical and research experience with assessment, I delved deeper and was not surprised to find an issue of validity.

An assessment is considered valid based on its ability to:

• Measure what it is intended to measure.
• Provide valuable and appropriate information for the intended purpose.

My daughter’s assessment covered the concepts of “less than” and “greater than.” The item marked incorrect displayed four images of an airplane. The instructions were to draw one set of counters representing less than the group displayed.

My daughter drew three different groups of counters instead of just one. One group had one counter, one group had two counters, and the final group had three counters. The item was marked incorrect because she drew all the possible answers when she was only required to draw one.

Here the assessment was violating the first principle of assessment validity, measuring what it was intended to measure. My daughter demonstrated that she could apply the skills to answer the question, but because the answer was considered incorrect my daughter was marked for remediation. This violates the second principle of assessment validity, providing valuable and appropriate information for the intended purpose.

Teachers need agency

If the teacher had exercised her agency to interpret the answer, she would conclude that my daughter had actually mastered the skill. In fact, my daughter could demonstrate all the possibilities for what “less than 4” looked like, eliminating the possibility that she made a lucky guess.

My daughter was marked as needing remediation not because she hadn’t shown mastery of the skills the assessment was intended to measure, but because the assessment was actually testing adherence to the subscribed assessment, fidelity to the answer key, and her understanding that a correct response would only include one example. Her test became a real-life example of assessment invalidity.

The purpose of this anecdote is not to bash my daughter’s teacher or school. As someone who taught K-12 as recently as 2022, I know teachers often are discouraged from using their own judgment in these circumstances. A simple conversation would have shown that my daughter understood the material and did not need remediation. But her teacher likely did not have the option to override the assessment results.

I tell my preservice teachers, “We teach children, not content or curriculum.” Yet, many schools have adopted scripted curriculums and intervention programs to meet state requirements and to capitulate to ongoing political pressures to obtain the carrots (reward and growth designations) and avoid the sticks (demerits and sanctions).

Backward design and accountability

When I was training to become a teacher two decades ago, I learned the concept of backward design. The idea is that when planning lessons, teachers were supposed to deconstruct a standard to identify key skills and content necessary for mastery. We then identified a desired outcome, decided how to assess its attainment, and planned instruction to get us there. Lesson planning was a road trip where we picked a destination, charted our route, and planned our stops and rest breaks for along the way. If we went off course or ran into a roadblock, we would plan a detour or retrace our route.

What has happened over time is a shift to assessment-backward design. States provide assessments, offer up rewards or punishments for performance, and then leave it to schools and districts to plan (or purchase) curricula to get students to perform well on those assessments. In many cases, the curricula map out even the day-to-day and moment-to-moment instructional decisions. It’s like going on a road trip with a map you must follow regardless of bad weather, car sickness, traffic jams, or even closed routes.

So, how did we get here? A general explanation is that since the days of Horace Mann, someone has always held American schools and teachers accountable for something. But as the scope of accountability has moved from schools to individual teachers, assessments have become more fine-grained; curricula have narrowed; and market and political pressures have coerced states, districts, and schools to adopt prescribed curriculums and, in many cases, enforce implementation at a high level of fidelity.

The work of many teachers has become highly micromanaged, leaving them little agency to decide what is best for students. New teachers now enter their careers in schools where they often must apply scripted curriculums with high fidelity, sometimes at odds with the pedagogy and methods they learned in teacher education programs.

Assessments and accountability are in no way inherently bad. I myself am a bit of a credential hog, having insistently maintained my National Board Certification, even as I have worked in higher education settings. However, I am reminded of social scientist Donald Campbell’s law (1976), which states, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Unintended consequences

Researchers have long discussed the so-called unintended consequences of accountability reform (Figlio & Loeb, 2011, as one example). One such consequence is the gaming of the systems. This may include benign efforts like improving school lunches on test days so students will perform better (Figlio & Winicki, 2005) to the more serious, like outright cheating (Hibel & Penn, 2020) or removing more vulnerable student populations from the testing pool (Booher-Jennings, 2006). Accountability policies have been linked to increased levels of dissatisfaction among teachers (e.g., Smith & Holloway, 2020). I would argue that other consequences include increased reports of student test anxieties (e.g., Segool et al., 2013; Von der Embse et al., 2018) and more parental support for alternative modes of educations, such as homeschooling (Wang, Rathburn, & Musu, 2019). The violation of Campbell’s law not only persists but has become normalized. The adoption of increasingly scripted curricula and assessments is one more example of this normalization.

Not every teacher is plugging through a script. Different contexts allow for different levels of adaption, supplementation, or even, when necessary, outright rebellion. But the fact remains that American schooling as a whole has increasingly shifted toward teaching for an assessment, not for mastery of transferrable skills.

What this means for the teaching profession

For the newest entries in the teaching workforce, this world of micro-accountability focused on the actions of individual teachers is all they know. One preservice teacher recently told me of her experiences in the field:

In a way it’s a relief that they just expect me to follow a script that tells me exactly what to do . . . I feel protected. Even if I fail to teach the kids, I’ve done my job and can avoid conflict.

We should be concerned that some new teachers are comforted by the requirement that they rigidly adhere to the script. A focus on just getting the “correct” version of “right” without understanding the why or the how could become the future of education. This is educational malpractice.

Overall, the deprofessionalization and mechanization of teaching has been a long process that has occurred along with the accountability movement. Those who have benefited from this shift (politically and/or financially) are often the same ones supporting anti-public education movements that often offer ways to circumvent accountability policies. We should question the motives of those supporting such agendas: There is much potential to gain personally from pushing systems that are doomed to fail and lead to the systematic dismantling of public education.

As a society, this all should scare us. Schooling matters. What happens if we are no longer educating children in schools but, rather, focusing on controlling actions and micromanaging behavior? I ask myself this question as a teacher, a teacher-educator, a scholar of education policy, and most importantly, as a mother.

References

Booher-Jennings, J. (2006). Rationing education in an era of accountability. Phi Delta Kappan, 87 (10), 756-761.

Campbell, D.T. (1976). Assessing the impact of planned social change. The Public Affairs Center, Dartmouth College. http://goo.gl/AAaPx

Figlio, D., & Loeb, S. (2011). School accountability. Handbook of the Economics of Education, 3, 383-421.

Figlio, D.N. & Winicki, J. (2005). Food for thought: The effects of school accountability plans on school nutrition. Journal of Public Economics, 89 (2-3), 381-394.

Hibel, J. & Penn, D.M. (2020). Bad apples or bad orchards? An organizational analysis of educator cheating on standardized accountability tests. Sociology of Education, 93 (4), 331-352.

Segool, N.K., Carlson, J.S., Goforth, A.N., Von Der Embse, N., & Barterian, J.A. (2013). Heightened test anxiety among young children: Elementary school students’ anxious responses to high-stakes testing. Psychology in the Schools, 50 (5), 489-499.

Smith, W.C. & Holloway, J. (2020). School testing culture and teacher satisfaction. Educational Assessment Evaluation and Accountability, 32, 461-479.

Von der Embse, N., Jester, D., Roy, D., & Post, J. (2018). Test anxiety effects, predictors, and correlates: A 30-year meta-analytic review. Journal of Affective Disorders, 227, 483-493.

Wang, K., Rathburn, A., & Musu, L. (2019). School choice in the United States: 2019 (NCES 2019-106). National Center for Education Statistics.

This article appears in the May 2024 issue of Kappan, Vol. 105, No. 8, p. 66-67.