Improve assessment literacy outside of schools too

Teaching and assessment have become separated, which has kept teachers from developing the assessment skills they need to truly enhance learning.

When I introduced the concept of “assessment literacy” in Kappan over two decades ago (Stiggins, 1991), I advanced the argument that teachers and school leaders absolutely must understand the basic principles of sound assessment practice. Specifically, if we are to develop truly effective schools, educators must understand how to gather dependable evidence of student achievement and use the assessment process and its results either to support or to certify student achievement depending on the context.

Since then my colleagues and I have spent decades developing and sharing print, video, and online professional development for preservice and in-service teachers and school leaders that maximizes their assessment literacy. Without question, our greatest challenge has been the dishearteningly difficult work of convincing school leaders to allocate resources for developing assessment literacy.

The purpose of this update is to explore why. I believe the answer resides in a series of societal and educational beliefs about school testing that, paradoxically, have been barriers to developing assessment competence in the classroom. I will share specific examples. Removing these barriers depends on our ability to build a foundation of assessment literacy in the community beyond school.

We need to provide a deeper, more complete understanding of basic principles of sound assessment practice to those outside of schools who drive policy, resource allocation, and classroom priorities.

These mistaken beliefs about assessment’s role in school improvement have led misinformed federal, state, and local policy makers to set assessment policies that require local educators to implement unsound practices. Further, those unsound practices have become so engrained in school/community culture that parents accept or even expect them. This has happened because those driving federal, state, and local school policy and practice don’t understand the differences between sound and unsound testing practices. As a result, our collective societal vision of excellence in assessment has been flawed for decades and in several ways. I’ve written extensively about these flaws and offered an alternative vision of our collective assessment future in Revolutionize Assessment (Stiggins, 2014). But more powerful visions of excellence in assessment will remain beyond reach until barriers of misinformation and mistaken beliefs beyond the schoolhouse are removed.

Assessment literacy barriers

Standardized achievement testing is one culprit. To be clear, I support using these tests in a certain limited array of instructional decision-making contexts. But a societal blind spot has been created by the common belief that standardized test results are the only truly acceptable evidence of student achievement. If the evidence comes from the teacher and the classroom, applies to just one group of students, and doesn’t yield comparable results beyond that context, then it’s deemed untrustworthy. Our collective faith in the power of standardized tests has become a light so brilliant in our collective eyes that we’re unable to see the severe limitations of the tests or that more powerful classroom applications of assessment can promote far greater student learning success.

For example, many outside of schools accept on faith that college admissions test scores predict academic success in college. In fact, these test scores correlate only very modestly with freshman college grade point average and not at all with grade point average after that initial year. But, regardless of this reality, if society holds onto the mistaken belief anyway, we can relegate responsibility for evidence gathering at the time of high school-to- college transition to the standardized test. With such a “powerful” test score in place, we don’t have to worry about, invest in, or certify the assessment literacy of teachers.

Many outside of schools believe that annual state or local standardized test scores reveal school quality. In fact, teacher and other in-school factors only account for a small percentage of the variance among students in annual test scores, with much of the rest explained by factors beyond the control of schools and teachers — home, family, community, socioeconomics, for example. If we ignore this complicating reality and trust these scores, we gain the appearance of academic rigor and don’t have to (or can choose not to) trust teachers to deliver it. So teachers don’t need to be assessment literate.

Many in the community outside schools believe that we can use annual test scores to detect differences in the quality of teachers, instruction across schools in the same district, or teaching across districts in the same state. But, in fact, the instructional sensitivity of these tests — that is, their ability to detect differences in the quality of instruction — has never been tested, let alone verified (See W. James Popham’s article in the September 2104 issue of Kappan). But if we simply ignore this truth, we don’t have to seek other more dependable sources of evidence of school quality, such as those that teachers might provide. We don’t need to trust teachers or train them to generate such evidence. So their assessment literacy is not relevant.

Recently, misinformed policy makers have banked on this mistaken belief to assert that we can and should use standardized test scores as evidence of student growth and a measure for evaluating teacher performance. But these tests have never been either designed or validated for this use. Indeed, they’re unacceptable for such a use for many practical technical reasons (see Popham, 2013; Stiggins, 2014). But if we ignore this, then we don’t have to ask teachers to provide evidence of the quality of their own instruction. So neither teachers nor their supervisors need to know about sound assessment practice.

As a society over the decades, we have not trusted teachers to gather and present evidence of the learning of their students for accountability purposes. As a result, a school culture has emerged separating those who test from those who teach, and never the twain shall meet — each with their own mutually exclusive competencies. Those who do the testing develop the competencies needed to do that testing job while those who teach develop instructional competencies. And because historically these two sets of professional standards don’t overlap in either direction, two interrelated problems result:

1. Teachers spend as much as a quarter to a third of their professional time involved in assessment-related work without the essential training needed to do it well; and
2. Testing people, who see their jobs as producing evidence for accountability, have difficulty understanding or producing the kind of results teachers need to inform the crucial kinds of instructional decisions they face.

Because of this long-standing, two-silo division of responsibility, important evidentiary requirements are not met in the classroom, and student learning suffers.

Expanding our collective assessment literacy

Given this array of apparent reasons not to train local practitioners in assessment, I contend that resources and opportunities for practitioners to learn about and implement sound classroom assessment practices have not been forthcoming. Unless and until we provide a deeper, more complete understanding of basic principles of sound assessment practice to those outside of schools, practitioners in schools won’t get the professional development needed to do their assessment jobs. Therefore, the time has come to remove those barriers by developing the assessment literacy of those who influence instruction from outside of schools.

Assessment literacy for policy makers, parents, and community

Federal, state, and local policy makers as well as parents, taxpayers, and our communities at large must learn a variety of lessons.

1. Society has changed the mission of its schools. We expect them to do far more than sort students along a continuum of achievement by the end of high school. Important new missions have been added, and our assessment practices must accommodate them.

More than sorting, we demand that schools promote higher levels of achievement than ever, while narrowing achievement gaps, assuring lifelong learner competencies for all students, and aspiring to universal high school graduation. Schools are expected to make all students ready for college and workplace training. As a result, assessment must serve both as an instructional tool to help students learn more, while still verifying or certifying student learning when appropriate. Both purposes are important, but they are fundamentally different reasons for assessing. When we seek to improve school quality in order to enhance student learning, then assessment — the instructional tool — is the key to success. But when the objective is to see if schools have improved, then assessment — the certification tool — is the key. In any particular context, users must decide which purpose is to be served and how to serve each purpose well. Those differences are summarized below.

2. Regardless of the purpose, assessments must yield dependable evidence about student attainment of achievement expectations.

Assessment is the process of gathering information to inform instructional decisions. Dependable evidence leads to good decisions and inaccurate evidence leads to counterproductive decisions. Student well-being hangs in the balance. There is reason to be concerned about the quality of assessments throughout the fabric of American education. Very few practicing teachers and almost no practicing school leaders have been trained to develop quality assessments or to use them in effective ways regardless of the purpose. Further, almost all tests are used for accountability purposes. Typically, these do not serve well as instructional tools, and their users do not understand this. Practitioners need the opportunity to learn to use classroom assessment to support learning. Without it, schools cannot improve.

3. The inferences students draw about themselves based on their interpretation of their own assessment results are as important in determining their school success as are the instructional decisions made by their teachers and school leaders based on their interpretation of those results.

Students judge their chances of future success based on their interpretation of their past success. As this unfolds in their minds over time, success or failure can take on a life of its own and can affect their sense of their own academic self-efficacy. If they give up on themselves, instructional decisions made by the adults around them no longer matter. Only if they remain confident can their teachers help them continue to learn and grow. They aren’t merely victims or beneficiaries in the testing game — they are players in that game. Their teachers need to know how to use the assessment process and its results to help students continue to believe that success is within reach if they keep trying. Few teachers have been trained to do this. That training is available; all teachers need is the opportunity to learn.

4. Standardized achievement test scores don’t measure the effect of school factors in combination with other factors beyond the control of teachers and school leaders.

It’s time to move beyond this obsessive belief in the universal potency of test scores and their effect on school quality. Perhaps a brief history lesson will help. We began large-scale standardized testing in the 1950s and ’60s with districtwide programs. We added statewide testing in the 1970s and more of it in the 1980s. We added national assessment about the same time, international assessments in the 1990s, national every-pupil testing in the 2000s, and the beat goes on with Race to the Top testing. All remain in place in some form today. So strong has been our blind faith in their power that we have yet to see any scientific research reporting a standard deviation gain in student achievement attributable to this particular form of “school improvement” that consumed investments of billions of education dollars over seven decades. We typically demand such evidence of the efficacy of proposed school improvement innovations. Why not here?

Further, please understand that teachers and school-related factors only account for a small portion of the variability in student performance on such annual tests. The remainder is explained by factors beyond the control of teachers and schools, including:

• School factors such as class sizes, curriculum materials, instructional time, availability of specialists, and resources for learning (books, computers, science labs, and more);
• Home and community supports and challenges;
• Individual student needs and abilities, health, and attendance;
• Peer culture and achievement;
• Prior teachers and schooling, as well as other current teachers;
• Differential summer learning loss, which especially affects low-income children; and
• The specific test used, which emphasizes some kinds of learning and rarely measures achievement well above or below grade level (Darling-Hammond et al., 2012, p. 8).

Therefore, it is crucial that we do not draw naïve causal links between test scores and school quality. If scores are not where we want them to be, the key question is, what can we in the community and in our schools do to support student learning? The point is not that annual standardized tests are without value. Rather, we must keep them in perspective and balance them with other assessment applications that have proven their worth in the classroom.

It’s time to move beyond the obsessive belief that we can get schools to improve by advancing shrill demands for higher test scores.

Assessment Literacy for Students: A Student’s Bill of Assessment Rights

I contend that students of all ages and in all educational contexts are vested with certain inalienable rights related to the assessment of their achievement and the use of their assessment results to influence their learning. Students and their families should be made aware of those rights, and educators should understand their professional responsibility to understand and protect them.

1. Students are entitled to know the purpose for each assessment in which they participate; that is, they have a right to know specifically how the results will be used and by whom.

Assessments and their results can support learning or certify it. Students are entitled to know which application applies to each assessment in which they participate. If the purpose is to help them learn more, then they should have the opportunity to understand how the assessment will support them; that is, what decisions they or their teacher will make based on the results that will promote their growth. If the purpose is to certify mastery of achievement standards, then students are entitled to know what accountability decisions will be made based on results. Teachers are responsible for verifying that students understand the purpose of all assessments.

2. Students are entitled to know and understand the learning target(s) to be reflected in the exercises and scoring guides that make up any and all assessments.

This standard is met when students have the opportunity to understand the learning target(s) to be mastered and assessed in student-friendly terms from the very beginning of the instruction. They’re entitled to understand the target before being held accountable for hitting it.

3. Students are entitled to understand the differences between good and poor performance on pending assessments and to learn to self-assess their progress toward mastery.

Students learn most effectively when they see and understand the pathway to success. They can see this only if the achievement destination is clear to them from the beginning of the learning (Right #2) and if their teachers help them understand the achievement increments that lead to ultimate success. Teachers are responsible for assuring that students understand these things.

4. Students are entitled to dependable assessment of their achievement gathered using quality assessments.

Assessment is the process of gathering information to inform instructional decisions. Sound decisions, either to support or certify achievement, require dependable evidence. Specifically, students are entitled to assessments that rely on a proper assessment method, sample their achievement appropriately, rely on high-quality exercises and scoring schemes, and are free of bias. Since students may be incapable of judging the quality of the assessments in which they participate, assessment-literate teachers and school leaders must assure that quality.

5. Students are entitled to effective communication of their assessment results, whether those results are being delivered to them, their families, or others concerned with their academic well-being.

The assessment process works effectively only if assessment results are communicated to the intended user in a timely and understandable way and in a manner that reflects the the assessment’s intended purpose. Students have the right to have all evidence of their achievement communicated to them or to others in a way that ensures that the recipient fully understands the assessment results. When the purpose is to support learning, students are entitled to communication that describes their work in a manner that helps them do better the next time. When the purpose is to communicate a summary judgment of the sufficiency of learning, the recipient is entitled to a detailed analysis of the evidence used in the evaluation (grade assigned, for example).

Conclusion

For decades, we’ve remained blind to practitioners’ lack of competence in classroom assessment. Our safety net has been our annual standardized tests. They’ve never really worked as such, but it really didn’t matter. Our societal faith in them has been unflappable. However, by expanding schools’ mission to include providing universal lifelong learner confidence and competence, narrowing the achievement gap, and reducing dropout rates, classroom assessment has become an essential instructional tool. To take advantage of its power, we must teach teachers and their supervisors to use assessment as a teaching and learning tool — not merely as a grading tool. We haven’t done that. We will remove those barriers only when we spread an understanding of sound assessment practices beyond the schoolhouse and into the community. The time has arrived to do just that. Let the learning begin — their lessons are clear.

References

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Teacher evaluation. Phi Delta Kappan, 93 (6), 8-15.

Popham, W.J. (2013). Evaluating America’s teachers: Mission possible? Thousand Oaks, CA: Corwin.

Popham, W.J. (2014). The right test for the wrong reasons. Phi Delta Kappan, 96 (1), 42-48.

Stiggins, R.J. (1991). Assessment literacy. Phi Delta Kappan, 72 (7), 534-539.

Stiggins, R.J. (2014). Revolutionize assessment: Empower students, inspire learning. Thousand Oaks, CA: Corwin.

CITATION: Stiggins, R. (2014). Improve assessment literacy outside of schools too. Phi Delta Kappan, 96 (2), 67-72.