The limits of ‘threat assessment’

No, education journalists, ‘threat assessment’ isn’t a magic cure for preventing school shootings.

By Maureen Kelleher

On December 9, the Washington Post ran a story with the optimistic headline: “How can schools detect potentially violent students? Researchers have an answer.”

The answer: threat assessment.

In the story, Post reporters Laura Meckler and Hannah Natanson focused on the fact that threat assessments bring a larger team and a multidisciplinary approach to determining whether a student poses a risk of harm to others.

“Without all the facts and context, it is not possible to judge what the Oxford school authorities should or should not have done,” University of Virginia threat assessment researcher Dewey Cornell is quoted saying.  “But it appears to have been a preventable event, as are most school shootings,” he said.

Lack of coordinated action is shaping up to be a key missing element of Oxford High School’s response to the warning signs presented by the school shooter, and possibly a $100 million mistake.

However, the Washington Post reporters — and, more likely, their editor — underplayed the limitations of the threat assessment process, particularly with regard to the civil rights of Black students and students with disabilities, leaving readers to think threat assessment is a complete answer when it isn’t.

The Washington Post reporters — and, more likely, their editor — underplayed the limitations of the threat assessment process, particularly with regard to the civil rights of Black students and students with disabilities. 

According to their main expert’s research, no racial or other disparities were found in the disciplinary consequences meted out to students who experienced the threat assessment process.

But other researchers have observed that school systems are more likely to refer Black students and students with disabilities for threat assessments. 

In fact, the reporters spoke with one such researcher, Jazmyne Owens of New American, but her concerns are buried deep in the story.

Concerns about misuse of threat assessment models in schools have been reported in the past.

In 2018, former Oregonian education reporter Bethany Barnes won the Education Writers Association National Award for Education Reporting on the strength of her nuanced story about the harm done to a young man erroneously targeted as a threat. 

Barnes’ story described how a Portland school had subjected one of its students to a threat assessment based on a report from the school librarian, who misunderstood what she had overheard in a conversation among students.

The targeted student was an autistic 16-year-old who was eventually pushed out of day school and the theater tech program he loved, into night school. He never received formal disciplinary action.

“They’ve effectively created a dropout,” said his father, who cooperated throughout the threat assessment process. 

Concerns about misuse of threat assessment models in schools have been reported in the past. 

Like any decision process, threat assessments are imperfect. They can be influenced by biases and assumptions about who presents a threat in school, which often target Black and Brown boys and young men.

Threat assessors can also be under-informed about students with disabilities like autism and perceive a threat where there is none due to that lack of information and understanding.

However, the reporters in this story concentrated their discussion of the challenges related to threat assessment on the difficulty of establishing whether a student is lying to the assessors.

This is of course relevant, since the Oxford school shooter is alleged to have lied about his intentions to school staff.

But more attention is devoted to the “messy” discipline situation caused by school efforts to reduce discipline disparities than to the reality of the longstanding bias in identifying students.

Given the well-established problems in schools related to racially-disparate discipline and arrests of Black and Brown boys and students with disabilities, it’s especially troubling to see education reporters minimize the potential for bias in the threat assessment process.

It’s especially troubling to see education reporters minimize the potential for bias in the threat assessment process.

In addition, the Post story also misses a key source to provide a cautionary view of threat assessments: Miriam Rollin, director of the Education Civil Rights Alliance and an attorney for the National Center for Youth Law.

As an attorney with extensive experience working on the ground with youth who have been targeted, she can point to the problems that frequently occur when threat assessments are conducted. Bias and misjudgment are common, she notes. (Disclosure: Rollin has written five columns for Education Post, where I am an editor.)

“Unfortunately, threat assessments can be triggered by unreliable, anonymously reported ‘threats,’ that may be subject to interpretation or lack any factual basis,” Rollin has written. “Threat assessments can also be triggered by normal child behavior—an attempt at humor, or acting out in temporary frustration—that poses no sustained threat of substantial harm with a defined target, timing, means and motive.”

Threat assessments might be helpful in some situations, but they’re no magic answer when it comes to preventing school shootings. 

The Solutions Journalism Network offers four guiding principles when writing a solutions story: focus on the problem and how the responses work in meaningful detail; focus on effectiveness, present available evidence of results; discuss the limitations of the approach; seek to provide insight others can use. 

By that measure, Meckler and Natanson — and their editor — went three for four on this story.

They missed out on a full discussion of the limitations of the approach, which could have strengthened the insights the story provided for other districts looking for answers to stopping school shootings. 

Threat assessments might be helpful in some situations, but they’re no magic answer when it comes to preventing school shootings. And, innocent young people have been harmed when threat assessments were poorly conducted.

The Post’s education team has a duty to make both those points crystal-clear to its readers.

Maureen Kelleher is a longtime education journalist and editor for the Education Post.  You can follow her on Twitter at @kellehermaureen.

Previously from The Grade

Inside Oxford High School (Cafeteria Duty)

What really happened before the Oxford shooting started? We still need to know. 

Parkland coverage, 12 months later