Giving Compass' Take:

• John Engberg and Catherine H. Augustine explain how a RAND report reveals both the promise and limitations of restorative justice interventions. 

• How can the subtitles of report of this nature be better conveyed to people who read only a brief summary? 

• Learn more about the difficulties of reports reaching their intended audience.

A recent blog post by Max Eden laments the tendency of the education media and researchers alike to succumb to the well-documented inclination of human beings to focus on facts that confirm their prior beliefs. At RAND, we know that this is an immense challenge, not only to human beings in the media but to human researchers as well. That's why we bend over backwards to make sure we present complex findings in the most balanced light possible.

In commenting on a new RAND report about the impact of restorative practices, Eden bemoans “RAND's unmistakably slanted editorial emphases” and quotes a journalist as saying that “[t]he negative test for effect for black kids is buried on like page eighty with no mention (that I saw) until then.”

Following U.S. Department of Education guidance, our report focuses on estimates of the impact of restorative practices on comprehensive outcome measures for all students. We define these measures in seven outcome domains and lead our report with the impact on suspensions, which was the primary objective of the intervention. We also present estimates of the impact on 50 secondary measures as well as on the impact on the seven primary measures for numerous student subgroups defined by race, income, special needs and English language learner status. It is not possible to discuss in depth all of these in the body of the report, much less in the executive summary or research brief targeted at policymakers and practitioners.

Although not mentioned by our critic, we did highlight the presence of negative findings in our reports. We emphasized negative findings in our research brief that was directed to the media, in the executive summary of our larger research report, and in the findings posted on our internet page: “Restorative practices did not positively affect all students in the treatment schools. Despite fewer suspensions, academic outcomes did not improve in the treatment schools. At the middle grade level (grades 6–8), academic outcomes actually worsened in the treatment schools, and suspension rates in those grades did not change…[bold from original]” Furthermore, although the impact on achievement is not the primary focus of the initiative, our presentation of the achievement impact findings is clearly labeled in our table of contents and features a lengthy discussion of the negative impact on African American students.

We definitely agree that every school-based intervention should be in part judged by its impact in achievement, even when achievement is not a focus of the intervention. We also agree that the impact on racial disparities in achievement should always be examined. We also know that many interventions cause a temporary dip in achievement, often seen most starkly among already struggling students, as teachers work out how to incorporate new practices into their routines. Therefore, initial achievement impact estimates should be interpreted cautiously.

Read the full article about restorative justice interventions in schools by John Engberg and Catherine H. Augustine at RAND Corporation.