Campbell's Law and the inevitability of school fraud
When discussing education policy there's few things more useful to understand than Campbell's Law:
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
There's a great piece on Campbell's Law and testing mania from a mathematician here (PDF). The implications of this dynamic are obvious in ed circles. Why do teachers cheat in a high-stakes environments? Why do charter schools cook the books while preaching about the importance of their mission? Why do I suspect that there's a mountain of cheating and corruption going on in our ed data that hasn't been discovered? Because of Campbell's Law and the stubbornness of academic inequalities.
Hard to think of a more apt example of the influence of Campbell's Law than this story out of a San Diego charter school. I highly encourage you to read this piece, as its the kind of diligent and important local journalism that is so deeply threatened today. It tells the story of a school where a leader with missionary zeal and a no excuses culture has conspired to pressure teachers into rampant grade inflation, sending young people into higher education with grades that don't remotely match their skills.
Forgive this lengthy excerpt but I think it's worth it here.
Teachers who have worked with 48-year-old Riveroll say he’s an inspiring leader, a visionary with extraordinary charisma and passion. Parents adore the man who has been named teacher of the year, educator of the year and selected as one of four principals nationwide to participate in the Public Education Leadership Program at Harvard University.
Yet data, documents and interviews contradict the Gompers brand of preparing every student for college. Gompers’ standardized test scores — one metric for college acceptance — are among the bottom of schools in San Diego County and California. These numbers are in contrast to students’ straight A grades with courses in precalculus, advanced biology and AP history.
Teachers say grades are inflated, and if students still can’t graduate, they are “counseled” to attend school elsewhere. The same teachers who praise Riveroll’s talent blame him, saying he shames educators who assign failing grades by telling them they are “murdering” kids.
“He knows he's not allowed to say, ‘Change their grades or else,’” said former Gompers chemistry teacher Ben Davey.
“But he can say, ‘You're killing these kids, are you sure you want to leave it as an F?’”
Many people have pointed to rising graduation rates as evidence of the effectiveness of ed reform. And more kids graduating from high school is a good thing indeed. But there's concerns about the graduation rate, involving juking the stats (again) and the fear that this stems from lower standards rather than objectively better students. (Take Renewal Schools here in NYC, for example.) I'm agnostic on the overall question, although I agree with pessimists who say, for example, that something like a third to a half of all graduating American high school students probably couldn't demonstrate the requisite level of algebra ability required to graduate from high school. The question is, what happens when you combine intense pressure from above to graduate students along with the reality that (as I keep insisting) student outcomes are not nearly as plastic as policy types like to imagine they are?
Standardized tests show proficiency in math and English language arts at Gompers has gotten worse from 2011 to 2016. Forty percent of 11th-graders are below basic proficiency in English. Ninety-one percent didn’t reach the state standard for mathematics....
Six percent of Gompers students were considered “college-ready” based on their SAT scores in 2015-2016. Five percent based on their ACT.
Twenty-two percent of the Advanced Placement (AP) tests taken that year were marked three or higher, the level at which college credit is granted. San Diego Unified averaged 59 percent.
However, inewsource learned that of the 113 students graduating this year, not one earned a grade lower than a C in the first semester of their 2015-2016 school year. More than half of the class had straight A’s with courses in advanced chemistry, AP history and precalculus. Some of those students failed several lower-level classes the year before.
The class averaged a 4.7 GPA out of 5 the first half of their junior year.
It's essential to say: this kind of dynamic, where a crusading spirit and insistence that everyone can achieve to the same level collides with the limitations of reality, makes fraud, lowered standards, or both inevitable. It's an entirely predictable condition; as long as you make people's jobs dependent on reaching metrics that they can't reach legitimately, they will achieve them illegitimately. It doesn't matter how much integrity they have. It doesn't matter if they're good people. It doesn't matter if they're really invested in their students success. Campbell's Law is not a normative claim but an empirical observation; educational fraud does happen under these conditions, no matter what we think about it morally. And these lowered standards inevitably come back to bite us in the end, as these students go on to colleges where they either fail out from lack of prerequisite ability or are graduated into jobs they then can't perform. That's a problem here in the CUNY system, for example, where 57% of undergraduates can't pass an algebra test. Advancing them in the system might seem humane at the individual level but in the broader perspective it's just amplifying problems.
This is a story that some might imagine inspires a level of bitter cynicism in me. But I don't feel embittered about the story; I just feel sad about it. It seems genuinely tragic to me, in that there are genuinely good intentions leading to these bad outcomes. The charter school world is no doubt full of profiteers and con men, but I also acknowledge that many people really believe their cheery, fingers-stuck-in-their-ears rhetoric about how every single child is capable of excelling. The problem is that this enthusiasm is destructive in a world where different students have very real differences in their individual ability and in the socioeconomic and environmental conditions in which they learn. Truly humane education policy would acknowledge those differences, not attempt to paper them over with cheerful, dishonest bromides. What we need to accept as a society is that what's really "killing these kids" is not their lack of academic preparation for college but an economic system in which only those who are so prepared have a meaningful shot at a comfortable and secure life.
She remembers a strict but supportive director who valued 12-hour days out of his staff, along with sacrificing vacation time, career goals and a personal life. But she also remembers that Riveroll would bring her coffee after she’d put in late hours the night before.
“It is very challenging to balance this work,” said Parsons. “It is truly missionary work.”
So much pathology If it is to be as effective and pragmatically useful as it can be at scale, teaching cannot be missionary work. This has always been the problem with the Dead Poets Society, one-inspiring-teacher-breaks-through-to-kids-in-the-ghetto narrative. Even if I was sure these narratives actually reflected what's best for kids, they are by nature not scalable or subject to being instituted by policy. It's a hard thing for teachers to accept, but true: by its very nature, inspiration cannot be required or replicated. It's a beautiful thing when the lives of students are changed in that way. But mass education has to be a system that works in the mundane constraints of real life.
Acknowledging those constraints is necessary if we're really committed to improving our system for the pragmatic benefit of all. And the first step to getting there is to recognize that insisting that all students can excel will inevitably result in these kinds of pleasant lies.