There Are No Miracles in Education

can we please not redo the Waiting for "Superman" era of ed discourse? please?

this kitschy horseshit did more damage than you can imagine

Northeast friends and fans! Please consider coming out to my book release event on October 7th at RJ Julia in Middletown, Connecticut, an outpost of a legendary local bookstore with a footprint that’s far larger than it’s indie status would suggest. Please RSVP here if you’d like to come. I’ll do a little reading, talk about the genesis of the novel, and answer questions, even insulting ones.

At the new wonky liberal site The Argument, Kelsey Piper has an exceptionally credulous piece about the supposed “Mississippi miracle,” which is the claim that Mississippi public schools have seen a sudden and dramatic increase in their quantitative educational metrics through the application of a little want to, a little know-how, and an extra dash of love. Supposedly, in defiance of a hundred years of experience in large-scale education policy in the developed world, some pedagogical tweaks have enabled educators in Mississippi and a couple other union-hating red states to ignore the conditions that have caused these interventions to fail again and again and again. And Piper is big mad that we haven’t just waved the magic wand and saved the kids, which she insists (again and again) is something we could just do, if only we had the will.

This is all very old hat, although I do think this new bit where people gin up extra argumentative oomph by implying that schools aren’t teaching because they’re too woke is a nice touch. Unfortunately for Ms. Piper, none of this optimism ever lasts. The odds are very, very strong that eventually it’ll turn out that students in Mississippi and other “miraculous” systems are being improperly offloaded from the books or out of the system altogether and this will prove to be the source of this supposed turnaround. That’s how educational miracles are manufactured: through artificially creating selection bias, which is the most powerful force in education. After all, there’s something very conspicuous in its absence in Piper’s piece: the so-called “Texas miracle,” a very, very similar scenario that played out some quarter-century before anyone was talking about a Mississippi miracle. The story was genuinely almost identical: George W. Bush won the governorship in Texas, made some “common sense” curricular changes, and began to demand ACCOUNTABILITY from public teachers and schools… and suddenly, a miracle happened. The schools got better! The test scores proved it!

There were real stakes here. The “Texas miracle” was the model for No Child Left Behind, the most sweeping, consequential, and disastrous piece of education legislation in the history of the country. And Bush was able to push NCLB in part because of his record as a reformer. Bush’s presidential campaign leaned heavily on these supposed test score gains in Texas, particularly in Houston, one of America’s outlier areas of concentrated failure. His record on education helped Jr. frame himself as a “compassionate conservative,” a Republican who still cared about good government who could help restore economic mobility to the country through ed policy. The story from ed policy types was that strong accountability and high standards had closed gaps and raised performance dramatically; this was part of why a non-insubstantial number of wonky moderate Democrats jumped ship to vote for Bush in the 2000 election, a trend which was later reversed thanks to the ugly optics of the War on Terror and Iraq fiasco. Unfortunately….

Later investigations revealed exactly what we should always expect to find in the face of supposed education miracles: the manufacture of selection bias through widespread underreporting of dropouts, “disappeared” struggling students, and data manipulation. Students who had dropped out were given phony classifications such as having transferred or moved into GED programs, meaning that negative dropout metrics weren’t reported; Sharpstown High School in Houston reported a 0% dropout rate in 2001-2002, even though hundreds of students left the school. Abuse of special education exemptions for testing/accountability was rampant, with the number of students in some districts doubling between 1994 and 1998. Meanwhile the state’s standardized tests were being consistently misinterpreted (in one direction, up) thanks to a complete failure to account for measurement error. The “miracle” collapsed when all of this skullduggery was revealed. Texas’s real outcomes were exposed as unremarkable, once the missing data was analyzed, and other state efforts to replicate the supposed miracle failed entirely. Whoops!

Mississippi’s supposedly miraculous results require confirmation in multiple ways. One, there has to be longitudinal (as in, following specific students, not cohorts) and truly independent verification testing, which means no participation by state education officials at all; a rise in SAT, ACT, and similar third-party tests to provide concurrent validity; a widespread and, again, fully independent audit of the administrative practices involved, with an emphasis on looking for students who have left the system, been moved into special education, or have otherwise found themselves off the books; and, most importantly, time. Time for fraud to be slowly revealed, time for more cohorts to pass through, time for stress testing and the inevitably performance attrition of this kind of “miracle.” I’m sorry, but the data we have currently does not come close to validating Mississippi’s methods.

It’s genuinely not possible that neither Piper nor her editors were unaware of the Texas miracle story. It’s a very famous policy failure with huge political and practical consequences, an immensely important cautionary tale. And you would think that the most basic journalistic responsibility would involve pointing out that this exactly kind of claim has been made before, under the exact same reasoning, with the exact same rhetoric, only for all of it to be revealed to be a matter of convenient administrative shuffling and out-and-out fraud. How do you write a piece about the “Mississippi miracle” and not mention the Texas miracle that wasn’t, a sunny, false fable which prompted immense policy consequences?

---

Let’s look at some other supposed American education miracles.

KIPP and the Charter School “Revolution” (1994 - )
The Knowledge Is Power Program (cute name!) was being held up as the exemplar of charter success long before most of these “accountability” programs were a glimmer in a neoliberal’s eye. They were some of the first to lay out the script, with the requisite amount of self-aggrandizement: high expectations, strict discipline, extended hours. Early results were strong, but later studies revealed (you’ll never guess) a heavy role for attrition. Vast numbers of the hardest-to-teach students leave KIPP schools prior to graduation. (40% of Black students!) The model has never scaled effectively to entire districts; massive, well-funded, doggedly-pursued efforts to recreate the dubious performance gains of charter networks like KIPP have failed again and again, likely because the new efforts aren’t as shameless as getting rid of struggling students. What other charter schools definitely have replicated, though, is pruning their student bodies through chicanery. (DC area charter schools, at one point, were expelling students at 72 times the rate of traditional public.) Again, this is the generic reality of high-profile charter schools: their sterling reputations are eventually found to be the product of efforts to exclude the poorest-performing students, and even with that inherent advantage, their outcomes still rarely reach the standards of even average schools in higher-income districts. Much more info here.

The Harlem Children’s Zone & “Promise Neighborhoods” (2000s)
Geoffrey Canada’s Harlem Children’s Zone was endlessly touted (famously by Waiting for “Superman”) as the model for ending poverty through education and wraparound services. The idea was exported nationally under Obama’s Promise Neighborhoods initiative. Yet while initial returns suggested that some charter schools in the Zone appeared to do better than public-school peers, as is repetitively true with high-profile charter schools, further investigations found all manner of ways that the system was pruning their student body. Some of this was the typical tools of “counseling out,” repetitive suspensions, and admissions chicanery, but they also took the novel step of expelling an entire class of students. They did so (explicitly) before they reached the high school level because those students were “too weak to found a high school on.” But that’s exactly what public schools can’t do and should never do! Of course outcomes look better when you’re getting rid of weaker students. And even still, reported academic gains were modest, inconsistent, and nowhere near the hype.

New Orleans “Rebirth” Post-Katrina (2005 onward)
The conversion of nearly the entire New Orleans system to charters after Hurricane Katrina was hailed as proof that radical reform could rapidly transform outcomes. Scores did rise - but in the context of massive demographic shifts (tens of thousands of the poorest children never returned after displacement), intensive state support, and years of philanthropic subsidy. Once again, the composition of the student body is always more powerful a predictor than any identifiable pedagogical, school, or teacher variable. Independent analyses suggest that the reported gains in New Orleans were overstated, uneven, fragile, and tied to unique circumstances that can’t be replicated. Today, the all-charter New Orleans school system is a failure factory, with half of New Orleans schools still receiving Ds or Fs on their Louisiana state Department of Education report cards, and that is not a state with what we would call the highest of standards. And yet you still sometimes hear about a “New Orleans miracle”!

Michelle Rhee and D.C. Schools (2007–2010)
Rhee became the national face of the “no excuses” reform movement, promising radical improvement in the District of Columbia’s deeply struggling schools through teacher firings, merit pay, and test-score accountability. For a few years, glowing media coverage suggested it was working. Then it all fell apart. In 2009, D.C. launched IMPACT, a high-stakes teacher evaluation system combining classroom observations with student test score “value-added” measures. Teachers with high ratings could earn big bonuses; low-rated teachers could be fired. Hundreds of teachers lost their jobs, and Rhee used the system to signal tough accountability. But as is true with merit pay systems in general, further investigations demonstrated that the system didn’t work; the system was mathematically imprecise, leading to arbitrary results. Unsurprisingly, attrition skyrocketed, particularly among Black teachers, and morale cratered among those that stayed. Cheating scandals erupted (a clear consequence of Campbell’s Law), test gains flattened, and the broader system never saw the transformation promised. Rhee exited for a lucrative career as a martyr, and the city’s schools remain as segregated and uneven as before.

Picket Middle School & Other Obama Favorites

In 2010, Barack Obama praised Philadelphia’s Pickett Middle School, newly managed by Mastery Charter, as a turnaround miracle: math proficiency had supposedly soared from 14% to nearly 70%, and violence plummeted. The example became a centerpiece of his rhetoric about school reform. But the gains didn’t survive scrutiny. When Pennsylvania adopted tougher Common Core–aligned tests, scores at Mastery schools fell sharply, in some cases worse than the district average. Investigations later noted high attrition - once again, many struggling students simply left, probably pushed - raising doubts about whether the improvements reflected true progress. A decade-plus on, Pickett and other “Renaissance” Mastery campuses perform below district norms, and the network itself has admitted early test results oversold the story. What was once heralded as proof that no school is beyond saving now reads as another cautionary tale about inflated turnarounds and the perils of building policy on temporary spikes. Undeterred, in his State of the Union address in 2011, Barack Obama named the Bruce Randolph school in Colorado as an example of the positive power of education reform and the school choice movement. Later investigation revealed that Bruce Randolph students were meeting state standards at a rate of 15% in English and 14% in math. According to the exact same standards of “accountability” endorsed by Obama and his administration, this of course meant failure. It’s unclear why Obama’s staff thought they could get away with this; apparently, the guy in charge at Bruce Randolph was a very effective marketer, and had sold a lot of local people on big promises about demanding excellence. Sadly, reality had other ideas.

What I would ask Piper and the rest of the team at The Argument and everyone breathlessly sharing that piece is simple: what’s more likely to be true? That the Mississippi miracle has actually occurred, a sudden massive turnaround in outcomes in conveniently-bounded administrative units where there’s inherent and immense pressure to fudge the numbers? That we’ve seen real improvement in metrics, when such improvement has proven to be illusory again and again and again, all with exactly the kind of minor administrative and pedagogical changes that have failed to produce gains in so many contexts? OR, is it more likely that the Mississippi miracle will prove to be like every other educational miracle that’s come before it, a product of moving poorly-performing students off the books?

And how on earth do you publish that piece without even raising the question?

---

What really bothers me in Piper’s piece is not the misguided policy it advocates for, though that’s bad enough. (It is unfathomable to me that people still think accountability is some new, untried idea in education, when successive two-term presidential administrations pushed accountability as hard they possible could, to no positive effect at all.) No, what really bothers me is the attitude at hand here, which is straight out of the mid-2000s ed reform playbook: maximally earnest, dismissive of all complication, self-righteous, supposedly speaking on behalf of THE CHILDREN, admitting no humility or modesty, and animated by the absolute certainty that we can solve our education problems simply by wanting it enough. All of this was a bipartisan norm for several decades, and that bipartisan effort was a horrific failure.

To the ed reformers, the liberal do-gooders, the concerned citizens…. I know many of you are more optimistic than I am. I get it: you want to believe in the power of institutions to move students, you assign narratives of redemption to schools, and you recoil at the thought that some educational problems might simply be stubborn and systemic rather than solvable by bright new interventions. I’m not asking you to adopt my supposedly nihilistic education views in total. But here’s my plea: don’t do a speed run of 2000s-era ed reform simplistic self-righteousness. Don’t be the person beating your chest, insisting that we could easily have a nation of geniuses tomorrow if we just STOP MAKING EXCUSES. Even if you reject much of what I say, at least don’t pretend this is all about grit, or belief, or spending. Please don’t say “We just haven’t tried hard enough.” Stop framing educational failure as a moral failing or as evidence of lazy leadership. Stop insisting that parents and teachers just have to demand more from students. Don’t force the narrative back into the Waiting for “Superman” mode, where the kids are all talented, oppressed, and innocently waiting for some heroic savior school or reform to rescue them.

It took decades of ruinous failure for the reform crowd to finally stop strutting around like they’d solved inequality with a stopwatch and a clip-board. For twenty-plus years, they sold the idea that the right mix of grit sermons and data dashboards could turn every eighth grader into a Stanford admit, and when that didn’t happen, they defaulted to blaming the teachers or parents or unions or Democrats. Only after cheating scandals, mass attrition, and the slow recognition that poverty is not a variable you can spreadsheet away did the volume finally come down. We cannot slide back into that sanctimonious era of teacher-shaming and miracle promises, not if we want to do good. Humility is not just a personal virtue here; it’s a political necessity. If we forget the profound limits of schooling, we’ll end up right back in the bad old days of sneering arrogance and deluded aggression, setting ourselves up for another generation of failure.

You might consider following the example of Matt Yglesias, who has long been an ed reform type but who has become much more pragmatic and realistic about the boundaries of the possible in the past decade. He and I still differ on this topic in myriad and major ways, about the evidence and where it points and what actually should be expected from our public school system and what its goals should be etc etc. But Yglesias has clearly evolved over the years in light of experience and evidence. He’s far more reserved about education’s potential for social transformation and also far less likely to assign blame to teachers or schools simplistically. He absorbed new information and allowed that information to inform his engagement! And, thankfully, he does not talk about this stuff with the Jon-Chait-in-2005 affect; he’s not out there engaging in self-righteous maximalism. He’s not saying “Don’t tell me these kids can’t learn!”, the Waiting for “Superman” battle cry that exemplifies this approach to education reform - the insistence that everything is possible, that failure is always a choice, and that anyone who asks any difficult questions is somehow anti-child.

I got a sinking feeling in the pit of my stomach when I watched this conversation between Yglesias and Jon Lovett of Crooked Media, because Lovett appears to have stepped right out of 2003 with no understanding whatsoever of the recent history, no backing in the relevant research, no knowledge of everything that’s happened in this century, and yet possessed of utter conviction that we could just choose to fix all the problems in our schools. It’s precisely the worst attitude to have in education debates, and he directly refers to Piper’s piece to justify his attitude. You guys, I promise, we are not going to get progress that way.

Here I must again point out the stark reality. Academic talent is normally distributed throughout the population, and so some degree of failure is literally inevitable. Not all students are equally talented, and this can never change. Differences in educational talent are real. Meanwhile, large-scale performance gaps between groups with profoundly different social and economic conditions cannot be closed on the school side; we have thrown an immense amount of money, effort, time, and manpower at those gaps at the school level for a half-century, with close to nothing to show for it. That makes perfect sense when you consider that schools control 10% or less of the variance in student academic outputs. The evidence that we can’t do what Piper insists we should do at the school level has been building for decades, and no amount of bleating the words “ACCOUNTABILITY” and “NO EXCUSES” is going to change that fact. By all means, be more critical of public schools than I am, be more supportive of charters than I am, have a more optimistic take on what’s possible at the school level than I am, whatever. But please, let’s not go all the way back. Waiting for Superman didn’t work out for us the first time. That’s the thing about miracles, after all - they aren’t real.

We’ll see if The Argument, a magazine seemingly founded on the premise that liberal good vibes can overcome every inconvenient fact and complication, will engage with this kind of criticism.