Education Commentary is Dominated by Optimism Bias

What follows is a piece that I was commissioned to write for an education publication called The Grade. On submission they declared that the piece was, quote, “too hot” for publication. As I said to them, that response demonstrates the piece’s thesis perfectly.

There’s a bias that runs throughout our educational discourse, coming from our media, academia, and the think tanks and foundations that have such sway in education policy. It’s a bias that exists both because of a natural human desire to see every child succeed and because the structural incentives in the field make rejecting that bias professionally risky. The bias I’m talking about is optimism bias, the insistence that all problems in education are solvable and that we can fix them if only we want to badly enough. At least a half-century of research, spending, policy experimentation, and dogged effort has utterly failed to close the gaps that so vex our political class. But still we hear the same old song about how we could close those gaps tomorrow if we really wanted to, an attitude that has distorted education policy and analysis for decades.

My first book, The Cult of Smart, was a commercial failure. It was released during the height of the pandemic and thus my ability to promote it was limited, but by any measure the market rejected it. It’s tough to produce a labor of love like that and find that few people were interested in it.

But there was a silver lining: since publication in 2020 I’ve heard from dozens and dozens of teachers, thanking me for putting their thoughts to print. These educators come from public, private, and charter schools, from schools with affluent study bodies and schools that are mired in poverty, from big city school districts and from low-population regional rural schools. And again and again, these teachers shared the same perspective: they agreed with the book’s overall argument, and often had thought similar things themselves for years, but felt they could not express them publicly for fear of professional consequences.

The essential argument of the book is that overwhelming empirical evidence shows that students sort themselves into academic ability bands in the performance spectrum early in life, with remarkable consistency; that the most natural and simplest explanation for this tendency is that there is such a thing as individual academic potential; and that the most likely source of this individual academic potential is [edit] likely influenced by genes. When we look at academic performance, what we see again and again is that students perform at a given level relative to peers early in schooling and maintain that level throughout formal education. (I make that case at considerable length here.) A vast number of interventions thought to influence relative performance have been revealed to make no difference in rigorous research, including truly dramatic changes to schooling and environment. Meta-analyses and literature reviews that assess the strength of many different educational interventions find effect sizes in the range of .01 to .3 standard deviations, small by any standards and subject to all sorts of questions about research quality and randomization. Even the most optimistic reading of the research literature suggests that almost nothing moves the needle in academic outcomes. Almost nothing we try works.

This implies that common sense is correct and that individual students have their own natural or intrinsic level of academic potential, which we have no reason to believe we can dramatically change. I believe that we can change large group disparities in education (such as the racial achievement gap) by addressing major socioeconomic inequalities through government policy. But even after we eliminate racial or gender gaps, there will be wide differences between individual students, regardless of pedagogy or policy. When Black students as a group score at parity with white students, there will still be large gaps within the population of Black students or white or any other group you can name, and we have no reliable interventions to make the weakest perform like the strongest.

My book’s argument is attractive to teachers because they’ve lived under an educational ideology that insists that every student is a budding genius whose potential waits to be unlocked by a dedicated teacher – and which holds teachers to that unachievable standard. From the right, they’re subject to “no excuses” culture, the constant insistence from the education reform movement that student failures are the result of lazy and feckless teachers; from the left, they’re subject to a misguided egalitarianism that mistakes the fact that every child is important and deserves to be nurtured for the idea that every child has perfectly equal potential. The result is a system that presses teachers to deliver learning outcomes in their classrooms that they can’t possibly achieve. But many of them feel that they can’t push back, for fear of professional consequences. If they speak frankly about the fact that different students have different levels of individual potential, they’ll likely be accused of shirking their duty.

These teachers, in other words, live with the change that’s well-summarized by this old cartoon: the odd way that the expectation that students are responsible for their own grades became an expectation that teachers are responsible for those grades. And this evolution in cultural thinking about education comes straightforwardly from optimism bias. If teachers are to blame, then we can imagine a policy fix that will solve our problems. You can replace a teacher, after all. If we recognize that there is a natural (and roughly normal) distribution of academic ability throughout the population, and that teachers are therefore frequently blamed for conditions beyond their control… what sunny, simplistic story of redemption are we going to tell?

(You’ll notice that parents are very rarely indicted in these discussions, and for a simple reason: there is no policy fix to bad parenting, and thus no optimistic story about how we’ll rise above and make every child a genius. If all you have is a policy hammer, all you can see are policy nails.)

The optimism bias in education circles has several orthodoxies.

1.      Every student is capable of academic flourishing, and every time a student does not flourish, it must be the result of some sort of error or injustice.

2.      There is no such thing as an unachievable academic standard for any student.

3.      Arguments that these expectations are unrealistic and create perverse incentives are to be dismissed alternatively as excuse-making or as evidence of bigotry.

4.      Proposed interventions that might increase student performance are assumed to be effective no matter how many times they fail to demonstrate gains in research.

5.      Anyone who disagrees with this doctrine hates children, supports inequality, and doesn’t care about poor people.

For what it’s worth, I can refute the last point easily. My own perspective is that, precisely because everyone has a different level of academic potential and this level is not chosen or under the control of the individual, we should concentrate our efforts on building a far more redistributive social safety net rather than continuing to bash our heads against the wall in the classroom. Look at how remarkably effective Social Security is at bringing senior citizens out of poverty. Why hang our hopes on eradicating poverty and racial inequality on the entirely unproven mechanism of education when we could just give people money?

But the notion that education is the cure to our economic problems – an idea that’s been explicitly stated by every president of my lifetime – has been a defining assumption of American politics for a long time. Some people want to protect the free market and keep taxes low; some simply don’t believe we’ll ever summon the political will for mass redistribution. Either way, they cling to the idea that only education can save us. And once you’ve decided that, you have to enforce optimism because you can’t countenance the idea that schools and educators have limits.

Until the last half-decade or so, the dominant discourse in education had been the neoliberal “reform” movement, which identifies untalented and apathetic teachers as the source of academic gaps and charter schools and private school vouchers as the solution. This entire movement floats on optimism bias; the basic premises of its adherents are dependent on the idea that educational problems have technocratic fixes and all we need is people smart and tough enough to implement them. Unfortunately, they didn’t work; the teacher merit pay research is a mess, private school vouchers have a deeply discouraging research record, and whatever positive results charter schools have seen are very small compared to the hype and questionable thanks to non-random distribution.

It's hard to think of a better example of optimism bias than the fact that people still talk about an educational miracle in New Orleans, thanks to a switch to an all-charter system, when those charter schools are absolutely riddled with failure.

Optimism bias makes the enthusiasm for particular interventions immune to evidence. A perfect example is pre-K. Pre-K programs have been sold as our salvation going back as far as the 1960s. It’s still perfectly common to hear politicians talk about funding pre-K as a simple solution to racial and socioeconomic achievement gaps. What they leave out is that pre-K doesn’t work; over the course of the late 20^th century and the beginning of the 21^st, results of pre-K research got more and more discouraging. Just last year, a very large, randomized, longitudinal study found that the children assigned to the pre-K condition actually performed worse than those who didn’t. A similar dynamic can be found in class size research, where early promising results have given way to years of conflicting results and underwhelming findings. But it doesn’t matter how many decades of research demonstrate that pre-K or class size reductions are a dead end. Optimism bias means that you never have to say that you’re wrong and that anyone who disagrees with you just doesn’t care about kids.

Let’s take a look at the education culture warrior solution du jour, the influence of educational spending on educational outcomes. This is the hot intervention of the moment. There always is one, a popular idea for how to “fix” our schools that becomes the fashion. (Usually they fall out of fashion as proposed interventions stubbornly fail to close gaps, as has happened with teacher evaluation reforms.) For many years, it was broadly understood that past a certain minimal level of funding, school spending had little influence on student outcomes. This attitude wasn’t hard to understand: richer countries, states, school districts, and schools simply didn’t reliably outperform poorer. There is no observable relationship of that type. Regressing per-pupil spending on outcome metrics like SAT and NAEP and PISA scores consistently found no relationship. Research confirmed these obvious dynamics. Though you sometimes hear absurd complaints that the United States has “defunded” education, we’ve in fact spent profligately, trying again and again to fix our problems with cash, to no avail. For a long time, reformers talked about needing to do more than spend money.

But more recent studies have argued that there is in fact a relationship. Demonstrating the optimism bias in its purest form, many education pundits and reporters raced to declare the issue settled. For example, ChalkBeat’s Matt Barnum has produced years of relentlessly optimistic coverage of these studies, never stopping to ask whether we should temper our enthusiasm due to the decades-long dynamic where hype for a proposed intervention gradually dies out over time as more pessimistic results are publicized. He reports that the question is “essentially settled,” which is rhetoric I’ve heard about education before.

As others have pointed out, there’s basic questions of research methodology at play when it comes to these studies, which tend to use quasi-experimental designs with dubious randomization and frequently amount to a lot of statistical gymnastics to find the desired outcome. I’ll set the methodological concerns aside for others for now. There’s still a difficult question for these researchers to answer: if increasing educational funding improves educational outcomes… why hasn’t increased educational funding improved educational outcomes? We’ve thrown vast fortunes of money at our public school system over the course of many decades, with public K-12 education spending increasing from 2.3% of GDP in 1950 to 5.5% of GDP in 2015. We have quite famously not seen dramatic improvements in educational results in that timeframe. Why not, if school funding is dispositive? Utah typically ranks at or near the very bottom in per-pupil student educational spending, New York at or near the very top, and yet in NAEP scores New York barely outperforms Utah. What explains this phenomenon? Barnum offers a lot of optimistic handwaving but no clear answers.

The United States spends six and a half times per pupil what Vietnam spends, and yet Vietnam performs almost as well as the United States on international educational comparisons. What can explain this dynamic? What headwinds could the United States be facing that would account for essentially equal outcomes at 6.5x the costs? Ed reformers love to trot out poverty as the determiner of all educational data, but Vietnam’s internationally-normed poverty rate is 22.2%, while the figure is 1.7% for the United States. Why isn’t the American advantage in the supposedly-crucial funding metrics helping us to beat a vastly poorer country? And this lack of a relationship between expenditures and performance is found all over the international data. The United Kingdom spends a third again per-student what South Korea does and gets far worse results. You can’t just dismiss these consistent findings.

To the extent that they answer these questions at all, those who insist that educational outcomes are a function of school expenditures suggest that the answer is all about how the money is spent. The authors of one such study have argued exactly that. But this isn’t an answer at all: why would Vietnam, with far fewer resources, a much smaller academic sector, and a much poorer government have spent so incredibly shrewdly while the United States spends unwisely? What’s the special sauce that Utah enjoys in their spending patterns that New York has (for some bizarre reason) refused to adopt? “You have to spend the money wisely” is quickly becoming a cliché regarding this topic. But there’s no systematic, empirically-verified explanation of what spending “wisely” means; if there was, states would be doing it. And for the record, there has been no sudden improvement in learning metrics in the decade or so that this new research on expenditures and performance first started appearing.

This has all had the unfortunate effect of deepening ignorance about American school spending. We know, for example, that majority-Black and Hispanic schools receive significantly more per-pupil funding than majority-white schools. This fact is so contrary to basic liberal assumptions that they often react angrily to hearing it. But this reality shouldn’t surprise anyone. After all, we’ve been shoveling money at the racial achievement gap for 40 years, to no avail. Part of the problem here is an assumption that public education is dominantly funded by local expenditure, which hasn’t been true for some time. In fact, state funding is at or near parity with local spending in the United States, and state funding is heavily tilted towards areas of perceived need (that is, failing schools or districts). Federal funding, including but hardly limited to Title I funding, is also dominantly directed towards poor or high-minority schools. The rising tide of think tank and foundation money that finds its way into public K-12 school is very hard to track, but we can safely assume that almost all of it is earmarked for the poorest students. We’ve been trying to spend our way our of this problem since before I was born! And yet people who should know better pretend not to understand this reality and repeat the complaint about local funding of public schools, despite the fact that that story is not true.

Why does optimism dominate education discourse? Consider the incentives. Almost everyone within the space has direct professional reasons to be relentlessly positive. This is most obvious in the think tank world, where consistently pessimistic voices are quite rare; nonprofit institutions are funded by do-gooders who want to believe that they’re getting something for their money, and donors don’t want to hear bad news. The dynamic for academic researchers is similar, where positive results get published much more often then those that find no effect and where the political biases of academia make pessimism professionally fraught. And people who get into education journalism tend to be idealists who pursued the career path because they were attracted to pleasant and simplistic narratives of poor kids rising up from the ghettos and going to Harvard. Undergirding all of this is the social pressure that stems from the fact that no one wants to deny the potential of any kids to flourish. (You might say that most people want to ensure that no child is left behind.) But someone has to be the bad guy. Someone has to be there to say that the empire has no clothes. Someone has to defend teachers and schools by pointing out that they simply don’t control outcomes in the way they’re usually assumed to. And someone has to point out that the history of modern American education gives us every reason to be pessimistic, as time after time, hype has given way to sad reality.

In his State of the Union address in 2011, Barack Obama named the Bruce Randolph school in Colorado as an example of the positive power of education reform and the school choice movement. Later investigation revealed that Bruce Randolph students were meeting state standards at a rate of 15% in English and 14% in math. That was apparently an example of education reform so incredible that Obama thought it was worth mentioning in the most important annual speech in American politics. His rhetoric was deeply deceptive, but hey - he was telling the people what they wanted to hear.

The only place where education does not suffer from optimism bias is reality.