At the Other End of Your Educational Technology There's a Student's Brain

and all brains are not the same

Let me start with my usual caveat: all human beings have strengths and something to contribute. Even if we didn’t, all of us would deserve equal rights, respect, and dignity. However, not all human strengths are of the same type, and only some of us are lucky enough to have the kind of academic skills that are rewarded under 21st-century capitalism. And my position has always been that, instead of continuing to try to force square pegs into round holes, we should expand our definition of human value and shape the kind of society where everyone is given an opportunity to flourish according to their strengths. You’re either lucky enough to be born in a time period where your cognitive strengths are economically rewarded or you’re not, and that dependence on chance is not a rational way to build an economy. I wrote a book about it, which you should read.

That out of the way… there’s a mountain of “artificial intelligence is going to solve our educational crisis” stuff going around right now, for three principal reasons. One, establishment media is stuck in the Official Dogma of Education, which insists that because we say we have an education crisis, there must be a policy fix that can solve it. (The tail always wags the dog when we talk about education, in that way.) Two, there’s a ton of new edtech startups that are using the big AI freakout to rake in VC cash. Three, neither of those groups (establishment media or startup leeches) has any grounding in the educational research record. So you’ve got a bunch of credulous rubes in the media who are eager to hear that, this time, revolutionary educational promises are going to come true, where they’ve failed again and again and again; you’ve got a lot of cynical opportunists looking to soak up some government cash with dubious technology, like when the Los Angeles school district spent over a billion dollars on iPads that did nothing to change student outcomes; and you’ve got people like Marc Andreessen who make their living by delivering outlandish claims about The Future and how Everything is About to Change and Nothing Will Ever Be the Same. Thus this current misguided moment.

You have, for example, the founder of Duolingo Luis van Ahn profiled by a too-credulous reporter, in a moment in which it appears to be totally outside of the realm of possibility for mainstream media to express skepticism towards AI. And the overwhelming optimism bias of educational media pervades; the New Yorker profile compares the notion of Duolingo teaching poor people in Guatemala to the distribution of free tablets as an educational venture and calls the results of such efforts “mixed.” There’s no citation, which is convenient because there’s no credible way to argue that those tablet programs have ever achieved anything meaningful at all. We’ve known for a decade that putting computers in the homes of children does nothing for their educational outcomes. Why on earth would it? Most of them are going to get those computers and watch YouTube for hours. What planet are you living on where you think that giving kids tablets will see them trawling Wikipedia for knowledge? Have you met any kids? They’re going to use them to play fucking Candy Crush! Because there’s a human being at the other end of that technology. Most people aren’t self-motivated autodidacts and never will be. And this is the problem with thinking Duolingo or ChatGPT or any other such system is going to “fix” education: such a belief requires removing the human side of a human process. It makes no sense.

The New Yorker piece does refer to the various social problems that can depress a country’s educational performance, such as in Guatemala, van Ahn’s home country. This is worth underlining. If you put Duolingo in the hands of every Guatemalan, you still have a country that’s wracked with poverty, instability, and the continuing legacy of American-back atrocities. How does van Ahn think Duolingo can change that? No doubt he doesn’t. But why then is he so sure that his software can educate people with “very high quality” while living under adverse environmental conditions? What if that’s not possible?

More importantly: has it occurred to van Ahn that, even if we somehow equalized the environmental impacts that drag a country like Guatemala down, the individual people in that country and everywhere else have very different academic potentials, meaning that their education system can’t actually be “fixed”? Even if we believe we can achieve major absolute learning goals at scale by means of this new technology, a) countries ahead of Guatemala currently will have access to the same technology and thus retain their advantage, and b) the distribution of ability within Guatemala will still contribute to socioeconomic inequality. As I have been saying for many years now, even if you were to improve everyone’s academic abilities equally and significantly, the gap between the haves and the have-nots would remain. I point to this dynamic all the time, and I’m always surprised by people who struggle to understand it. It’s why the racial achievement gap is so stubborn, for example, as improvements to Black scores are matched by improvements by white students, resulting in a kind of treadmill. If transformative educational technology actually exists, it will help everyone, and no gaps will close. And you probably can’t actually achieve massive leaps in absolute performance anyway, given that educational technology has proven to be a boondoggle over and over again.

---

Let’s get some things out of the way in handy bullet point form.

• There is no “educational crisis” in the United States. Our median student does pretty well. Our top-performing students are among the best in the world. Our averages are dragged down by a relatively small number of students who perform truly terribly, who are largely concentrated in very poor high-minority urban school districts (think Detroit) and in largely-white rural school districts (think he Ozarks). (The students and their families are poor, not the schools.) These students are plagued by large-scale structural social problems that ed tech types have no plan for addressing.

• Any honest accounting of the general state of education across contexts must start with the observation that students sort themselves into an ability band very early in life and maintain that ability band with remarkable fidelity, such that data collected in the earliest days of schooling provides useful predictive information about summative outcomes like the likelihood of going to college. In some metrics, the stability of relative outcomes increases over the course of life. This is true even in the face of major interventions designed to change relative placement in the performance spectrum. Here is a massive amount of research summarizing research that demonstrates this dynamic.

• This stability in relative performance despite vast educational interventions and environmental changes strongly suggests that there is some sort of stable and endogenous property that we might call “academic potential.” If there was no endogenous element of academic performance, only exogenous variables (as some liberals and leftists suggest), then students from very similar environments would reliably perform very similarly; in fact, students who match on a remarkable number of demographic variables often have very different academic outcomes. If there was no exogenous influence of environment on academic performance, only genes (which essentially only strawmen suggest), then there would be no consistent correlations between various environmental variables and student outcomes; there are.

• This property we call academic ability is thus almost certainly the product of gene-environment interactions, like most human behavioral outcomes.

• The fact that a trait has genetic influence does not mean that it cannot be modified; however, the fact that a trait has environmental influence does not mean it can be modified. (See, for example, our inability to ameliorate the effects of lead on cognition.) Whatever the specific admixture of genes and environment, the reality is that summatively, students just don’t move around in the performance spectrum much, as explained above.

• Given all of this, the tacit assumption that underpins the philosophy of people like van Ahn - that the source of academic gaps is poor teachers or schools - cannot withstand scrutiny. To the extent that it exists, teacher quality is likely of small effect and extremely contextual and interactive with individual student learning styles. There is no such entity as “school quality.”

As I’ve quoted before, this paper relays the basic reality effectively:

Over the last 50 years in developed countries, evidence has accumulated that only about 10% of school achievement can be attributed to schools and teachers while the remaining 90% is due to characteristics associated with students. Teachers account for from 1% to 7% of total variance at every level of education. For students, intelligence accounts for much of the 90% of variance associated with learning gains.

So here’s the question for everyone saying that AI is going to revolutionize teaching: if we estimate that something like 90% of the variance in educational outcomes is student-side - that is, reflects some stable quality of individual students rather than their schools, teachers, or curriculum - how much difference can AI really make? If 10% is our estimate of what’s controlled, we can’t expect the delta of AI over actual teachers to be anything but single digits, and unless you think human teachers are adding literally zero value, we would expect it to be in the low single digits. Does that match the soaring horizons of the New Yorker piece? But it gets worse when you consider all the things that teachers do that aren’t matters of quantifiable education metrics. Teachers help students with socialization, make them feel welcome and safe, establish discipline, interact with parents, report potential neglect or abuse to social services, and do all manner of other essential tasks, for low money and a lot of disrespect in our media. Duolingo can’t do any of that. And if we were ever to actually pay attention to the evidence and reorient our education system - away from achieving learning gains that have never been achieved in the history of schooling and towards nurturing individual interests and strengths - then the value of kind and patient human teachers grows even more and the value of (thus far purely theoretical) AI teachers falls further.

As is always the case, it’s essential to point out that people don’t know what they want when it comes to education. This huge mountain of discourse is perched precariously on a set of intuitive notions about what education is for that can’t bear scrutiny. Most of the rhetoric is uselessly vague, stuff like “every kid can learn!” If the goal is for most everyone to gradually get smarter over time, congratulations, that’s been happening for at least a century and is almost certainly because of better living conditions and the endowments of modernity rather than pedagogical improvements. If the goal is to create a period of great educational mobility where the first shall be last and the last shall be first, we have no reason at all to believe that any such thing is possible, and anyway moving people around an unequal hierarchy does not make that hierarchy equal and does not change the fact that people at the bottom of the hierarchy will suffer for it. If the point is to achieve proportional racial representation along the performance spectrum such that there’s about as many white/Black/Hispanic/Asian students in the top 1% or 20% or 50% as there are in the population as a whole, you’re still left with the moral challenge that the bottom 50% or 20% or 1% are going to face a great deal of personal hardship for that status. If the point is to achieve equality in educational outcomes, well, good luck with that, but also doing so would necessarily eliminate the possibility of excellence and necessarily destroy the advantage of being educated in the first place, which is a zero-sum advantage.

Now, if you were really invested in educational “equality,” then you’d be someone who supports strong redistributive policies and probably socialism, and a lot of these ed reform warriors conspicuously aren’t. More, though, you’d have to be willing to traffic in some sort of widescale genome manipulation. At the crude end, what’s available now is eugenics, not in the “everything is eugenics” sense when that was the bad word of the month, but actual eugenics; that is, selective breeding of the intelligent at the most benign and forced sterilization of the unintelligent at the most drastic. This is, to put it mildly, not morally defensible. And it also wouldn’t eliminate academic variation entirely, even if we controlled environments to a degree that exceeds the plausible. The other stab at mass change to the educational ability distribution we could (and someday probably will) take is through direct genome editing. We’re nowhere near the capability to do that now, and when it arrives it will certainly be unequally distributed based on socioeconomic class, and it comes with all the usual Gattaca moral concerns as well. But on a long enough timescale, I think it’s inevitable that humans will be editing the genome to try and improve cognitive outcomes. This almost certainly will be used to exacerbate existing inequality, in the short term, but maybe someday we’ll gene-edit our way to true equality of human ability, which would alter our basic moral and political condition so entirely we really have no idea how to imagine it.

If I give you two different sets of construction materials and tell you to build the exact same house with each, you’d say I was crazy. That’s what people are trying to do when they say that educational technology (or a new pedagogical technique or a new curriculum or a new kind of school or whatever else) will solve our educational problems. How is it that we’ve come to the point where the dominant discourse on education completely ignores the reality of different students having different brains? I don’t know. I do know that guys like Luis van Ahn don’t really know what they want and can’t actually achieve what they say - but I’m also sure they’ll get rich anyway.