we've been throwing money at our problems for 40+ years
I am curious how much the "what" could be the culprit here: anecdotally, my mother (who teaches in a wealthy district) and my roommates (who teach in a very poor district) spend approximately the same amount of money on school supplies each year, but what they buy varies a lot. My mom buys "fun" items: stickers, fidgets, different kinds of furniture so kids have options as to how they want to sit/stand, games, science experiment kits, etc. My roommates buy basic supplies their kids can't afford: pencils, notebooks, folders, etc. In poorer districts, is a lot of money being spent just to get kids to the same place their richer peers are? Does stuff like free lunches/breakfasts get accounted for? I'd imagine those are expensive programs. Throw in school districts chasing fads like iPads or anti-racism (my roommate's poor district spent hundreds of thousands of dollars on DEI training), and you can easily get lots of variation in what is spent. Doesn't help you figure out what specifically is useful to spend on, as you noted, but it does seem to me like the most interesting variable.
I'm curious if the research has delved into potentially "invisible" money like how much parents and teachers spend out of pocket? If District A is spending $500,000 a year on tutoring and the _parents_ of District B are spending $500,000 on tutoring, District A might look like it's spending a lot more even though the spending per community is similar.
From first quick glance at the two papers that try to use causal inference methods, they look pretty good. So spending probably does help. That still leaves the question of what are the main headwinds.
I suspect that the big spending increase in the 1950-1970 period was mainly just due to the big reduction in female oppression. Educated women had been pretty much forced into teaching or nursing with few alternatives. That kept wages artificially low. Now they can be doctors or lawyers etc. Perhaps it's an ironical reminder of how much more we reward lawyers, MBA-managers, financial advisers, etc. than we do people performing essential functions like teaching.
There's still a question of why subsequent increases didn't show up in improved outcomes, and that's where the headwind issue comes up.
"despite decades of data from armies of researchers, no one really knows what kind of spending actually is strategic"
Surely no one seriously suggests that if we put more spending into armies of administrative bureaucrats, instead of the classrooms, we will get better results . . . ? Yet anecdotal evidence from discussions with current and former teachers suggests to me this is exactly where the increased spending is going (and of course, to jobs for the boys, which explains all those iPads).
(1) Educational spending is to a great extent spending on (a) real estate and (b) college-educated personnel, both of which are particularly expensive in urban areas. I therefore enthusiastically endorse Richard Gadsden's opinion elsewhere in these comments that school spending figures should be adjusted for PPP (purchasing-power-parity, basically local cost of living) in any analysis.
(2) Along those lines: in the early 1990s a business group in Massachusetts struck a deal with education reformers. There would be substantially higher spending on education AND there would be substantially more standardized testing including graduation requirements. The increased spending took the form of a "foundation budget": the state undertook to calculate a "foundation budget" that each school district would need to educate its student body, taking into account local cost of living (i.e., PPP) and the composition of the student body (students in poverty, disabled, English as a 2d Language, etc). Each school district was guaranteed a budget at least equal to 100% of its foundation budget.
This brought about a very substantial improvement in the quality of public K-12 on Massachusetts: the metrics in Massachusetts are much better than the rest of the US. If there is a best practice in public education finance it is the foundation budget model.
That said, funding disparities continue: if you look at only $/student, urban school districts are on a par with the wealthy suburban school districts, but if you look at spending as a % of foundation budget, the urban schools are at 100-105% of FB while the suburban school districts are frequently in the 120-160% range — the suburbs charge themselves more in local property tax so that they can spend more on their schools. The FB formula was revised in 2019 in an effort to narrow this gap, but it certainly has not been closed.
“The key assumption is that the exact timing of events is as good as random.” (Lafortune et al) Hah. My advisor would have drawn a big X on that.
There are so many “quasi” experiments in education. I wish some of these zillions of dollars had been spent on true experiments with actual random assignment. I know it’s hard to convince policymakers to allocate money in this way, but we’d know a lot more.
If education policy doesn’t impact outcomes, the difference must be what happens outside of school. Economic security, less trauma, parents helping with homework (having the time and skills). Selection, because parents with good jobs and resources GTFO of “bad” school districts. (All of my overeducated friends left the city for the suburbs when their kids turned 5.)
I thought the linked post from 2017 (where small-group tutoring has the biggest impact) was interesting. Perhaps this is the best way to compensate for lack of parental involvement. The other interventions with bigger effects (feedback and progress monitoring; small-group instruction) also seem to point to individual attention. I can imagine this having a greater impact than small class size, which still leaves a teacher with about 20 students.
I googled the CREDO project and the first result was the Cosmic-Ray Extremely Distributed Observatory that was looking for cosmic rays and dark matter. That, combined with your comment about being conspiratorial, and I thought, good god it’s happened. Break out the tinfoil. But, turns out, that was the wrong CREDO project and the wrong conspiracy.
>"I am on record as saying that the best bang for our buck may lie in small group tutoring."
Sure sounds right to me.
1. Regardless of the underlying cause there is a consistent and statistically significant disparity in IQ scores between different races. If you think that IQ naturally correlates with test scores/grades then it would be expected to see a disparity there as well.
2. "Culture" may be difficult to quantify but is it unreasonable to believe that models such as tiger parenting have an impact? In S. Korea or Japan it's not unusual for students to head off to cram school after regular schools hours for a further five or six hours worth of study before calling it a day.
I'm late to this, but wanted to say a few things.
First, I agree with Freddie that the research is probably correct and money makes no difference in academic outcomes. Whether or not money makes a difference in engagement, and whether engagement can make a difference in academic outcomes is a slightly different story that has been investigated, but possibly not well.
That said, a huge amount of money spent on urban high poverty schools is
*TItle I initiatives--a lot of money is given to low income schools, but it has steel hawsers tied to it and much of it gets wasted because you can't spend it on what you want and you don't need what you can spend it in.
*immigrant English classes--many Title I schools get asylum or refugee students right away, speaking no English, and they basically take up an entire teacher to kind of sort of learn English in class while they more effectively learn English outside the classroom--although that's getting less likely, given the language ghettos each nation has in most schools. Still, this is expensive.
*special ed funding--IDEA is a terrible law, but each kid designed sped costs on average twice as much as a non-sped. In practice, though, severely disabled kids--who should really not be in public schools--cost upwards of $100K/year, and there are a lot of these kids in Title I schools.
* in the case of majority black schools, job creation. As Dale Russakoff said in the New Yorker article that The Prize was developed from, "The ratio of administrators to students—one to six—was almost twice the state average. Clerks made up thirty per cent of the central bureaucracy—about four times the ratio in comparable cities. Even some clerks had clerks...."
None of these money buckets exist at high income schools except sped. So even though I agree that dollars aren't likely to make a difference, I tend to get snippety at people who say we spend a fortune on poor schools. School law is a horrific thing.
Not all low income schools fit the stereotype. Everyone tends to envision high poverty schools as only desperately poor, 80% low income, multi-generational welfare and usually black or Hispanic. But Title I starts at 35% poor. There are a lot of schools with a wide range of abilities and they could be, like mine, the only low income school in an otherwise rich district. It's impossible to visit these schools and compare them to their wealthier siblings in the same district and not see the inequities. Academic outcomes, maybe not. Opportunities? Different story. And it's not just the poor kids, but the non-poor kids in a Title 1 school. And as a teacher in a very diverse, mixed ability school, it's impossible not to see the contortions we're forced into because we have a 3-4 year ability range in non-honors classes.
Next, people who say this is about the disappearance of high ability female teachers are wrong twice. First, they're wrong in assuming that the average ability of teachers has declined. It has been roughly constant or slightly increased. Teachers, particularly high school teachers, are far smarter than the general public assumes. The credential tests keep out a lot of would be teachers. Then they're wrong in assuming that smarter teachers = better student outcomes. Research has demonstrated consistently that race is the best correlate with improved academic outcomes, and test scores being what they are, high cut score credential tests are leaving out black teachers who might work better with black kids. (If you're going to cite the 20-30 year old study about how math teachers with master's degrees in math do better teaching honors kids, don't bother. The difference was small, and the ability range of kids coming into advanced math classes these days is far and away wider than it was back then).
States have noticed this, and right now some are in the process of eliminating credential tests. This is a bad move, generally, but reducing the demonstrated cognitive proxy requirements is probably a good move.
On "smart" teachers--as someone who is kind of a useless genius (I qualified for the genius DNA study a few years back), I can say categorically that my brain wattage gives me a huge advantage when I work on ways to teach low ability kids. The average smart person who goes into teaching is not interested in this particular skill. That person wants to teach smart kids, and is really disillusioned at what goes in to teaching lower ability kids.
About the only thing I disagree with Freddie on is the utility of small group tutoring. This is useful for two situations:
1) Motivated kids who just need more help understanding the material. This is an *incredibly* small group. Most teachers pitch their difficulty at the average for the class, so any motivated kid can succeed. The most likely place this tutoring might help is in a socioeconomically diverse group with obnoxious honors teachers who like to brag about how hard their classes are, and don't mind failing kids for stupid reasons. That's....not a big problem. But it would be useful in cases where SES and racially diverse schools are under pressure to have more Hispanics or blacks in honors courses, so they put them in these courses only to have them get Fs, thus screwing up their GPA. So instead of challenging them appropriately and upping their game, ruin their careers.
2) Unmotivated, low ability kids needs to finish his homework and understand the material to the best of his ability to take tests seriously. That is, he won't get to proficient but he'll know enough to show what he knows on quizzes and tests (state and otherwise). This will improve his grade, he'll fail less often, and so on. Those all save the state money and might bump the test score bottom up.
But in neither case will tutoring actually improve the underlying ability of students from below average, non-functional to proficient. At best, take a basic and bump to proficient or far below basic not even trying to a functional 2.0 GPA. With the occasional miracle kid to keep us all happy.
So what would work, and would money help?
1) In my opinion, the single best thing to improve learning outcomes in high school (the only place they matter) is to stop restricting curriculum and allow classes to teach kids where they are. This is basically what we did in the 80s and got sued out of it due to disparate impact. I don't see this ending any time soon. Along with this, we should have tiered diplomas.
2) A big problem is we keep focusing on elementary school, in the delusion that improving k-6 scores will lead to higher HS scores. 30 years of flat HS NAEP scores may possibly have finally worked to convince people this is fool's gold. But Common Core and other "let's make K-6 more difficult through abstraction" efforts have set us back quite a bit.
3) So long as colleges accept all students, very little is going to change. Congress needs to get serious about putting restrictions on Pell grants, and states need to get serious about using taxpayer dollars to put illiterate students through college. The end of college admissions test is a disaster that's going to even further obliterate the value of the college degree signal. But this, too, will be difficult to achieve because of disparate impact.
In closing, before you make a single suggestion to fix public schools, ask yourself if it will have a disparate impact on the student population--that is, the racial balance will be out of whack. Abandon that idea because you'll get sued out of it. And if you think improving teachers or curriculum will achieve anything, you're in the wrong game.
What's that leave? What we have. Which isn't as bad as people think. Not as good as I'd like, of course.
I have lots of articles at my blog on this stuff.
Wonder when we'll finally figure out that funding should follow the child, not default to the administrators?
The work of Jonathan Kozol over the years has framed the public sphere of educational funding starting with his __Death at an Early Age: The Destruction of the Hearts and Minds of Negro Children in the Boston Public Schools__ (1967). He is still active: https://www.jonathankozol.com/about
I wonder what it looks like if you use some version of PPP dollars rather than nominal ones. Probably not all that different, though it might explain New York's high expenditure, for instance. In the city, some, at least, of that money is going towards higher real estate prices (schools need quite a lot of land) and compensating teachers for the higher cost of living, neither of which you would expect to convert into better teaching.
IIRC, if you measure salary relative to the state/district/territory mean, then DC has the lowest teacher's salaries in the US. This is because DC has so many well-paid people, not because their teachers are paid badly.
My bigger thought, though, is how much is more spending "pushing on a string". That is, there is an upper bound in school performance set by spending, but increasing spending does not push a school system that is not close to the upper bound forward at all, because that system doesn't know what it's doing when spending the money it has.
It’s almost as if academic ability is innate and can not be meaningfully improved.
Just curious, in this video from the NY Times they show the example of Cook County Illinois as an example of how wealthier districts DO benefit from keeping all of their educational dollars local and do exhibit the predictable result (better facilities, more money to spend per student).
Does this contradict any of the claims you've made here?
Thank you for a great, thoughtful post.
"Neither Evans nor either paper can explain what aspects of higher spending actually provide efficient returns, or what this causal relationship actually looks like in simple real-world terms, although Jackson offers useful supposition."
From the second linked paper:
1. Table 4 suggests low-income districts by and large used extra $ on (i) capital outlay and (ii) hiring more teachers (and in consequence, decreasing class size.)
The authors make no comment, to Freddie's point, on relative efficiency of these interventions, only that, bundled together, they seemed to have an effect on kid learning:
"It is important to note that our research design is poorly suited to identifying the optimal allocation of school resources across expenditure categories ... It allows us only to say that the average finance reform... led to a productive (though perhaps not maximally productive) use of the funds."
2. I sort of wonder whether this mix of stuff -- more teachers, smaller class sizes, better and safer buildings -- doesn't really need to be optimally efficient so long as benefits > costs. All of these reforms are politically popular.
If that can get you the modest achievement bump that the authors indicate it did on NAEP, you perhaps take that deal. On cost/benefit, the authors make a (qualified) economic case for doing so:
"An increase of $424 per pupil in spending each year from kindergarten through grade 8 .... implies a benefit-cost ratio of 1.5, even when only earnings impacts are counted as benefits... Although these sorts of calculations are quite imprecise, the evidence appears to indicate that the spending enabled by finance reforms was cost-effective..."
I don't really know what to make of all this. Hard problem, and again great post describing it.
3. Separately very interested in Vietnam. RISE has a whole team dedicated to studying what they're doing:
You are intelligent and well-read, but when it come to the visual presentation of data—well, supply your own adjective. As someone who wrote about NAEP for a living for 19 years, I will point out that the average score you give for DC is about 480, while that for Massachusetts is about 570, a difference of about 90 points. In other words, Massachusetts’ score is 20% higher than DC’s. But your graph “shows” that Massachusetts’ score is 900% higher than DC’s. You should know better.
In addition, you pretend that you do not know that the student is the most significant variable affecting student performance and that therefore the endless attempts to erase subgroup differences reflecting race/ethnicity are simply an excuse for the education industry to fund itself. Differences in performance among these subgroups is the result of culture rather than genetics, but they are not easily erased.