This was a great read. Unfortunately, it seems very rare to address counter arguments in good faith as is done here.
I would propose one addition to #3: SES correlates with academic performance and parental academic performance is also correlated with household SES. There's so much noise here, but I would suggest three things are going on here among the elite: an advantage in material resources, likely a stronger culture of valuing education and some hereditary inheritance of ability. The last point (hereditary inheritance) seems the hardest to talk about (I can feel the thunderclouds forming over my head). It also suggests that there will be some limit in generational economic mobility, i.e. the economic quintile you were born into will almost always be correlated with where you wind up no matter how much redistribution you have.
Here's my technical case along the same lines for the GRE for physics grad school:
Here's some of my work that bolsters your case, although it concerns GRE's rather than SATs. The common theme is the extraordinary lengths of embarrassing pseudo-scientific foolery to which the anti-test types resort.
My one quibble is that I think you've sort of muddled the description of collider stratification bias. It's distinguished from simple range restriction because unlike simple range restriction the stratification induces a correlation between the suspected cause and the other variables with which it collides on the stratified variable. If those other variables are outside the model, that typically induces a negative bias in the coefficient estimate.
I think people would be less wary to talk about inherent differences if so many people didn't use them as an excuse to further marginalize minorities. For example, I've seen people argue that it's perfectly reasonable to defund schools in poor neighborhoods because students there have lower academic achievements. Now, I'm not saying you or anyone else who agrees with your point is necessarily in this camp, but it seems unfair to go on about how stupid liberals are being without acknowledging there just might be an actual reason for their attitude. I don't even think the liberals are right about this and frankly, I think the whole education system is broken, I also don't think factors like that should be omitted.
This is a very minor point so I won't go too in depth but as someone currently studying in a STEM field, I can tell you it really does change how you view the world even of it's not what it aims to do. Maybe the CA DoE's incentives are weird but I see teaching math and science as being less about regurgitating facts and applying formulas and more about exploration to be a good thing. It's way closer to what actual scientists do and I belive it does more to encourage students' interest in those fields (I mean, those who aren't already passionate about differential math at age 12 :P). I for sure wasn't as passionate about it until I started my study and realized how interesting it actually is when it's less about memorization.
I don't think you're actually wrong about what the best approach to schooling is, but some arguments you make just bother me and I don't think they really support your cause.
I like this post. I can't say for sure, and you touch briefly on meritocracy in the post, but I sometimes wonder if the resistance to the SAT and GRE is partly because of academia's view of itself as a meritocracy is something like a fundamental belief. As you know, it is not and has probably never been a meritocracy, but when discussed the lack of "true" meritocracy is usually stated as due to external factors preventing it (student wealth, parental connections, bureaucratic reasons, etc.) and only very rarely as due to anything academia itself is doing wrong. To see the SAT and GRE as having validity means that increasing diversity implies lower-ability students are intentionally being admitted over higher-ability students, which poses an existential question to academia even though, as you exactly stated, an academic ability gap does not imply an academic potential gap. Unfortunately, there's not really anything academia can do to erase all the sociocultural bullshit that kids have gone through their past 17+ years driving the ability gap. Which is all just to say, that academia wants to admit a more racially diverse student body (which is a good thing!) but the test results showing an ability gap, if valid, clash with their fundamental self-perception as a meritocracy, hence the tests showing must be invalid in order to preserve the self-perception of meritocracy.
Of course, I'm speculating about the beliefs of wide swaths of people which is always wrong, because the swaths don't believe things, people do.
My view on gifted and talented programs is that they don’t teach smart kids math or writing or whatever but instead they teach effort. I can only speak for myself but I basically sleep-walked through K-12 and undergrad and my masters program. I briefly went off to a masters at UChicago and just did horribly because it was actually vaguely hard and I had basically never put in effort academically at any point in my life. Gifted and talented programs in theory could help kids like that put in effort by forcing them to from a young age instead of coasting to success. This matters because a lack of effort certainly limits your high-end educational and professional out outcomes.
One major thing that I think gets ignored by pretending that some students don't have more ability than others is the damage it does to students and their upward mobility.
Letting a student in who is bright but not overwhelmingly above average into a very rigorous school is a great way to increase their chances of dropping out of college dramatically while at the same time possibly strapping them with student loan debt. The same student may have done very well at a less rigorous school and completed their degree.
The goal of whistling past the graveyard w/r/t to the SAT's ability to predict student success seems to be to create more diverse student bodies with the apparent end goal of improving the outcomes of minority students, but it will almost certainly end up doing the reverse in many cases.
"And throwing money at our educational problems, while noble in intent, hasn’t worked. (People react violently to this, but for example poorer and Blacker public schools receive significantly higher per-pupil funding than richer and whiter schools, which should not be a surprise given that the policy apparatus has been shoveling money at the racial performance gap for 40 years.)"
I appreciated this piece but I have mixed views on this point: I don't disagree that the studies show what they show in terms of the marginal dollar but in my view to really move the needle you'd need to spend a good deal more money. In Massachusetts, where I live, urban schools that serve Black students are much better funded than in years past, but they still have (i) decrepit physical plants, (ii) no really comprehensive afterschool or summer programming, (iii) school breakfast and lunch programs that are slightly better than prison food, etc. Those things could be fixed, and they should be, but that would cost thousands of dollars per student per year and those investments have not yet been made.
Instead of abolishing the SAT, Kendi and others should focus on abolishing Admissions. Open admission schools particularly in STEM fields would given everyone a chance.
Instead of abolishing the SAT, Kendi and others should focus on abolishing Admissions. Open admission schools particularly in STEM fields would given everyone a chance.
Confession: I haven't yet read your book, so you may have already addressed this, but I'm stymied by how we can encourage both academic achievement and other forms of achievement as equally valid and valuable in the current educational system.
How do we administer tests to sort students, which I don't necessarily think is useless, without damaging the self-worth of students who do poorly, or even just not as well, as the highest achieving test takers?
I guess that's one of those "liberal" values that are panned. But isn't a positive view of self and what one can contribute a key ingredient in achievement? So how can we shift our education model to be able to identify the academically gifted students and set them on their path while offering equal enthusiasm and dignity to students who would do better and be more fulfilled on a path that doesn't include calculus?
My brother and I are great examples of differing skills. School came easily to me; it didn't for him. Part of it was simply down to interests. He has great interpersonal skills, he was a fantastic after-school-care giver to elementary kids, he was a skilled athlete, and he's great at his current job doing tech support exactly because he doesn't talk to people like they're idiots. That's valuable. But he wasn't taught that his skills were valuable in school in a meaningful way, because school only offers The One Way.
Seems like an awfully big assumption that gifted and talented kids will be fine "no matter what." Cohorts matter, boredom can have serious consequences, and it's not as if there's data on the kids who could have but didn't get into programs. And after all we're talking about the tails of the ability distribution here, it's in society's best interest to make sure those people are found and supported. The assumption almost seems to directly oppose the rest of your post - if individuals have varying natural abilities, AND environment matters, then doesn't it make sense to find the people with greater abilities and improve their environment?
I'm mad at the fact that my children's school includes a class on getting better scores on the ACT.
How messed up are one's incentives that a public school has to offer tutoring on a standardized test (as opposed to, say, a statistics class or a literature class or literally ANYTHING else)? What possible, possible social good is that going to further?
This post is right on point. I have worked in the entertainment and restaurant publishing industry for the past 40 years and what makes those industries tick are people with exceptional talent. And not only on the creator side. It takes exceptional talent and a lot of practice to determine what is going to be a successful recording or to taste the food of the chef that has the best chance to make an important contribution to the craft. What I find surprising about the current trend in universities that you describe, is that only people who do not have real life experience of investing their own money in a business could believe that exceptional talent doesn't exist. But I assure you that if you could force Ibram X Kendi to use his own money to try and pick hit rap records, his worldview will change pretty quickly.
But your article also makes me wonder about the following that is a bit difficult to articulate. There was a time when having an Ivy League education was a rarified experience. In fact there was a time when simply having a college degree was rarified. No longer. A college degree is so common, and so many people have degrees from top universities, including advanced degrees, that their value has been diminished. And I am wondering if we need to create a more rarified educational experience for the exceptionally talented than the ones we can offer today. How that manifests itself is above my pay grade. Because as you say, it is the employers who want the educational system to do sorting of students to make it easy for them to identify potential candidates. But maybe the time has come for Microsoft & Google etc. to get into that business.
"an attempt to shut down those structures that make educational inequality visible" - partly to avoid uncomfortable conversations as you say, but also middle-class self interest: "Someone Has To Fail" by David Labaree, HUP 2012.
As someone making hiring decisions, if I see someone who got perfect grades at a noncompetitive regional state school, I consider that person as impressive as someone who got good-but-not-perfect grades at a highly selective school. Grades are a way for those who went to North Dakota State to be able to compete with those who went to Duke. Ending assessment may flatten the range of outcomes within an institution (though I'm skeptical of that), but it will certainly exacerbate the range of outcomes between institutions.
This was a great read. Unfortunately, it seems very rare to address counter arguments in good faith as is done here.
I would propose one addition to #3: SES correlates with academic performance and parental academic performance is also correlated with household SES. There's so much noise here, but I would suggest three things are going on here among the elite: an advantage in material resources, likely a stronger culture of valuing education and some hereditary inheritance of ability. The last point (hereditary inheritance) seems the hardest to talk about (I can feel the thunderclouds forming over my head). It also suggests that there will be some limit in generational economic mobility, i.e. the economic quintile you were born into will almost always be correlated with where you wind up no matter how much redistribution you have.
Excellent article.
Here's my technical case along the same lines for the GRE for physics grad school:
Here's some of my work that bolsters your case, although it concerns GRE's rather than SATs. The common theme is the extraordinary lengths of embarrassing pseudo-scientific foolery to which the anti-test types resort.
Princeton Merton seminar: https://www.youtube.com/channel/UCHEvLUxTWGsAjNjR3epRiQw
arXiv paper with references to earlier technical publication in Science Advances: https://arxiv.org/abs/1902.09442
discussion on Andrew Gelman's blog : https://statmodeling.stat.columbia.edu/2020/12/14/debate-involving-a-bad-analysis-of-gre-scores/
My one quibble is that I think you've sort of muddled the description of collider stratification bias. It's distinguished from simple range restriction because unlike simple range restriction the stratification induces a correlation between the suspected cause and the other variables with which it collides on the stratified variable. If those other variables are outside the model, that typically induces a negative bias in the coefficient estimate.
I think people would be less wary to talk about inherent differences if so many people didn't use them as an excuse to further marginalize minorities. For example, I've seen people argue that it's perfectly reasonable to defund schools in poor neighborhoods because students there have lower academic achievements. Now, I'm not saying you or anyone else who agrees with your point is necessarily in this camp, but it seems unfair to go on about how stupid liberals are being without acknowledging there just might be an actual reason for their attitude. I don't even think the liberals are right about this and frankly, I think the whole education system is broken, I also don't think factors like that should be omitted.
This is a very minor point so I won't go too in depth but as someone currently studying in a STEM field, I can tell you it really does change how you view the world even of it's not what it aims to do. Maybe the CA DoE's incentives are weird but I see teaching math and science as being less about regurgitating facts and applying formulas and more about exploration to be a good thing. It's way closer to what actual scientists do and I belive it does more to encourage students' interest in those fields (I mean, those who aren't already passionate about differential math at age 12 :P). I for sure wasn't as passionate about it until I started my study and realized how interesting it actually is when it's less about memorization.
I don't think you're actually wrong about what the best approach to schooling is, but some arguments you make just bother me and I don't think they really support your cause.
I like this post. I can't say for sure, and you touch briefly on meritocracy in the post, but I sometimes wonder if the resistance to the SAT and GRE is partly because of academia's view of itself as a meritocracy is something like a fundamental belief. As you know, it is not and has probably never been a meritocracy, but when discussed the lack of "true" meritocracy is usually stated as due to external factors preventing it (student wealth, parental connections, bureaucratic reasons, etc.) and only very rarely as due to anything academia itself is doing wrong. To see the SAT and GRE as having validity means that increasing diversity implies lower-ability students are intentionally being admitted over higher-ability students, which poses an existential question to academia even though, as you exactly stated, an academic ability gap does not imply an academic potential gap. Unfortunately, there's not really anything academia can do to erase all the sociocultural bullshit that kids have gone through their past 17+ years driving the ability gap. Which is all just to say, that academia wants to admit a more racially diverse student body (which is a good thing!) but the test results showing an ability gap, if valid, clash with their fundamental self-perception as a meritocracy, hence the tests showing must be invalid in order to preserve the self-perception of meritocracy.
Of course, I'm speculating about the beliefs of wide swaths of people which is always wrong, because the swaths don't believe things, people do.
My view on gifted and talented programs is that they don’t teach smart kids math or writing or whatever but instead they teach effort. I can only speak for myself but I basically sleep-walked through K-12 and undergrad and my masters program. I briefly went off to a masters at UChicago and just did horribly because it was actually vaguely hard and I had basically never put in effort academically at any point in my life. Gifted and talented programs in theory could help kids like that put in effort by forcing them to from a young age instead of coasting to success. This matters because a lack of effort certainly limits your high-end educational and professional out outcomes.
One major thing that I think gets ignored by pretending that some students don't have more ability than others is the damage it does to students and their upward mobility.
Letting a student in who is bright but not overwhelmingly above average into a very rigorous school is a great way to increase their chances of dropping out of college dramatically while at the same time possibly strapping them with student loan debt. The same student may have done very well at a less rigorous school and completed their degree.
The goal of whistling past the graveyard w/r/t to the SAT's ability to predict student success seems to be to create more diverse student bodies with the apparent end goal of improving the outcomes of minority students, but it will almost certainly end up doing the reverse in many cases.
"And throwing money at our educational problems, while noble in intent, hasn’t worked. (People react violently to this, but for example poorer and Blacker public schools receive significantly higher per-pupil funding than richer and whiter schools, which should not be a surprise given that the policy apparatus has been shoveling money at the racial performance gap for 40 years.)"
I appreciated this piece but I have mixed views on this point: I don't disagree that the studies show what they show in terms of the marginal dollar but in my view to really move the needle you'd need to spend a good deal more money. In Massachusetts, where I live, urban schools that serve Black students are much better funded than in years past, but they still have (i) decrepit physical plants, (ii) no really comprehensive afterschool or summer programming, (iii) school breakfast and lunch programs that are slightly better than prison food, etc. Those things could be fixed, and they should be, but that would cost thousands of dollars per student per year and those investments have not yet been made.
Instead of abolishing the SAT, Kendi and others should focus on abolishing Admissions. Open admission schools particularly in STEM fields would given everyone a chance.
Instead of abolishing the SAT, Kendi and others should focus on abolishing Admissions. Open admission schools particularly in STEM fields would given everyone a chance.
Confession: I haven't yet read your book, so you may have already addressed this, but I'm stymied by how we can encourage both academic achievement and other forms of achievement as equally valid and valuable in the current educational system.
How do we administer tests to sort students, which I don't necessarily think is useless, without damaging the self-worth of students who do poorly, or even just not as well, as the highest achieving test takers?
I guess that's one of those "liberal" values that are panned. But isn't a positive view of self and what one can contribute a key ingredient in achievement? So how can we shift our education model to be able to identify the academically gifted students and set them on their path while offering equal enthusiasm and dignity to students who would do better and be more fulfilled on a path that doesn't include calculus?
My brother and I are great examples of differing skills. School came easily to me; it didn't for him. Part of it was simply down to interests. He has great interpersonal skills, he was a fantastic after-school-care giver to elementary kids, he was a skilled athlete, and he's great at his current job doing tech support exactly because he doesn't talk to people like they're idiots. That's valuable. But he wasn't taught that his skills were valuable in school in a meaningful way, because school only offers The One Way.
Seems like an awfully big assumption that gifted and talented kids will be fine "no matter what." Cohorts matter, boredom can have serious consequences, and it's not as if there's data on the kids who could have but didn't get into programs. And after all we're talking about the tails of the ability distribution here, it's in society's best interest to make sure those people are found and supported. The assumption almost seems to directly oppose the rest of your post - if individuals have varying natural abilities, AND environment matters, then doesn't it make sense to find the people with greater abilities and improve their environment?
I'm mad at the fact that my children's school includes a class on getting better scores on the ACT.
How messed up are one's incentives that a public school has to offer tutoring on a standardized test (as opposed to, say, a statistics class or a literature class or literally ANYTHING else)? What possible, possible social good is that going to further?
This post is right on point. I have worked in the entertainment and restaurant publishing industry for the past 40 years and what makes those industries tick are people with exceptional talent. And not only on the creator side. It takes exceptional talent and a lot of practice to determine what is going to be a successful recording or to taste the food of the chef that has the best chance to make an important contribution to the craft. What I find surprising about the current trend in universities that you describe, is that only people who do not have real life experience of investing their own money in a business could believe that exceptional talent doesn't exist. But I assure you that if you could force Ibram X Kendi to use his own money to try and pick hit rap records, his worldview will change pretty quickly.
But your article also makes me wonder about the following that is a bit difficult to articulate. There was a time when having an Ivy League education was a rarified experience. In fact there was a time when simply having a college degree was rarified. No longer. A college degree is so common, and so many people have degrees from top universities, including advanced degrees, that their value has been diminished. And I am wondering if we need to create a more rarified educational experience for the exceptionally talented than the ones we can offer today. How that manifests itself is above my pay grade. Because as you say, it is the employers who want the educational system to do sorting of students to make it easy for them to identify potential candidates. But maybe the time has come for Microsoft & Google etc. to get into that business.
Excellent piece! I was lucky to have never taken the SATs myself (HS dropout, but I ended up doing the GED and then an Associate's and finally a BSCS.
"an attempt to shut down those structures that make educational inequality visible" - partly to avoid uncomfortable conversations as you say, but also middle-class self interest: "Someone Has To Fail" by David Labaree, HUP 2012.
As someone making hiring decisions, if I see someone who got perfect grades at a noncompetitive regional state school, I consider that person as impressive as someone who got good-but-not-perfect grades at a highly selective school. Grades are a way for those who went to North Dakota State to be able to compete with those who went to Duke. Ending assessment may flatten the range of outcomes within an institution (though I'm skeptical of that), but it will certainly exacerbate the range of outcomes between institutions.