158 Comments
Commenting has been turned off for this post
Sep 13, 2021Liked by Freddie deBoer

Not quite a paradox, but something everyone falls for:

A has a positive correlation with B.

B has a positive correlation with C.

"Statement about A being positively correlated with C."

But correlations are not transitive. It is possible for A and C to have no correlation, or even a negative correlation. The above claim can only be made with correlations very close to 1. Which pretty much never happens.

I'd like more citizen distrust, or at least hesitancy in this area. The default response to any random statistic should be "that's probably wrong". There are so many easy ways to make mistakes. We know that most Doctors don't understand sensitivity vs specificity. We are living through the replication crisis.

So yes, please more citizen thinking in this area. But it should mainly be critical thinking for the purpose of falsification.

Expand full comment

I have been ranting about this on the Slow Boring Substack for months now. As a PCP I think it's super important to be as objective and unemotional about Covid risks as possible. I can't tell you how frustrating it is to have a lack of clear, firm data and statistics. Numbers are freakin' all over the place depending on which study you read. We just don't have a firm handle on numbers vaccinated and numbers of breakthrough infections. The data on hospitalizations, ICU admissions and deaths are a little clearer. It makes it very difficult to know how to advise individual patients about their individual risks, especially for my vaccinated seniors. For them, there is a risk to both underestimating the risk, which could lead to a breakthrough infection (even w/o hospitalization, that could take quite a toll on a senior), and to overestimating risk, leading to repeat isolation and depression. How great is their risk if they are >6 months out from their second shot? How effective are boosters going to be? Don't know. Difficult to advise.

Expand full comment
founding

The same problem causes us to underestimate the strength of correlations, and I believe that is happening with data on vaccine effectiveness. Just yesterday, there was a big CDC press release that “unvaccinated people are 11 times more likely to die” from covid-19. But they’re using surveillance data.

Vaccinated people with asymptomatic infections won’t show up in surveillance data because (for the most part) those people won’t know to get tested. So we aren’t counting a bunch of cases where vaccinated people had no symptoms. If we had data on those infections, the correlation between vaccination status and death would be stronger.

Specific studies where everyone gets tested will pick up asymptomatic infections, but we can’t necessarily generalize because conditions vary across time and place (new variants emerge, etc).

Expand full comment

Berkson's Paradox sounds like a particular form of survivor bias, or in some cases sampling bias? Also compare with Simpson's Paradox, which isn't about missing data points, but rather how you divided them up into different bins before taking averages.

Expand full comment

"Every person who makes it to the league is an insanely skilled genetic marvel that’s been through a decades-long process of evaluation by coaches and scouts."

Apparently a significant proportion of 7ft plus people in the US are in the NBA. Some of them must be pretty mediocre at everything other than being tall - genetic marvel height wise, but that's it.

Also, like the phrase "Genetic marvel", so much nicer than genetic freak.

Expand full comment

I sucked at math too. But your basketball example made sense. Interestingly there's lots of videos about Simpson's paradox (which actually feels more complicated to me) than Berkson's.

I don't know how this effects Covid data specifically, (although like you I'd bet it does), but I'd also bet that our collective failure to grasp many of these large data math issues (paradox's, exponential growth, cost benefit analysis etc) contributes not only to misunderstandings about Covid and SAT scores but many other issues being argued by millions of pigeon chess champions on Twitter daily.

Sports, school, race bias, disease. "Do your own research!" No. Because I at least know that I'm not informed enough about how statistics work to do my own research. Most people don't.

(See how I centered myself at the end there like a good narcissist. Ha!)

Expand full comment

The obvious underlying variable is "general immune system effectiveness". Presumably an actual immunologist would be able to describe this better, but people vary in the strength of their immune response and the vaccine works by priming immune response. If your immune system is bad (immunocompromised) the vaccine does relatively little to help you and you're much more likely to get seriously sick from covid, with or without the vaccine. So the hospitalized, vaccinated patients probably have worse functioning immune systems than the hospitalized, non-vaccinated patients.

I expect the immunocompromised are still benefited by getting the vaccine. I wouldn't be surprised if this is another effective selection bias, where people with certain health conditions are much more likely to be vaccinated and so make it appear the non-vaccinated are relatively safer than they really are (because those pre-existing conditions are disproportionately in the vaccinated population).

Expand full comment

I suspect that the narrative on mask wearing is much more complex than the popular narrative.

Consider: in 2017 about 60,000-80,000 US'ians died from the flu. It was a particularly bad season. But even over "normal" years tens of thousands of people in this country can be expected to perish. Add that up over the course of decades and it's in the range of hundreds of thousands of fatalities.

Mask wearing has been common in places like Japan and S. Korea for decades. as a countermeasure not for Covid but for the flu. And yet US health authorities never seriously considered issuing guidance on mask wearing, even during the flu scare during Obama's administration. And global authorities, such as at the WHO, have been even more reluctant to endorse mask wearing, even in the face of Covid. Why?

My guess is that it's simply because all of the trials and observational studies that examined mask wearing in Asia versus the rest of the world was never able to produce a statistically significant effect. And, most critically, initial mask guidance on Covid was 100% consistent with mask guidance on the flu: namely, that they weren't needed.

Expand full comment

In general I suggest we all NOT think about basal rates, cutpoints, and excluded portions of sampled populations as we continue to stumble our way through this pandemic, because almost none of us are capable of thinking about such things usefully.

If you want "to cultivate a respect for the work we all need to do in order to better understand where we are with this disease", the key point is that this work is not possible for the vast majority of us.

As in almost all areas of life, the best strategy is to identify a set of trustworthy experts, and follow their advice. That trust is earned through previous accomplishments.

Expand full comment

The value of masks vs mask mandates? In the case of New York State the first mask mandate was April 15 at which point mask use was already very high. And obviously mandating something that people are already dry doing is going to have a minimal effect. Some folks latched onto this to say A. This proves masks don’t work. B. This proves mask mandates don’t work.

Expand full comment

Favorite line "(Distrust anyone who thinks less citizen thinking is better than more.)"

Hasn't that been the problem? We don't trust institutions that don't trust us to handle the transparent truth.

People are mad at Rogan for daring to say things that run against the mainstream because he has such a large platform and so undermining the push for universal vaccination. But he has such a large platform because he's willing to talk and think about issues giving air to various positions in an honest way. If our institutions were more honest they would retain greater influence and wouldn't need podcasters to keep in line. But they weren't...because they didn't trust their audience.

Fauci's fib about masks early on was ridiculous as it was something he was going to have to 180 on eventually. And it was pretty much right out of the gate to the most polarized audience of all time.

Expand full comment

Bravo! FdB goes out of his way not to call people dumb, just lacking information - which I do not believe for a minute. It doesn’t require complex statprob logic to understand how fear porn can be exploited on a gullible, receptive public. Per his example, Testing on the vaccine (technically a ‘shot’) excluded anyone who was likely to have complications. Including the aged, the overweight, diabetics, immune compromised , and pregnant women because, well, it would be unethical to test it on them, but were then immediately pushed to get vaccinated. Talk about subselecting!

The real question is, how are numbers being manipulated today?

The current diatribe: hospitalizations. Nevermind that hospitals aim for a standard 90% ICU occupancy rate anyway - just in time mode that doesn’t even leave room for a 10 car pileup. Now a Stanford study shows that half of all ‘COVID’ hospitalizations are not even due to COVID, but to other causes where patients tested positive but are asymptomatic.. MOOOOOO! MOOOOOO! Stampede!

Expand full comment