It’s All About the Effect Size

I’m not an expert in statistics, but I have completed a bachelors and masters in psychology with a focus on research methods and statistical analysis, so I do know a thing or two. Familiarity with statistics provides many benefits, including an improved ability to think critically about numbers. This actually does come in handy when it comes to issues surrounding veganism.

I’ve noticed that a familiarity with effect sizes could do a lot for people arguing in favor of veganism. An effect size can be summarized with just two words: “How much?”

Most people are familiar with statistical significance, or at least think they are. The measure of statistical significance, the p-value, refers to the probability that the result happened by chance. So a p-value of .05 (typically the cutoff for determining significance) indicates a 95% likelihood that the result did not happen by chance.

But a statistically significant result isn’t necessarily a meaningful one. Here’s an example that will explain why.

Let’s say we gather together 1000 men and 1000 women. We want to know which group reads more pages of books in a year. Let’s pretend that the results of this study can tell us something — maybe we think that the persisting gender wage gap is in part due to a habit among men to read more books than women (I’m pretty sure women read more than men — just bear with me here). And let’s pretend that the results show that there was, in this sample, a statistically significant difference between men and women in pages read. Can we say that the gender wage gap is due to men reading more than women?

You know the answer is no, because correlation does not mean causation. And you are right — but I want you to go further. Let’s ask the question of effect size: how much? Note that I didn’t indicate how much more, on average, men read compared to women — I just said that there was a statistically significant difference. That difference might be as small as a single page. It would be ludicrous to conclude that just a few hundred words explains any meaningful difference in income by gender. And yet, this sort of mistake is extremely common — even in the realm of veganism.

This is particularly prevalent when it comes to diet. People often say that a diet that includes meat is superior to one that does not. In what way do they mean? Typically, they’re referring to animal-based proteins being absorbed more readily, or being “more complete,” that sort of thing. Let’s say this is true — that there is a difference between animal-based foods and plants when it comes to how the body handles them. How much different they are is what we should be concerned with. What if plants are absorbed 1% “worse” relative to animal-based foods? I suspect that even meat eaters would recognize this small effect size as not worthy of concern.

It’s also relevant to environmentalism. I am not going to pretend to be privy to the numbers, but veganism is undoubtedly far more efficient than omnivorism — livestock animals need food to eat, land to live on, and water to drink. Plants need some of those things, but far less. The effect size here is huge. But non-vegans like to bring up the fact that livestock can convert non-arable land into land where crops that humans can eat may be grown. Does this diminish the effect size that marks the difference in environmental impact between veganism and non-veganism? Probably, but also probably not by very much.

There are numerous other examples, but you get the idea. Non-vegans like to argue that vegan food is inferior, that vegans are less healthy, that omnivores are stronger, and so on. Some of these things may be true, but the differences between vegans and non-vegans, and between veganism and non-veganism, may be so negligible as to be unworthy of a moment of concern. In fact, we might have good reason to assert that overcoming the ethical imperative of veganism would require numerous huge effect sizes in various realms, all favoring a non-vegan diet. That’s unlikely to be the case.

If you enjoy our work, please consider supporting us on Patreon

Advertisements

Leave a Reply