What the pot and pain pill overdose study teaches us about ecological fallacies

I am delighted to offer Mind the Brain readers a guest blog written by Keith Humphreys, Ph.D., John Finney, Ph.D., Alex Sox-Harris, Ph.D., and Daniel Kivlahan, Ph.D. Drs. Humphreys, Sox-Harris, and Finney are at the Palo Alto VA and Stanford University. Dr. Kivlahan is at the Seattle VA and the University of Washington.

Follow Professor Humphreys on Twitter @KeithNHumphreys.

 

Image Credit: Bogdan, Wikimedia Commons
Image Credit: Bogdan, Wikimedia Commons

A team of scientists recently reported that states with laws permitting medical marijuana had lower rates of opioid overdose than states without such laws. In a New York Times essay, two members of the team suggested this state-level association between medical marijuana access and deaths reflects the behavior of individuals in pain:

 

If enough people opt to treat pain with medical marijuana instead of prescription painkillers in states where this is legal, it stands to reason that states with medical marijuana laws might experience an overall decrease in opioid painkiller overdoses and deaths.

 

At first blush, saying it “stands to reason” seems, well, reasonable. But in the current issue of the journal in which the study appeared, we point out that the assumption that associations based on aggregations of people (e.g., counties, cities and states) must reflect parallel relationships for individuals is a seductive logical error known as the “ecological fallacy.”

 

Once you understand the ecological fallacy, you will recognize it in many interpretations of and media reports about science.   Here are some examples that have been reported over the years:

 

 

Such differences are counter-intuitive and therefore a bit baffling. If individuals having heart attacks who receive high quality care are far more likely to survive, doesn’t it follow that hospitals that provide higher quality care to larger percentages of their heart attack patients would have substantially lower mortality rates? (Answer: No, their results are barely better). Why don’t patterns we see in the aggregate always replicate themselves with individuals, and vice versa?

 

The mathematical basis for the ecological fallacy has multiple and complex aspects (our detailed explanation here), but most people find it easiest to understand when presented with a simple example. Imagine two states with 100 people each residing in them, with each state population including a comparable proportion of people in pain. Potsylvania has a loosely regulated medical marijuana system that 25% of residents access. Alabstentia, in contrast, limits access to medical marijuana so only 15% of residents can obtain it.

 

Potsylvania

Medical Marijuana User Medical Marijuana Non-User Totals
Died of Opioid Overdose 2 3 5
Did Not Die of Overdose 23 72 95
Totals 25 75 100

 

Alabstentia

 

Medical Marijuana User Medical Marijuana Non-User Totals
Died of Opioid Overdose 4 6 10
Did NotDie of Overdose 11 79 90
Totals 15 85 100

 

Ganja-loving Potsylvania has a lower opioid overdose death rate (5%) than more temperate Alabstentia (10%).   Does this prove that individuals in those states who use medical marijuana lower their risk of opioid overdose death? Nope. In both states, medical marijuana-users are more likely to die of a pain medication overdose than are non-users: 2 of 25 (8%) of marijuana users dying versus 3 of 75 (4%) marijuana non-users dying in Potsylvania; 4 of 15 (26.6%) of marijuana users dying versus 6 of 85 (7.1%) of non-users dying in Alabstentia!

 

Embracing the ecological fallacy is tempting, even to very bright people, but it must be resisted if we want to better understand the world around us. So, the next time you see a study saying, for example, that politically conservative states have higher rates of searching for sex and pornography on line and want to immediately speculate about why conservative individuals are so hypocritical, pause and remember that what applies at the aggregate level does not necessarily apply to individuals. For all we know, alienated liberals in red states may just be feeling lonely and frustrated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s