Recognizing Simpson's paradox

Even given that one understands Simpson's paradox, it would be nice to have some way to recognize that a new situation involves the paradox. In other words, given our hammer (i.e. understanding of Simpson's paradox) how can we go "hunting" for nails? (This is generally not a good way to solve problems, but it seems like a good way to learn.)

A preliminary list of steps:

1. Encounter some new situation.
2. Create a causal graph for the situation.
3. Build tables for the success rate of the intervention, both unadjusted and adjusted for confounders. It's actually not necessary to build all the tables because step (5) tells you which table to use (hence to build), but you need all the tables to see if Simpson's paradox occurs.
4. (You might observe Simpson's paradox now, looking at the tables.)
5. Use the causal graph to determine which of the tables you should use.

Michael Nielsen talks about the moment when one's intuitive reasoning goes astray. (Search for all occurrences of "moment" on that page.) If you can train to notice such moments beforehand, that seems like a way to discover potential instances of Simpson's paradox.

See also

• Reverse regression