Correlation
Correlation measures the degree of association between two variables, but it does not imply causation. It can be understood geometrically as the cosine of the angle between vectors in high-dimensional space, and it is not a transitive relationship.
THE IDEA
Background
It is difficult to overstate the impact of Galton’s creation of correlation on the conceptual world we now inhabit—not only in statistics, but in every precinct of the scientific enterprise. If you know one thing about the word “correlation” it’s that “correlation does not imply causation”—two phenomena can be correlated, in Galton’s sense, even if one doesn’t cause the other. This, by itself, was not news. People certainly understood that siblings are more likely than other pairs of people to share physical characteristics, and that this isn’t because tall brothers cause their younger sisters to be tall. But there’s still a causal relationship lurking in the background: the tall parents whose genetic contribution aids in causing both children to be tall. In the post-Galton world, you could talk about an association between two variables while remaining completely agnostic about the existence of any particular causal relationship, direct or indirect.
Geometric Interpretation
Were I to write down Pearson’s formula right now, or were you to go look it up, you would see a mess of square roots and ratios. But in fact, Pearson’s formula has a very simple geometric description. The correlation between two variables is determined by the angle between the two vectors representing the data. The correlation is the cosine of the angle. When the angle is 0°, correlation is 1; when 180°, correlation is -1; when 90° (right angle), correlation is 0. Variables at an acute angle are positively correlated, at an obtuse angle negatively correlated, and orthogonal (perpendicular) variables are uncorrelated.
Non-Transitivity of Correlation
Correlation is not transitive. If variable A is correlated with B, and B with C, it does not necessarily follow that A is correlated with C. This is illustrated with examples:
- Political Example: Being from a rich state is correlated with voting Democratic, but rich individuals overall are still more likely to vote Republican. The correlation is not transitive across levels.
- Mutual Fund Example: Laura’s fund (half Facebook, half Google) is correlated with Sara’s (half Honda, half Facebook), and Sara’s with Tim’s (half General Motors, half Honda), but Laura’s and Tim’s returns are not correlated.
- Medical Example: High HDL cholesterol is correlated with lower heart disease risk, and niacin increases HDL, but niacin does not necessarily reduce heart disease risk—correlation is not transitive.
Additional Insights
The geometric view clarifies that orthogonality means no linear relationship. The term "orthogonal" is increasingly used in everyday language to mean unrelated. The non-transitivity of correlation explains why many promising medical interventions fail despite affecting known biomarkers.
When to use it
Evaluating medical treatments based on biomarker correlations without assuming chain effects.
Interpreting political polling data with caution about aggregation levels.
Analyzing financial portfolios where stock returns are correlated; understanding that diversification requires more than just combining correlated assets.
Designing experiments to test causal hypotheses rather than relying on observed correlations.
FROM THEORY TO EVERYDAY LIFE
In practice
California Temperature Correlation
Average January temperatures in ten California cities for 2011 and 2012 are strongly positively correlated (0.989). When the data are centered (each value minus the mean), they become ten-dimensional vectors. The small angle between these vectors indicates high positive correlation.
Wealthy Liberal Elitist Paradox
Although rich states tend to vote Democratic and being rich correlates with living in a rich state, rich individuals overall still lean Republican. Correlation is not transitive: county-level wealth and state-level wealth correlate with voting differently.
HDL, Niacin, and Heart Disease
High HDL cholesterol is correlated with lower heart disease risk. Niacin increases HDL, but large trials found it does not reduce heart attacks. This shows that correlation does not chain: niacin → HDL, HDL → lower risk, but niacin → lower risk does not follow.
A MOMENT TO REFLECT
Put it to work.
Can you think of a situation where you assumed a drug would work because it affected a biomarker that was correlated with a health outcome? How does this example relate to non-transitivity?
Why is it dangerous to assume that a correlation observed at one level (e.g., state) applies at another level (e.g., individual)?
How can the geometric interpretation of correlation (cosine of angle between vectors) help you understand when two variables are unrelated?
Often confused with
References
- Bobos in Paradise — David Brooks (2000)
- Red State, Blue State, Rich State, Poor State — Andrew Gelman (2008)
- Galton's correlation — Francis Galton
- Pearson's correlation coefficient — Karl Pearson
CONTINUE THE CONVERSATION
Source texts
- How Not to Be Wrong The Power of Mathematical Thinking (Ellenberg, Jordan)