Why Correlations Are Unreliable Risk Indicators

I just finished reading Maneet Ahuja’s new book, The Alpha Masters: Unlocking the Genius of the World’s Top Hedge Funds. It’s a great book, packed with investing insight. I have no idea how she got all these guys to talk on the record: Ray Dalio, Tim Wong, Pierre Lagrange, John Paulson, Mark Lasry, Sonia Gardner, David Tepper, Bill Ackman, Daniel Loeb, James Chanos and Boaz Weinstein, plus an introduction by Mohamed El-Erian and an afterword by Myron Scholes.

Unlike some profiles that focus on lifestyles or dramatic events, these concentrate on how the subject built a world-class hedge fund and how he or she invests. These topics are not as closely connected as you might think. You obviously need to be a great investor to found a hedge fund that succeeds in the long run, but it’s far from the only requirement. The book is fun to read and contains important advice on almost every page.

I’m not going to review the book here. Instead I want to expand on a great point made by Ray Dalio of Bridgewater Associates. “To create the proper balance and diversification is even more important than any particular bets, which is the opposite of how most investors operate.”

He gives a simple example of an investor with 15 different uncorrelated bets, each with an expected return of 3% and a standard deviation of return of 10%. If you combine two bets, your standard deviation falls 29% to 7.1%. If you take all 15 bets, your standard deviation falls to 2.6% (the book inexplicably claims the reduction from the second bet is 15% and that 15 bets reduces your risk 80%, in fact that takes 25 bets). This is basic statistics, with equal volatility uncorrelated bets the standard deviation declines with the square root of the number of bets.

Then Dalio makes a less familiar point. “The most important rule is not to compare the correlations against each other in a quantitative sense, but according to their drivers.”

One problem with using correlations in risk analysis is you may misestimate them; things you think are uncorrelated may not be. A common way to misestimate is to assume correlations from the recent past will continue into the future. But it is possible to form moderately reliable correlation estimates by combining careful study of history with fundamental analysis. I believe portfolios constructed from these estimates can deliver long-term superior performance.

There is a more subtle problem, one that can hurt you even if your estimates are perfect. Suppose Dalio’s bets above are coin flips, heads you win 13%, tails you lose 7%. These bets have the required 3% expected return and 10% standard deviation of return. If you make one bet, there is a 50% chance you will lose money. If you make 15 bets, you might think you reduce your probability of losing money by the square root of 15, to 13%. If you think that, you’re wrong. Correlation tells you what happens to your standard deviation, which is not necessarily what happens to your risk.

The key problem is correlation is only a pairwise concept. Since the bets are uncorrelated, we know that if we pick any two, the chance is 1 in 4 that both will lose 7%, 1 in 4 that both will make 13%, and 1 in 2 that one will lose and one will win. Knowing that all the pairwise bets are uncorrelated tells us very little about how 15 bets will turn out. That is what Dalio means by understanding the drivers.

This next part gets a bit mathematical, but it’s worth following because it’s so important. Suppose there are two possibilities: the economy will do well and nine of the 15 bets will pay off, or the economy will do badly and only five of the 15 bets will pay off. The probability of a good economy is 5/8, and the probability of a bad economy is 3/8.

The expected number of bets that win is (5/8)*9 + (3/8)*5 = 60/8 = 7.5, so there is the required 50% chance for each bet to win. Suppose we are told that one bet paid off. This increases the probability that we are in the good state from 5/8 to 3/4, by Bayes Rule. If we are in the good state, the probability is 8/14 that any given other bet will win. If we are in the bad state, the probability is 4/14. The unconditional chance that any given other bet will win is (3/4)*8/14 + (1/4)*4/14 = 28/56 = 1/2. This is all “uncorrelated” means: that the chance of any bet winning is the same whatever I learn about the outcome of any other bet.

If this is the situation, the chance of losing money making all 15 bets is 37.5%, almost three times the 13% naïve calculation that was done above. The problem is that although there were 15 uncorrelated bets, there was a single driver (whether the economy was good or bad).

This may seem like a technicality that is not important for real investing. Nothing could be farther from the truth. It is easy to find uncorrelated bets, thousands of them. Take any stock, for example, and hedge it with its industry index. The result will be close to uncorrelated with some other stock hedged with its industry index. But a diversified portfolio of 100 stocks hedged with their respective industry indices will not have 10% of the risk of any single position. It may have 10% of the standard deviation, but the probability of a tail event may be almost as great as holding a single position.

If there are too many numbers in that explanation, here is a less quantitative example. Suppose you wanted to bet on the percent of the vote the winning candidate got in the 2010 US congressional election. The median was 63%. If I told you the incumbent was a Republican, the median is still about 63%, the incumbent’s party is uncorrelated with your bet. If I told you the winning candidate was a Democrat, the median is still about 63%. The winning candidate’s party is also uncorrelated. But if I told you the incumbent was a Republican and the winning candidate was a Democrat, the median drops to 53% (there were only three such races), and you will lose all the time if you bet the winning candidate will get over 63% of the vote. Two pieces of information are individually uncorrelated with the bet, but when combined they have a high correlation with the bet.

Here is one final example that illustrates another good way to think about correlation. Suppose Dalio’s bets are all bonds that cost $100 and are supposed to pay $1 per quarter interest and $101 at maturity in one year. The bonds will all make their three quarterly interest payments but may default if the issuer cannot refinance at maturity. The probability of that is 1/101 for each bond, and the defaults are uncorrelated.

The expected return from each bond is (100/101)*$4 - (1/101)*$97 = $303/101 = $3, as required. The standard deviation is $10. Because the bonds are uncorrelated, the probability of any two defaulting is one in 101 squared, or 1/10,201. This leads some people to assume the probability of more than two bonds defaulting must be negligible.

Imagine I have a hat with 10,201 slips of paper in it. Some of the papers have the names of bonds written on them. I’m going to draw one slip to see which bonds default. Since I know each bond has 1/101 chance of default, each bond must be on 101 different slips of paper. Since I know the defaults are uncorrelated, each pair of bonds must be on exactly one slip of paper. But that’s all I know. There are many ways to write the bond’s names on papers to satisfy these conditions.

For example, I could write all 15 bonds on one piece of paper, then each bond alone on 100 other pieces of paper each, leaving 8,700 blank slips. That makes the chance that all 15 bonds default 1 in 10,201. That’s low, but if you take risks like this every day, sooner or later one will catch up with you. When it does, someone half-trained in mathematics will say the probability of the event was 1 in 101 to the 15th power, something that should never have happened in the history of the universe. They will be wrong because they do not understand correlation. Uncorrelated bets are no guarantee that you will not have extreme tail events. As in the first example, you have 15 uncorrelated events, but only one driver.

Another thing I can do is write three bonds on 35 pieces of paper. Each individual bond is on seven of these slips, each time with a different pair of other bonds. Then I write each bond by itself on 94 slips of paper and leave 8,756 blank slips. Now the probability of more than two defaults is 35/10,201, or 0.3%. It’s low, but it’s not negligible.

This raises the question of how many drivers there are in the world, on which you can bet enough money to be meaningful to a portfolio. I don’t know the answer, but I’d guess it’s something like Dalio’s 15. Call it five that are available to any investor in low-cost, simple vehicles, five that are for professionals, and five that are cutting-edge hedge fund strategies. In fact, one important economic purpose of hedge funds is to seek out new drivers, which eventually become well-understood and liquid enough to be offered in cheaper form to less sophisticated investors, and eventually to be incorporated in index funds for everyone at minimal cost.

Fifteen drivers doesn’t mean you only make 15 investments. Finding uncorrelated bets that depend on the same driver still reduces your risk, just not all the way to zero, and perhaps not your extreme tail risk at all. Correlation is a powerful tool for building portfolios, but never confuse a tool with a driver.