Monday, April 4, 2011

Life Isn’t Always Linear

Being an Investment Analyst often involves looking at a vast array of indicators that are supposed to be correlated to different degrees with some relevant financial variable. When a given indicator is said to be either positively or negatively correlated to a variable, in most instances this refers to linear correlation. However, two variables may very easily be non-linearly related in which case testing for linear correlation alone will produce misleading results. Responding to this issue, we developed a tool that applies a number of different non-linear transformations to the data in order to test for non-linear relationships between an indicator and a financial variable of interest. In addition, these transformations are applied on different moving averages and percent changes of the raw data, and the correlation is measured not only concurrently but also using different lags from 1 month to 24 months. In the chart below, the black dashed arrow represents a simple linear transformation while the other data points represent the different non-linear transformation that we also consider.

Each time this tool is applied to a single indicator, it produces over 1,500 data series derived from the original one and estimates the correlation of each one of them with the relevant variable. When performing such a massive data mining exercise, we are bound to find some significant correlations simply by chance. If statistical inference doesn’t betray us, for each one hundred correlations that we estimate we would expect, on average, to find at least one of them to be significance at a 99% confidence level. In these instances, it is important to remember that correlation does not necessarily imply causation, and the latter is what we are looking for. A good example is the famous Super Bowl Stock Market Indicator, based on the observation that in 29 out of the 36 years from 1967 to 2003 an NFC victory preceded positive stock market returns while an AFC victory preceded negative stock market returns. Even though it is undisputed that such a correlation existed historically, it would be foolish to expect this phenomenon to repeat itself in the future, unless there is a valid reason to believe that an NFC victory caused the stock market to rise and an AFC victory caused the stock market to fall.

This is where the big brains of Pinnacle’s Investment Team come into play: each time we run into a variable that seems to have a strong predictive power based on empirical tests, it is a team effort to determine whether we believe an underlying causation effect based on sound economic theories actually exists. If the conclusion is negative, then the indicator is discarded as there is no reason to expect the historical correlation to repeat itself in the future. In conclusion, let’s hope that the current NFL lockout gets resolved, otherwise the stock market may not move at all next year.