A CAPS Hypothesis - Mathematical Outperformance Using Collective Intelligence
I’ve been playing with The Motley Fool’s CAPS website for awhile (and doing quite well). It attempts to combine the knowledge of tens of thousands of investors with a fancy algorithm to find the best stocks. After running its calculations, it gives each stock a rating from 1 to 5 stars proportionate to how well it is expected to do in the future. I read an article (I couldn’t find the link) by a Fool writer who had run some statistics on the CAPS database to evaluate how effective it has been in the past.
They showed that 5-star stocks had averaged a 28% return, nearly 3x the market average for the same time period. That sounds impressive, but it is a bit misleading. You couldn’t have bought those stocks then since you didn’t know what their rating was going to be a year later. Since their algorithm weighs the opinions of better performing players higher than lower performing players, there is a natural bias for stocks that have performed well in the past to be rated higher than stocks that haven’t since players that rate it highly will have performed better and their rating will be worth more. What would have been interesting is the average return of a portfolio that bought stocks when they became 5-star stocks and sold them when they lost the rating, unfortunately they keep their database private so we can’t find out.
However, they showed something even more interesting: A portfolio that always bought 4-star stocks and always sold stocks that were no longer 4-star would have returned 42% average annual returns since they started, and you could have actually bought those since you only needed to act on information available at the time! You still can! A ‘4-star system’, like any well defined system however, will only produce enormous market beating returns while it remains unknown to the amount of capital required to improve the market efficiency to the point that the price reflects the knowledge.
I think I have an even better way though. I am hypothesizing that the 40k members of CAPS provides a representative sample of all investors and thus a representation of the market mind-set that determines real stock prices. Therefor, members in the 40-60th percentiles represent the mind-set that determines the stock price since they are in the average. Well actually, the market has its own weighting process; investors with more money have a much larger effect on the stock price and likely have that larger amount of money due to (in general) a higher skill in finding under priced stocks. So let’s instead say members in the top 50-70 percent represent the mind-set that determines the stock price. Further, we can extend that to suggest that members in the 90-100th percentile have the mind-set that more accurately represents the actual value.
A 5-star stock gets its rating when the weighted whole of the member base is very positive. However, if the top 10% can be said to represent the value of the stock, and the member base as a whole represents the actual price, then we should be able to find stocks that are most undervalued by looking for a large differential between the smart money and the average money. If everyone likes a stock, great, but if the smart money likes a stock and the average money doesn’t, it could be a very strong indication that the market is undervaluing the stock.
There is a crude way to use the CAPS website to identify this differential. On each page there are three bars that indicate the amount of members in each of three categories that are for and against that particular stock. I’m most interested in the first two categories: all members, and members in the top 20 percent. This isn’t quite the top 50-70% vs the top 10%, but it does provide an indication of the differential we’re after and provides the base of the information I’m going to use in my calculations.
First though, we must recognize that the CAPS service has many more times positive ratings than negative ratings. This is because people look for companies that are going to outperform more often than they look for companies that are going to under perform. This results in every stock having a hopelessly unrealistic amount of optimistic positive ratings that is not an accurate representation of the opinions whole member base. The star ratings take this into account because stocks are only rated relative to themselves and not absolutely against their ratings, but if we are using the data before these calculations occur, we have to take it into account ourselves.
Unfortunately the data is not readily available (without scraping the contents of their whole database from all the pages on the site, which may be worthwhile, despite likely breaking their rules) so I make the crude estimation that each stock has an equal number of people that think it will out perform as those that think it will under perform. I will use this estimation to normalize the ratings of both groups by increasing the weight of each negative rating so that the group that represents the total has an equal amount of negative and positive ratings, then adjust the top 20% group accordingly.
For example, consider company XYZ with ratings as follows:
Whole - Positive: 80, Negative: 20
Top 20% - Positive: 9, Negative: 1
Using the ‘whole’ group, we can multiply the negative ratings by 4 to get an equal amount, which changes both groups to the following:
Whole - Positive: 80, Negative: 80
Top 20% - Positive: 9, Negative: 4
The ‘whole’ will always be 50%, our baseline. The top 20% in this case now has 9/13 ratings positive. If we transform that about the 50% baseline we can get a decimal between -1 and 1. 1 would represent the maximum differential between the ’smart money’ and ‘average money’ (at least up to the accuracy of our crude estimations) and -1 would be the opposite. Following the assumptions about smart money/the whole above, a 1 indicates the most under priced (and best buys) stocks, a 0 indicates a fairly priced stock, and a -1 the most over priced (and worst buys).
My example has a rating of .38, a moderate indication that it is under priced (excluding the obviously large margin of error due to the low number of top 20% ratings). I predict that keeping a portfolio of only the 20 stocks with the highest ‘under priced’ rating would significantly outperform the market over time. I plan on setting up a test portfolio on this site to track that prediction, but in the mean time here are some interesting stocks to note using this method:
AAPL: .19, GOOG: .09, ONT: -.70
ATVI: .05, EXPO: 1.00*, TASR: -.2
*Expo has a very large margin of error due to a very small number of negative ratings
There are a number of errors in this method due to the assumptions we had to make and the varying biases in the source of the data both mentioned and still undiscovered. The ratings produced also do not represent a linear comparison between two stocks (AAPL is not necessarily twice as under priced as GOOG).
However, these inaccuracies only dampen the overall accuracy, and I am fairly confident that when taking a large enough sample, stocks with a positive rating will perform far better than those with negative ratings; I’m very curious to see just how much.
[...] the highest ratio of All-Star recommendations to total recommendations as defined in my first post here. It takes periodic samples of the ~3000 stocks with more than 50 ratings from the CAPS website and [...]