Prediction markets lose to Nate Silver's FiveThirtyEight

By a hair:

Were prediction markets consistently better than a well-performing site like fivethirtyeight.com? No.

Were prediction markets consistently worse than a site like fivethirtyeight.com? No.

They perform largely the same, though the final accuracy of fivethirtyeight.com was a tad bit better than InTrade/IEM. But the purpose of the two types of sites are different.

Fivethirtyeight.com and similar sites take current data and process it to extrapolate trends. These sites lag real events; Nate Silver mentioned on a number of times that he expected his model’s forecast to move, but that it hadn’t because the relevant polls hadn’t hit the model yet. Between the time a poll closes, the result released and then incorporated into the model is anything from a day or two to several days. So while it looks quite accurate, a poll aggregator is a lagging indicator. (Another couple of election cycles will tell us if their accuracy continues or was just a fluke in 2008.)

But driving with just the rearview mirror is not the best we can do:

Prediction markets show the results of what a group of traders believe what will happen. This includes polling data, but also reacts to real-time information. A candidate makes a huge gaffe, and the market price will reflect it in minutes, where a poll aggregator could take days to see any effect.

This is the “social utility” of prediction markets, to answer a question that Chris Masse always poses. While they are on par with the accuracy of the best poll aggregators, their forecasts are real-time and reflect the state of the race right now. No other mechanism does this. While markets are certainly fed by polls, that isn’t the whole puzzle in and of itself.

Prediction markets worked quite well again this election cycle. Though their final forecasts were on par with the best poll aggregators, their real-time forecasts throughout the election season is the reason why they should be examined and discussed more broadly.