Saturday, January 05, 2008

Good discussion between Justin Wolfers and Bob Erikson

on the predictive quality of markets:
I [Justin] like Bob's paper a lot, and I'm glad you raised it, because I think it is a bit under-appreciated, and also a bit misunderstood. It turns out that Bob and I disagree a bit about the message from his paper (although after a few long chats, we don't disagree *that* much).

My [Justin's] thoughts:

1. Bob and Chris has four elections in their data, so it is hard to draw too much from it. That said, I draw two conclusions. First, markets beat an unconditional use of polls as forecasts. Second, correcting the polls, the two are pretty darn close. Based on a sample of four elections, I'm not sure I am willing to call one method or the other the winner.

2. My conclusion is that this actually tells us a lot of what prediction markets do: they digest and aggregate the polls, and create a pretty useful adjusted forecast. If I don't have the time to do the careful aggregation that Bob and Chris have, then I'm glad to have the market to do this for me. So I interpret their paper as telling me something about the mechanism by which prediction markets do well in forecasting elections. (This suggests an interesting puzzle: why did prediction markets do well in the pre-polling era?)

3. Your comment that polls are a snapshot, not a forecast, strikes me as a bit beside the point. Many people use polls as a forecast, and so it is reasonable to ask if they are a good forecast. (And pollsters sell them as forecasts, until you suggest they don't do well, and then they fall back on the "snapshot" argument.)

Read the whole thing.

No comments:

Post a Comment