Selling Palin Short

"I don't worry about the polls. Polls are just a fancy way of systematically predicting what's going to happen."

Thus spoke Tina Fey-as-Sarah Palin on Saturday Night Live. It made for a good line—but are polls really that systematic? Just how well do polls do at predicting the outcome of an election? How can you take the results of multiple polls and paint a convincing picture out of them? Finally, what other mechanisms are there for predicting the outcomes of elections?



There are a number of sites devoted to analyzing the myriad of polls that come out every day as the election nears. Two good sites that take very different approaches are RealClearPolitics.com and FiveThirtyEight.com. To compare and contrast their analyses, let's take a look at Pennsylvania, which many analysts deem a pivotal state in this election. Here's a list of Pennsylvania polls taken during October:

Poll	Date	Size	Obama	McCain
West Chester U.	10/5	504	52.3	42
Strategic Vision	10/6	1,200	54	40
Rasmussen	10/6	700	54	41
SurveyUSA	10/6	653	55	40
Marist	10/7	757	53	41
Zogby Interactive	10/11	737	51.6	40.2
SurveyUSA	10/12	516	55	40
Susquehanna	10/18	700	48	40
Morning Call	10/19	594	52	41

These polls all varied in their methodology—whom they polled, how they did it (robocall or a human being making a phone call), and how they weighted the results.

RealClearPolitics.com chooses to average the polls it judges are the best reflection of public opinion. Recently, it listed only the last three polls and averaged them together to predict Obama with 51.7% and McCain with 40.3%. (Note that these don't add up to 100% because of undecided voters.)

FiveThirtyEight.com takes a much more sophisticated approach. It adjusts for trends within Pennsylvania, across the nation, and among demographic groups. Instead of just choosing a few polls, it weights them according to methodology and how out-of-date they are. Then it reports three numbers: the raw weighted average of polls, the trend-adjusted average, and the results of a regression analysis that takes into account demographic and other factors. It also makes a prediction of Election Day based on the likely last-minute decisions of undecided voters. Finally, with all of that analysis in mind, the site makes a prediction as to the likelihood that each candidate will win Pennsylvania on Election Day:

	Obama	McCain
Polling Average	50.8	41.8
Trend-Adjusted	52.3	41.0
538 Regression	52.1	40.7
Projection	54.1	44.4
Win %	98%	2%

So far we've looked only at sites which analyze polls. But there's another kind of site devoted to predicting the elections: prediction markets. The thinking behind prediction markets is that, just as prices aggregate private information about buyers' values and sellers' costs in a market for goods and services, a market for future events can aggregate private information about the likelihood of those events happening. Two well-known political futures markets are intrade.com and the Iowa Electronic Markets.

Take a look at the intrade.com page on Pennsylvania. There you can see the probability the market places on Obama winning the state—as I write this, it's about 85%. It's not quite a bet, it's more like a stock price. This is how it works: there's an option that pays $10 if Obama wins Pennsylvania, and $0 if he doesn't. The price of that option is currently about $8.50. In other words, people are willing to pay $8.50 for a piece of paper that pays $10 if Obama wins the state; therefore, we can interpret this to mean that the "market thinks" there's an 85% chance Obama will win Pennsylvania's 21 electoral votes.

Discussion Questions

1. Suppose you wanted to predict the probability that Obama will win Pennsylvania. How would you go about doing it?
2. We saw above that FiveThirtyEight.com assigns a 98% probability to Obama winning Pennsylvania, while the Intrade price assigns an 85% probability. Why the difference? Under what conditions would a prediction market be a better predictor of an outcome than a poll? Under what conditions would the reverse be true?
3. Not all prediction markets have the same price for each contract, despite the fact that arbitrage opportunities abound. Why might this be the case? (For those interested, Nate Silver at FiveThirtyEight.com has an interesting analysis of this phenomenon here.)
4. There are some nonzero prices on intrade for very unlikely events. For example, you might notice that there's an intrade bet on whether Sarah Palin would drop out as the Republican nominee for Vice President, and another market on whether Al Gore will be on the ticket on November 4. Why do these contracts exist? Why do you think they have positive prices?