In my AFR piece today, I make a new case for following betting markets instead of polls – they’re more boring, so they leave more room for talking about ideas.
â€œForget the Polls, Ask the Hard Questionsâ€, Australian Financial Review, 13 September 2007
There are three ways to report on an election: as a horserace, a football game, or a national conversation. Horserace reporting focuses on the leader â€“ what are they doing, how much weight are they carrying, and whoâ€™s going to win? Football reporting analyses the team â€“ who is likely to take the field, are they fit enough, and how well are they playing together?
Reporting on an election as a national conversation is rarer, and harder. It involves analysing the policies put forward by the parties, thinking about whatâ€™s being left off the agenda, and asking questions like â€˜whatâ€™s your vision for Australia in 2025?â€™.
The trouble is, with a limited number of newspaper pages and political journalists, a growth in horserace and football-type reporting tends to crowd out deeper analysis. So it helps to ask: what simple rules of thumb can minimise the time spent analysing the races, and maximise the amount of airplay devoted to the big issues?
The first is to recognise how little opinion polls can tell us. Suppose a poll tells us that one party has 52 percent of the vote. In theory, with a sample of 2000 voters, the confidence interval is approximately Â±2 percent. So 95 percent of the time, the true result should lie between 50 and 54 percent. The other 5 percent of the time, the true result will lie below 50 percent or above 54 percent.
Unfortunately, this assumes that the sample is random. Since pollsters undersample mobile phone users, and miss the many households who hang up on them, the assumption of random sampling is probably a stretch. As a result, polls are probably much less precise than the above results suggest. Based on the yo-yo volatility we observed in pre-election polls in 2004, Justin Wolfers and I estimated that the true confidence interval for Australian polls is probably closer to Â±12 percent.
Given that polls are about as accurate as a drunken dart-thrower, it is surprising that any newspaper ever gives them front-page billing. Even more surprising is that the media gives attention to changes in opinion polls. If two polls are measured with error, then the difference between them will be even more imprecise. Any commentator who reports on small poll changes without mentioning error margins is at best wasting readersâ€™ time, and at worst actively misleading them.
A simpler alternative to polls is to use betting markets. These have three chief advantages over polls. The first is that they are more accurate. Over the past decade, researchers in Australia, Germany, Sweden and the United States have compared the predictive power of polls and betting markets. Every time, betting markets have been found to perform at least as well, and usually better, than the polls.
While most academic research on election betting markets is new, the markets themselves are not. In the United States, thriving gambling markets existed on Presidential elections since the late-nineteenth century, and the odds from those markets have been shown to be an accurate predictor of the results in the 1884-1928 elections.
The second advantage of betting markets is that they are available on a wide range of outcomes. In addition to the half-dozen bookmakers who are currently offering odds on the headline race, Portlandbet has opened betting markets on every federal seat, while Sportingbet has a betting market on the most likely election date.
The third advantage of betting markets is that they are more boring than polls. At present the markets suggest that the Coalition is a 30 percent chance of winning, and that John Howard has a 52 percent chance of retaining his own seat. After Howard steps down as leader, the markets suggest that Peter Costello is a 53 percent chance of succeeding him and Malcolm Turnbull a 23 percent chance (the other contenders together have a 24 percent chance). Rather than naively hope for certainty, the efficient approach is to estimate probabilities as best we can, and move on to more interesting questions.
Which brings us back to elections as a national conversation. Elections are not just about choosing a party to lead the country. They also provide the chance to talk about the best and worst in the nation, what must be preserved, and what needs changing. This is too valuable an opportunity to be squandered with sportsplay reporting. Enough with the polls â€“ letâ€™s talk about the kind of Australia we want our kids to live in.
Dr Andrew Leigh is an economist in the Research School of Social Sciences at the Australian National University.
Postscript: There was a line I nearly included, but cut from the last draft. After raising the possibility that polls might be afflicted by more than sampling error, I was going to say “Professional pundits, if this sounds like statistical mumbo-jumbo to you, maybe it’s time to ask: Am I really political commentator material?”.