Three Ways to Report an Election

In my AFR piece today, I make a new case for following betting markets instead of polls – they’re more boring, so they leave more room for talking about ideas.

“Forget the Polls, Ask the Hard Questions”, Australian Financial Review, 13 September 2007

There are three ways to report on an election: as a horserace, a football game, or a national conversation. Horserace reporting focuses on the leader – what are they doing, how much weight are they carrying, and who’s going to win? Football reporting analyses the team – who is likely to take the field, are they fit enough, and how well are they playing together?

Reporting on an election as a national conversation is rarer, and harder. It involves analysing the policies put forward by the parties, thinking about what’s being left off the agenda, and asking questions like ‘what’s your vision for Australia in 2025?’.

The trouble is, with a limited number of newspaper pages and political journalists, a growth in horserace and football-type reporting tends to crowd out deeper analysis. So it helps to ask: what simple rules of thumb can minimise the time spent analysing the races, and maximise the amount of airplay devoted to the big issues?

The first is to recognise how little opinion polls can tell us. Suppose a poll tells us that one party has 52 percent of the vote. In theory, with a sample of 2000 voters, the confidence interval is approximately ±2 percent. So 95 percent of the time, the true result should lie between 50 and 54 percent. The other 5 percent of the time, the true result will lie below 50 percent or above 54 percent.

Unfortunately, this assumes that the sample is random. Since pollsters undersample mobile phone users, and miss the many households who hang up on them, the assumption of random sampling is probably a stretch. As a result, polls are probably much less precise than the above results suggest. Based on the yo-yo volatility we observed in pre-election polls in 2004, Justin Wolfers and I estimated that the true confidence interval for Australian polls is probably closer to ±12 percent.

Given that polls are about as accurate as a drunken dart-thrower, it is surprising that any newspaper ever gives them front-page billing. Even more surprising is that the media gives attention to changes in opinion polls. If two polls are measured with error, then the difference between them will be even more imprecise. Any commentator who reports on small poll changes without mentioning error margins is at best wasting readers’ time, and at worst actively misleading them.

A simpler alternative to polls is to use betting markets. These have three chief advantages over polls. The first is that they are more accurate. Over the past decade, researchers in Australia, Germany, Sweden and the United States have compared the predictive power of polls and betting markets. Every time, betting markets have been found to perform at least as well, and usually better, than the polls.

While most academic research on election betting markets is new, the markets themselves are not. In the United States, thriving gambling markets existed on Presidential elections since the late-nineteenth century, and the odds from those markets have been shown to be an accurate predictor of the results in the 1884-1928 elections.

The second advantage of betting markets is that they are available on a wide range of outcomes. In addition to the half-dozen bookmakers who are currently offering odds on the headline race, Portlandbet has opened betting markets on every federal seat, while Sportingbet has a betting market on the most likely election date.

The third advantage of betting markets is that they are more boring than polls. At present the markets suggest that the Coalition is a 30 percent chance of winning, and that John Howard has a 52 percent chance of retaining his own seat. After Howard steps down as leader, the markets suggest that Peter Costello is a 53 percent chance of succeeding him and Malcolm Turnbull a 23 percent chance (the other contenders together have a 24 percent chance). Rather than naively hope for certainty, the efficient approach is to estimate probabilities as best we can, and move on to more interesting questions.

Which brings us back to elections as a national conversation. Elections are not just about choosing a party to lead the country. They also provide the chance to talk about the best and worst in the nation, what must be preserved, and what needs changing. This is too valuable an opportunity to be squandered with sportsplay reporting. Enough with the polls – let’s talk about the kind of Australia we want our kids to live in.

Dr Andrew Leigh is an economist in the Research School of Social Sciences at the Australian National University.

Postscript: There was a line I nearly included, but cut from the last draft. After raising the possibility that polls might be afflicted by more than sampling error, I was going to say “Professional pundits, if this sounds like statistical mumbo-jumbo to you, maybe it’s time to ask: Am I really political commentator material?”.

About these ads
This entry was posted in Uncategorized. Bookmark the permalink.

12 Responses to Three Ways to Report an Election

  1. Michael W says:

    I’m a little skeptical about applying US conclusions to the Australian context, especially when applied to the seat by seat results. This is based on not knowing how “thick” or “thin” these betting markets are. While in theory the “wisdom of crowds” might be giving better info than the polls, are we really sure there are more than a handfull of people betting on each of these Australian seats? We really have no measure of the level of “error” contained in an estimated probability based on the odds if we don’t know how many have laid a bet.
    (i think your general argument is probably right, but I just have that nagging doubt).

  2. Fred Argy says:

    Andrew, very professional but I have some nagging doubts too. I don’t think you have fully explained two anomalies,

    First, why are the national polls out of line with the seat by seat polls? The latter suggest a tight outcome and even a Howard victory.

    Secondly, don’t the national opinion polls have a significant (although perhaps not decisive) effect on the betting markets? Recent trends in Australia suggest the latter follows the former rahter than the other way round. There is some interaction between the two that needs examination.

  3. Fred Argy says:

    Sorry I meant to add a third question: why are the national betting markets out of line with the seat by seat betting markets? The former tell a very different stroy from the latter.

  4. Michael Potter says:

    Andrew – v good article. I also agree completely with the postscript, but can understand why you didn’t put that in the article.

  5. Sinclair Davidson says:

    Good piece – well done.

  6. Fred Argy says:

    Andrew, here is another teaser. Are there many betters like me who bet for the Party they hope will LOSE the election? It is perfectly rational to do this if you want a consolation prize to drown your pain. But it weakens the predictive powers of betting markets.

    I do not question your finding that betting markets are “more accurate” than opinion polls. But I do not trust either of them. I will start to trust them a week from the election date (which could be two months or more from now). There are too many uncertainties at this stage

  7. Andrew Leigh says:

    Fred, I had no idea you were so interested in the betting markets! Three quick answers.
    * Simon Jackman has a nice post just now on local vs national betting markets.
    * As to whether the markets simply aggregate the polls, the easy answer is “a bit” (see our 2005 paper). However, if they were merely an aggregation of the polls, then they wouldn’t have greater forecasting power (there was some debate on this point here a few months back).
    * Even if you’re betting for insurance purposes (or, conversely, to maximise marginal utility), you’ll still take account of prices. For example, suppose you’re deeply worried that a Rudd government will socialise the means of production; you’ll still be more likely to place an insurance bet on them at a time when the odds look attractive.

  8. Fred Argy says:

    Thanks Andrew. If I understand Simon Jackman, he is saying that both the national and seat by seat betting markets are pointing to a win by Labor but “there is still some distance between the national headline result and the seat by seat betting”.

  9. LuckyDave says:

    Hi Andrew,

    Agree that the National dialogue is crucial to get good plans and policy.

    Also agree that the polls are not perfectly accurate. However, I strongly suspect you are overplaying the telephone availability error factor substantially and thereby dramatically inflating the error margin for polls.

    A simple test of your hypothesis would be to determine the degree of variation that exists between the F2F polling (such as Morgan) Vs the telephone surveys (News, Galaxy and AC Nielsen). While there is some variation, where Morgan face-to-face sees to slightly increase Labor’s 2PP vote (corresponds intuitively to your implied arguement : youth, working, mobility etc), it ends up making a barely significant difference between the other major telephone derived polls nor the actual election outcome in most cases.

    Even moreso because it is also very possible that Gen Y and people who are higly mobile are less likely to be enrolled or actually vote; so the telephone induced minor error margin that you refer to may have a degree of self correction through the same voter’s non-participation behaviour.

    All the polls move in lock-step with the same underlying trends emerging over time, they are remarkably consistent. Not sure what definition of true confidence interval you have but if the polls were +/- 12 surely no-one would rely on them. Modern quantitive polling, close to the poll date is exceptionally reliable – so a of 12 number is too harsh; about 3-4 seems right, tightening to about 2 on election eve.

    I suspect that political betting markets are influenced more by ideology and personal passion than – if such a thing exists – a more rational betting market such as roulette.

    While watching the betting markets is informative and fun — it seems to lag polling not lead, but they may in-turn further influence each other and the national dialogue itself.

    Infact, the margin of error introduced by the bookies fee probably makes the betting markets inferior to most polling that directly asks a sample of voters. We should be demanding larger polling samples : the ultimate being the actual election itself.

  10. well, but... says:

    I’m afraid I don’t agree with the argument you’re making. You appear to be suggesting that if newspapers cited polls less and betting markets more this would lead to an increase in the number/quality of policy discussions that would take place in the national press, and perhaps amongst voters.

    In other words, your claim is that the difference in the degree of “boringness” in the reporting of betting to polling is sufficient to change behaviour, and that behaviour is likely tend towards greater policy discussion on national issues, as opposed to, say other frivolous reporting or gossiping unrelated to the “national conversation” you desire.

    It seems to me you are making a mighty leap of logic, which is fine in a frivolous article. But I second your decision to remove your sentence questioning the professional qualifications of pundits not fluent in statistical reasoning. It is neither worse (nor less professional) than using statistical reasoning to reach a wholly unsubstantiated conclusion, as you appear to have done here.

  11. Simon Jackman’s site the last time I looked predicted Labor for 76 seats on seat-by-seat betting.

    This is a darn site closer than the national betting would say, and this leads me to the conclusion that betting markets are just as useless as polls (but cheaper to get together than using pollsters).

    Surely the problem with using a betting market is that it is not efficient as a market? A properly fuctioning market relies on equal access to information, but this is not the case, clearly people placing bets rely on polls for their information, so betting odds are at the mercy of polls (just watch them change the day after every poll is released) and major political events as portrayed in the media.

    If the people engaging in the betting market are getting their information from the same sources as the people being polled by pollsters, surely this makes them just as inaccurate?

    Also isn’t a partisan person more likely to make a large bet than an undecided/swinging voter? Doesn’t political partisanship take the ‘rational investor’ part out of the equation?

    If that is the case, surely focus group polling of certified swinging voters is the most accurate measure (of random polling, betting markets and focus groups), given it is generally accepted swinging voters are what decides an election?

  12. Pingback: More About the Election Betting Market » Stephen Lloyd

Comments are closed.