I’ve been banging on for a while about the fact that margins of error are particularly important when papers are reporting on changes in voteshare.Â Putting some empirical grunt behind the argument,Â Simon Jackman has a fascinating graph showing the poll sample size required to detect a change in support. He concludes:
Two polls with sample sizes of about 1,500 each have about a 50-50 shot at finding statistically significant changes when the real change is 4 percentage points, and it only gets worse for smaller changes in vote share. By the way, four percentage points is a lot of change, relative to the variation in actual vote shares in Australian electoral history.
The worst offenders on this score in the 2004 election were the SMH, who found a 1% increase in the Coalition’s 2pp vote from one poll to the next, and put it on the front page, under a headline “Surprise surge for PM“.