In a discussion about whether journalism students should know more about economics, Mark Bahnisch said this morning: “I think one thing aspiring journos need is a grasp of social and behavioural statistics and how to interpret them.” And as if proving his point comes the front page from today’sÂ Sydney Morning Herald and Age.
Both papers give plenty of column space toÂ a “lift” in Labor’s poll performance. Their evidence is two polls of about 1400 voters. On 20-23 April, AC Nielsen found Labor had 51% of the two-party preferred vote. On 18-20 May, they found that Labor had 54% of the two-party preferred vote.
The kicker is in the fine print. If you believe that the polls were afflicted only with sampling error*, then both polls had an 95% error margin of 2.6%. So the right way to interpret the two most recent AC Nielsen polls is:
- 20-23 April poll: There is a 95% chance that Labor’s two party-preferred vote was between 48.4% andÂ 53.6%
- 18-20 May poll:Â There is a 95% chance that Labor’s two party-preferred vote was between 51.4% and 56.6%
Notice the overlap? Alternatively, we can do a formal test for equality of means, which shows that with a sample of 1400 voters, the difference is significant only at the 11% level, meaning that there’s an 89% chance the movement is real, and an 11% chance that it’s mere statistical noise. Most social scientists regard movements that are not significant at the 5% level with a healthy dose of scepticism.
Do journos apply a similar approach? I would hope so. For example, I’d hope that Australian political editors would not put a tip on their front page if their source told them that there was an 11% chance it was wrong. Which means they might consider being a less circumspect when spruiking badly-measured polls.
Not playing fast and loose with statisticsÂ probably sells less papers. But that’s true of all kinds of misleading things one could do in journalism. And after all, the profession has its own ethical standards. The Australian Journalistsâ€™ Association Code of Ethics begins:
Report and interpret honestly, striving for accuracy, fairness and disclosure of all essential facts. Do not suppress relevant available facts, or give distorting emphasis.
I don’t thinkÂ today’s reporting meets that standard.Â
* This is being very generous to the pollsters. Justin Wolfers and I argue in a paper forthcoming in the Economic Record that the polls are much too volatile to be afflicted only by sampling error. In our view, the true standard error of the poll could be several times larger.