Christine Neill and I have a paper out today, re-analysing a much-publicised 2006 study on the effect of the gun buyback on gun deaths. The original paper – by Jeanine Baker and Samara McPhedran – isn’t available online. But here’s how they summarised it on the Sporting Shootersâ€™ Association of Australia website* last year:Â
â€œThe reforms did not affect rates of firearm homicide in Australia.Â
The reforms could not be shown to alter rates of firearm suicide, because rates of suicide using other methods also began to decline in the late 1990s. …Â
It must be concluded that the gun buyback and restrictive legislative changes had no influence on firearm homicide in Australia.Â
The lack of effect of a massive buyback and associated legislative changes in the requirements for obtaining a firearm licence or legally possessing a firearm has significant implications for public and justice policy, not only for Australia, but internationally.â€Â Â
Our paper questions those conclusions. Here’s our abstract:
Weak Tests and Strong Conclusions: A Re-Analysis of Gun Deaths and the Australian Firearms Buyback
Christine Neill and Andrew Leigh
Using time series analysis on data from 1979-2004, Baker and McPhedran (2006) argue that the stricter gun laws introduced in the National Firearms Agreement (NFA) post-1996 did not affect firearm homicide rates, and may not have had an impact on the rate of gun suicide or accidental death by shooting. We revisit their analysis, and find that their results are not robust to: (a) using a longer time series; or (b) using the log of the rate rather than the level (to take account of the fact that the rate cannot fall below zero).Â We also show that claims that the authors had allowed both for method substitution and for underlying trends in suicide or homicide rates are misleading. The high variability in the data and the fragility of the results with respect to different specifications suggest that time series analysis cannot conclusively answer the question of whether the NFA led to lower gun deaths. Drawing strong conclusions from simple time series analysis is not warranted, but to the extent that this evidence points anywhere, it is towards the firearms buyback reducing gun deaths.
Doing time series analysis (ARIMA 1,1,1 to be precise) to test the impact ofÂ the gun buyback on deaths isn’t our favourite way of doing things. Statistical tests consistently reject the hypothesis that homicide rates follow a non-stationary process. But even when we use this approach, we find that all it takes is a couple of small tweaks in methodology to get much bigger coefficients on the effect of the gun buyback.
Overall, our paper estimatesÂ that 128-282 lives have been saved every yearÂ by the gun buyback. In other words, 1000-2500 Australians who are alive today would not be here if it hadn’t been for the buyback. This is broadly in line with two other Australian papers (Ozanne-Smith et al 2004 and Chapman et al 2006), and with what I consider to be the best US study (Mark Dugganâ€™s Journal of Political Economy paper More Guns, More Crime).
The standard Australian estimate of the value ofÂ a statistical life (from a recent paper by Peter Abelson) puts it at $2.5 million. Since the gun buyback cost $500 million, this indicates that even if we use the lowest reasonable estimates of its impact on homicide and suicide (128 lives saved per year), it paid for itself in just two years.
Most paper have an odd genesis, but there are a few unusual aspects to this one. The first is that I never would have gone to read Baker & McPhedran’s paperÂ if it wasn’t for my father picking up the phone to say ‘you should have a look at this study and see if it’s right’. That led toÂ a couple of blog entries, after which commenter Christine Neill and I (egged on by Justin Wolfers, who has done some great work re-analysing dodgy death penalty research), decided to write up a brief comment paper. Without appendices, it’s only 12 double-spaced pages, so do read it if you have time.
* The SSAA website seems to have been redesigned since then, and this research summary no longer appears there.