Helland and Tabarrok's paper 'Using Placebo Laws to Test "More Guns, Less Crime"' has been published in Advances in Economic Analysis & Policy. Their objective was to correct for serial correlations in the crime data. I explained earlier how, if crimes rates in adjacent counties tend to behave in the same way, results could wrongly appear to be statistically significant. There is a similar problem with crime rates in the same county in two successive years tending to be the same. Helland and Tabarrok use a technique (placebo laws) that deals with serial corelation as well as clustering within a state. They find that almost all of Lott's results are no longer statistically significant once corrected for serial correlation and clustering within state. Out of fifteen different estimates of the effect of carry laws on violent crime, only murder in the trend model was statistically significant, and you would expect one out of twenty estimates to be statistically significant by chance.
About the only bright note for Lott in Helland and Tabarrok's paper was the "Cross-Equation Restrictions" where they find statistically significant the fact that there were increases in property crime combined with decreases in violent crime. Unfortunately this only happens with the dummy variable model which Lott abandoned in the second edition of More Guns, Less Crime. In that edition Lott switched to looking at changes in crime trends, where there were decreases in violent crime and decreases in property crime. If increases in property crime are evidence for Lott's thesis, then decreases in property crimes must be evidence against it.
So what is Lott's take on Helland and Tabarrok's paper? Let me quote his entire comment(1/18/04 entry on his blog):
Eric Helland and Alex Taborrak's paper in Advances in Economic Policy and Analysis uses a "placebo law" approach to test the impact of right-to-carry laws on crime rates. Unlike the clustering approach, the placebo approach can also solve autocorrelation problems and Helland and Tabarrok find that murder, rape and robbery rate trends fall consistently after right-to-carry laws are adopted. Examining county level crime data for the U.S. from 1977 to 1997 they find that:
even with the revised standard errors the trend model indicates that shall-issue laws cause a large and significant drop in the murder trend rate."
Lott makes it look like their paper supports his thesis. He does not mention that the placebo approach produces larger standard errors than the clustering approach, making his results less certain. Nor does he mention that almost all his results become insignificant. Instead, he implies that they found significant decreases in rape and robbery when they did not. He also gets the dates wrong for their trend analysis, which just uses data from 1977 to 1992. This is an important difference, since if you use 1977 to 1997 the reduction in murder trends is halved and might not even be significant under the placebo approach.
Lott also makes it look like they set out to test his "More Guns, Less Crime" hypothesis and confirmed it. But as they made clear, their goal was different:
In this paper, we focus attention on the uncertainty surrounding the estimated effects, and we primarily follow Lott s formulation in the spirit of examining his results under the best-case scenario. Our goal is to explain, illustrate, and apply the placebo law technique to an issue of importance.
The best-case scenario for Lott is that only the murder trend declined significantly and even that is likely to go away if more data is considered.
This isn't the first time Lott has misrepresented their paper. Look here and here for more.