Do we need lawyers to resolve scientific disagreements?

In most cases, scientific disagreements are resolved in the labs, at conferences, or in exchanges in journals. Sometimes the disagreements are drawn out, sometimes feelings are hurt, but it hardly ever comes to a defamation suit.

Someone forgot to send John Lott the memo.

From the Chicago Tribune:

John Lott Jr. of Virginia, a former U[niversity] of C[hicago] visiting professor, alleges that [Steven] Levitt defamed him in the book ["Freakonomics"] by claiming that other scholars had tried and failed to confirm Lott's conclusion that allowing people to carry concealed weapons reduces crime. Publishers Weekly ranked "Freakonomics" eighth this week for non-fiction hardcover books.

According to Levitt's book: "When other scholars have tried to replicate [Lott's] results, they found that right-to-carry laws simply don't bring down crime."

But according to Lott's lawsuit: "In fact, every time that an economist or other researcher has replicated Lott's research, he or she has confirmed Lott's conclusion."

By suggesting that Lott's results could not be replicated, Levitt is "alleging that Lott falsified his results," the lawsuit says. ...

Lott acknowledged in the suit that some scholars have disagreed with his conclusions. But he said those researchers used "different data or methods to analyze the relationship between gun-control laws and crime" and made no attempt to "replicate" Lott's work.

(Bold emphasis added.)

Let's review. The defamation lawsuit claims:

1. It is defamatory to claim that attempts to confirm another scientist's conclusions were unsuccessful.
2. Claiming that attempts to replicate research have not succeeded amounts to claiming that the original research was falsified.
3. Lott knows for a fact that every economist or other researcher who has tried to replicate his research has suceeded.

Hey, maybe economics doesn't work the same way science does. But, in the case that it's trying to, it's worth noting that there are lots of explanations for failed attempt to replicate results. Maybe the researcher trying to replicate the results made a mistake. Maybe the original researcher made a mistake. Maybe there's some parameter not yet sufficiently controlled in the studies (perhaps because it hasn't been noticed yet, or recognized to be important to the results). Falsification is way down the list of things you check when your attempts to replicate research founder.

Also, there's something perplexing about Lott's claim that researchers who disagreed with his results "used 'different data or methods to analyze the relationship between gun-control laws and crime' and made no attempt to 'replicate' " his research. Presumably the point of Lott's research was to demonstrate that there was a robust relationship between gun-control laws and crime -- one that wasn't an artifact produced by one particular data set or one particular methodology for analyzing it. At the very least, wouldn't meaningful replication include seeing if the relationship persists when additional data are examined? Or does Lott thinks "replication" amounts to no more than checking his math on the problem he set up?

To the extent that science is about figuring out how things really are, a scientist needs to have a thick enough hide to cope if his or her results end up not being confirmed. It may sting a little, but it's better to have a clearer picture of what's what than to protect fragile egos by refraining from subjecting results to serious scrutiny.

And by the way, calling out the lawyers when other researchers disagree with your conclusions tends to make it look like you have something to hide. I'm not saying one shouldn't take legal action, when appropriate. But was anyone even suggesting falsification before Lott mentioned it in his suit?

(Hat tip: Deltoid)