More skepticism about the "psychotics prefer Bush" study

Yesterday, I explained why a study that purports to show that psychotic patients tended to vote for President Bush in the 2004 election and is presently making the rounds to snarky gloating through the left-wing blogosphere is so utterly flawed that almost certainly does not mean what the author claims it does, given the data dredging, small sample size, and the failure even to consider alternative hypotheses to explain the observations. In my discussion, I complained that I had only found one skeptical take on the study among the credulous acceptance and use of the study to imply (or outright state) that Bush supporters are mentally ill.

Now I've found another skeptical take by Alon Levy. He emphasizes the small sample size and emphasizes, more than I did, the nonrandom sampling of the large population about which the study purports to make observations. He also goes on to discuss the problems with metastudies (the kind that are often presented as further evidence of a link between conservatism and various mental pathologies):

The buzzword in areas of social science that generate numerous studies is "meta-study." It's easy to botch these too, but when something draws enough buzz for there to be a hundred different data sets about it, a political hack will be able to use five that show the correlation is statistically significant.

In fact, usually there will be many more than five, because published studies have an existing bias in favor of data sets that show significant correlation. "There's no link between these two things" won't get you published unless it's a real hot-button issue like racial IQ differences, and even that only diminishes that bias but does not eliminate it.

This is the same sort of problem we have with metanalyses in medicine and is known as publication bias. The problems with metanalyses is a topic I've been meaning to blog about for a while now. Maybe next week. But in the meantime, I'll share a quote I heard from renowned burn surgeon Dr. Basil Pruitt while listening to SESAP 11:

"I have viewed metanalysis and its frequent use as sort of a means of turning a lady of the night into a Vestal Virgin."

Heh.

I'm not meaning to say that metanalyses can't produce valid results (many do), but physicians tend to forget that, according to the time-honored computer principle of GIGO ("garbage in, garbage out"), metanalyses are no better than the studies upon which they are based. They need to remember that, and evaluate metanalyses accordingly. So does anyone else who reads and cites such studies.

More like this

I have to take this opportunity to express a bit of disappointment in one of my fellow SB'ers. When I encounter a study that seems to confirm my biases, as a skeptic, I try very hard to be even more skeptical than usual, because I would hate to be caught trumpeting a weak or bogus study as evidence…
My fellow SBer Craig Hilberth at the Cheerful Oncologist writes about a meta-analysis that purports to show the positive effect of intercessory prayer. Neither Craig nor I have access to the full paper. But what we know is that the claim is that the meta-analysis shows a result of g=-0.171, p=0.…
One of the most persistent narratives latched on to by advocates of "integrative medicine" is that the "mind" can somehow "heal" the body. Sometimes, the claim is that such interventions work through "powerful placebo" effects. Sometimes it involves the abuse of emerging science, such overblown…
I sense a disturbance in the skeptical blogosphere. It is something that I half-expected, but, even so, it nonetheless somewhat surprised me when it arrived in the form of comments on my blog and e-mails from readers, fellow supporters of science-based medicine, and others asking me what I thought…

Skeptic Magazine, Mchael Shermer, Dawkins and Deepak Chopra

Comment by a skeptic and a response by Deepak Chopra reposted.

The God Delusion? Part 6

http://www.intentblog.com/archives/2006/12/the_god_delusio_5.html

Comment:

"You are so unfair Deepak. Here we have Richard Dawkins, who happens to be the world's foremost evolutionary biologist, and you put the following garbage in his mouth:

Deepak: "Dawkins falls prey, not to the delusion of God, but to the delusion of an all-mighty chance acting mindlessly through matter. He cannot admit the possibility of an ordering force in Nature".

With this statement you show your unbelievable ignorance!" posted by Skeptisch

Response:

"Dear Skeptisch,

It is not important who is in whose camp. We are all expressing our point of view from what little we know. I think there is some validity in all points of view including those of our harshest critics. I have invited Michael Shermer, Editor and Chief of Skeptic Magazine, to engage in debate at Caltech and also Harvard Divinity School. Michael has very graciously accepted this invitation. As we finalize the dates, I will keep everyone posted. In any case, he and I are going to express our points of view in Skeptic Magazine. You should see something in the next two weeks in eskeptic (http://www.skeptic.com/eskeptic/index.html).

Michael Shermer also invited Richard Dawkins to respond to a review of The God Delusion that I did for Skeptic Magazine. Michael replied,

"I still haven't heard back from my Caltech contact about availability of Beckman auditorium, but I just sent a reminder email requesting dates.

Deepak, Dawkins declined to respond to your review. He didn't offer a reason, just said he did not wish to respond.

Michael"

As I have mentioned in previous posts, Dawkins and I did have a very spirited exchange which has been filmed by Channel 4 (UK) and which I thoroughly enjoyed. We will post once it has been aired.

Skeptisch, I do enjoy reading your comments as they are extremely intelligent and do make one think. I am sure you are aware of the difference between skepticism and cynicism. Healthy skepticism and open-mindedness are essential ingredients for creativity. Michael Shermer is a good example of that. Cynicism and particularly closed-minded cynical mistrust is, on the other hand, quite unhealthy. In many studies it has been shown to be high risk factor for premature death from cardio-vascular disease.

Love,
Deepak"

By ChopraFan (not verified) on 01 Dec 2006 #permalink

Review by Deepak Chopra in Skeptic Magazine, questions about, Reading the book and Random Evolution.

A comment and a response reposted from:

http://www.intentblog.com/archives/2006/12/the_god_delusio_5.html

"Once again, Deepak, you really ought to read the book before criticizing it. You make yourself sound foolish when you criticize Dawkins for saying something he doesn't say.
Do you know how many times in "The God Delusion" Dawkins states that the theory of natural selection is the exact opposite of chance? Yet you state that "Dawkins falls prey, not to the delusion of God, but to the delusion of an all-mighty chance acting mindlessly through matter."
This is the opposite of what Dawkins says. The opposite. Read the book." Mithch_Wilkers

"Dear Mitch_Wilkers,

I have read Dawkins' book. You will see the review in Skeptic Magazine soon. As to your point, natural selection is not random but mutations are, a point that Dawkins makes as well. For now I am going to stop commenting on these threads. I thank you for all your feedback. I intend to do a full post addressing all the points that have been raised on these boards. I will do so, however, after I have finished writing the complete series.

Love,
Deepak"

By ChopraFan (not verified) on 01 Dec 2006 #permalink

I can't help but point out that presently, the term 'conservatism' is being applied to a faction of the republican party, which has pushed a radical spending program, a radical invasion, radical revisions to education, emergency planning, national security, intelligence gathering ... etc.
None of these purported links between conservatism and mental illness have adequately addressed the simple fact that 'conservatism' and related words are being used in a wide variety of starkly contradictory ways.

I dont see what is so peculiar about this study.

Americans in general seem to prefer Bush, at least in the 2004 election, so why shouldn't psychotics reflect that?

In fact Bush and his fellow travellers seem to be actively promoting conspirational world views of current events that should provide highly attractive explanatory paradigms for psychotics.

The idea of secret world wide fanatical muslim conspiracies that will stop at nothing to destroy imagined "freedoms" seem to be drawn from the same poisoned well as the Protocols of the Zion.

It seems, in reality, that the mass of the American elite seem to be gripped by same psychosis post 911 that gripped pre-Revolutionary Russia.

Except instead of Jew we have interpolated Islam.

By Rashid Harouin (not verified) on 01 Dec 2006 #permalink

Choprafan is just nicely illustrating how goofy and idiotic Chopra and his fans are by posting comments in posts that have nothing to do with Chopra. But maybe Choprafan is just naturally too dense to understand how to comment on the appropriate blog post.

Also... The statement by Rashid Harouin that "Americans in general seem to prefer Bush, at least in the 2004 election"... that is a debatable point. Also, it may not be true NOW! (evidenced by last month's election, which was not favorable to Bush)

On the comments by ChopraFan: Posting comments unrelated to the topic being discussed is just a cutpaste spamming. I do not have time, nor interest, to wade through all the PuffHo and Intentblog on this, btw. I have read pages of it and I see no reason to continue. Deepak is boring in his willful ignorance and repeated misrepresentation of Dawkins, Einstein, physics, biology, neurology...

On the "psychotics prefer Bush" study: the ready acceptance of and dissemination of this lousy study and its unwarranted conclusions is another example of the rampant confirmation bias that groupthink enhances.

I see this in folks who consider rightwing or leftwing as an important part of their identity. Very common in alties of either wing, for example, who suspend critical thought when they can just buy into a (marketing) gloss of, well, "alternative", boldly against the dominant gestalt as they perceive it.

So the very same nostrum can be embraced by a rightwinger as representing independent, do-it-yourself and pay-as-you-go, free-market medicine (vs the Big Brother socialistic bureacratic version he thinks the Left wants), and embraced by a leftwinger as representing alternative, touchy-feely, anti-patriarchy, non-Western medicine (vs the Big Brother fascistic bureaucratic version he thinks the Right wants). And each of these wingers can fancy their side the underdog.

Snarky gloating is cheap fun, and doing it on the backs of strangers with mental illness is creepy.

Rashid,

The study's author claims that it demostrates that as psychotic symptoms increase, the patient is increasingly likely to have voted for Bush '04. The point of the critiques of it is that the study appears to be so poorly designed that even if the conclusion is true, the study is most likely worthless as evidence of it due to: 1) not using proper data mining techinques, 2) the small size and non-representative selection of the sample, and 3) a design that does not include controls for obvious confounding factors.

You know, I'm four to six times more likely to vote for Bush before I've had my morning coffee as I am after I've had a full breakfast.

By Melissa G (not verified) on 01 Dec 2006 #permalink