A cautionary note about fMRI studies

I've been distracted lately — it's end of the world semester time — and so I didn't have time to comment on this recent PNAS paper that reports on dramatic sex differences in the brains of men and women. Fortunately, I can just tell you to go read Christian Jarrett, who explains most of the flaws in the study, or you can look at these graphical illustrations of the magnitude of the differences. I just want to add two lesser points.

First, let's all be really careful about the overselling of fMRI, 'k? It's a powerful tool, but it's got serious spatial and temporal resolution limitations, and it is not, as many in the public seem to think, visualizing directly the electrical signaling of neurons. It's imaging the broader physiological activity — respiration, oxygen flux, vascular changes — in small chunks of the brain. If you're ever going to talk about fMRI, I recommend that you read Nick Logothetis's paper that cooly assesses the state of affairs with fMRI.

The limitations of fMRI are not related to physics or poor engineering, and are unlikely to be resolved by increasing the sophistication and power of the scanners; they are instead due to the circuitry and functional organization of the brain, as well as to inappropriate experimental protocols that ignore this organization. The fMRI signal cannot easily differentiate between function-specific processing and neuromodulation, between bottom-up and top-down signals, and it may potentially confuse excitation and inhibition. The magnitude of the fMRI signal cannot be quantified to reflect accurately differences between brain regions, or between tasks within the same region. The origin of the latter problem is not due to our current inability to estimate accurately cerebral metabolic rate of oxygen (CMRO2) from the BOLD signal, but to the fact that haemodynamic responses are sensitive to the size of the activated population, which may change as the sparsity of neural representations varies spatially and temporally. In cortical regions in which stimulus- or task-related perceptual or cognitive capacities are sparsely represented (for example, instantiated in the activity of a very small number of neurons), volume transmission— which probably underlies the altered states of motivation, attention, learning and memory—may dominate haemodynamic responses and make it impossible to deduce the exact role of the area in the task at hand. Neuromodulation is also likely to affect the ultimate spatiotemporal resolution of the signal.

Just so you don't think this is a paper ragging on the technique, let me balance that with another quote. It's a very even-handed paper that discusses fMRI honestly.

This having been said, and despite its shortcomings, fMRI is cur- rently the best tool we have for gaining insights into brain function and formulating interesting and eventually testable hypotheses, even though the plausibility of these hypotheses critically depends on used magnetic resonance technology, experimental protocol, statistical analysis and insightful modelling. Theories on the brain’s functional organization (not just modelling of data) will probably be the best strategy for optimizing all of the above. Hypotheses formulated on the basis of fMRI experiments are unlikely to be analytically tested with fMRI itself in terms of neural mechanisms, and this is unlikely to change any time in the near future.

The other point I want to mention is that there's a lot of extremely cool data visualization stuff going on in fMRI studies, and also that what you're really seeing is data that has been grandly massaged. Imagine that I take a photo of my wife's hand, and my hand. If I just showed you the raw images, the differences would be obvious, and you'd probably have no problem recognizing which was the man's and which was the woman's. This is not true of the raw data from two brain scans from a woman and a man — without all kinds of processing and data extraction (legitimate operations, mind you) it would look like a hash of noise. But do we look at two people's hands, with obvious differences, and announce that we've made a dramatic discovery that sex differences are hardwired? So why do scientists get away with it if it involves sticking heads in a very expensive machine that makes funny noises?

Furthermore, the processing done in this distance was designed to abstract and highlight the differences, amplifying their perception. Take the photos of my wife's hand and mine, and now do some jazzy enhancement to subtract out anything that is the same, so the bulk of the images are erased as unimportant, and then pseudocolor the remainder into neon reds and blues, and display it in 3 dimensions, rotating. That would be a weird, complex image far removed from the mundane familiarity of the shape of the hand, but it would emphasize real differences to an extraordinary degree, while obscuring all of the similarities, and give a false impression of the magnitude of the differences.

Let's not assign all the differences to something genetic, either (although of course, some are modulated by biological -- but not really genetic -- differences). If you were to do the same comparison of my hand to my father's, you'd see much grander differences than between mine and my wife's. He was a manual laborer and mechanic, and I recall doing the comparison myself: his hands were muscular, powerful, calloused, deeply lined. I should have gotten a photo while he was alive so I could publish it in PNAS, touting significant biological differences between father and son.

(via Stephanie)

Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453(7197):869-78. doi: 10.1038/nature06976.