Why the Mona Lisa's eyes follow you around

[originally posted on March 16, 2005]

I've taken only two pictures of the Mona Lisa, and both turned out about the same: they captured the frenzied attempts of dozens of tourists trying to take a picture of the most-recognized image in the world. Here's the one I took last summer:

I hadn't noticed it until now, but the motion of the painting in the background seems to mirror the chaotic struggle of the tourists with their cameras. I wonder if the Louvre's curators placed it there as a sort of an inside joke.

But this post isn't about museum curators, it's about one feature of the Mona Lisa that's supposed to mark Leonardo as a true genius. As many have observed, "her eyes follow you around the room, moving as if by magic." Of course, it's not magic that makes her do that, it's simply an artifact of the fact that she's gazing directly at the painter. You can see this same effect in a snapshot of my daughter Nora:

In an article that received a fair amount of popular press, four researchers (Jan J. Koenderink, Andrea J. van Doorn, and Astrid M. L. Kappers of the University of Utrecht, and James T. Todd of Ohio State University) claim to have solved the mystery that has perplexed art fans for centuries ("Pointing Out of the Picture," Perception, 2004).

There are really two possibilities of why the eyes seem to follow us: either we actively "correct" for the fact that the picture is rotating out of our field of view, or we simply accept the rotated picture as "close enough."

If you rotate your computer monitor about 45 degrees to the left, you should see an image of Nora that looks something like this:

She appears to be a bit thinner, but still definitely "Nora," and still looking straight back at you. Koenderink et al. designed stimuli for four different possibilities: looking straight on at a "normal" picture (actually a computer display depicting a framed picture hanging on a wall), looking at that same picture with the computer monitor rotated 45 degrees, looking at a skewed picture like the picture of Nora above from straight on (simulating what the picture would look like if the monitor was rotated), and finally a skewed picture designed to look as if you were viewing it straight on when the monitor is actually rotated 45 degrees.

In every case, observers behaved as if the portrayed image (actually the rear of a nude sculpture) was facing towards them. They did seem to notice that the image was a bit "thinner" when it was rotated away from viewers, but this didn't bother them. In contrast, they could instantly tell that the "wall" that the picture was hanging on was "facing" a different direction. It's as if viewers have separate sets of rules for understanding the physical world and the world of flat pictures.

It's interesting to compare this study to Langton et al.'s work on how we determine the direction of someone's gaze. They had found that the outline of the head was the key determiner of perceived gaze direction. But when a picture is rotated, the outline of the head doesn't change—only how "thin" it is—and so we still think the image is "looking" at us.

I've noticed that most of the time when I see a plasma TV in a sports bar or some other venue (I haven't succumbed to the urge to buy one for my home), the standard TV picture is stretched to fit the extra wide "cinema" display. The result is that all the actors appear to be a bit heavier than they really are, but otherwise, they look just fine. Perhaps as cinema displays become more prevalent, the result will be a general preference for heavier people in real life, since that's the way everyone looks on TV. Wouldn't that be a satisfying development?