There's been an abundance of PowerPoint advice in the science blogosphere lately. Based on my personal experience, I'd say Chad and Amy give some good advice -- and it's advice that probably serves them well in their own presentations.
But I was curious about something different. There are plenty of places where you can find tips about PowerPoint. The bigger question is, do these tips actually help poor speakers improve their presentations? In other words, can offering basic presentation advice without actually showing someone how to be a more engaging speaker help them give better presentations?
We've all all seen or slept through god-awful talks when the speaker droned along in a somnambulant monotone. We've even seen other, more engaging speakers give similar talks -- and not surprisingly, the talks were more interesting. But if you can't (or won't) change your personality or speaking style, will listening to Chad's tips help?
To try to answer the question, I created five different presentations. Each presentation had the identical oral soundtrack (me reading from a perception textbook), but the accompanying PowerPoint slides were different. Participants were randomly assigned to one of the five presentations, and then told to watch carefully because they would be tested at the end. Before the test, they also read an unrelated passage from the same book, which they were also told they'd be tested on. In reality, we were only interested in the PowerPoint talk, so we never tested people on the reading passage.
The different slides were designed to follow (or ignore) specific bits of Chad's advice. In just a two-minute presentation, I couldn't work in every piece of advice, so I focused in on three:
* Text is death
* Explain your graphics, and
* Keep the background simple
In one version of the slides, I included only text: just long, bulleted lists summarizing what I was reading. In another version, I used two graphics, one of which was explained in detail, and one which was not. I also created a version of this presentation with no text at all, only the two graphs. Finally, I made a version of the text-only slides with a horrendous "pain" background graphic I downloaded from a chiropractor's web site. The fifth presentation was audio-only, with no slides at all.
If Chad's advice works, people should remember the information presented in the presentation with graphics that I explained the best. The text-only presentation with the distracting background should result in the poorest recall. So, how did our viewers do? Here are their average scores, broken down by the type of presentation they watched:
Holy psychedelic tie-dye, Batman! The worst presentation fared the best! The people watching the presentation with text-only slides and a repugnant background image had the highest scores. They scored significantly higher than any of the other groups. What could possibly be going on here?
One thing I had noticed while the results were coming in was that a surprisingly large number of respondents didn't answer any of the questions. One possibility is that they didn't know the answer, but perhaps a more reasonable explanation is that they were bored by the presentation and simply exited the survey before finishing. Let's re-analyze the data with these participants removed from the results:
Ah, that's better. Not only did the scores significantly improve, but now there is no significant difference between the scores for any of the presentations.
Wait a minute! No significant difference? But that would mean that the presentation format doesn't matter at all. You might as well not even bother with PowerPoint.
But there's one more possible explanation: all these averages may be distorting the larger picture. Let's take a closer look at the data:
Now we're looking at question-by-question averages for two different presentations: the presentation with charts and text, and the presentation with text only (both with a plain background). Even though there was no significant difference between the overall test scores, there were significant differences on a per-question basis. Let's focus in further, on question 1, which asked "When holding your hand in hot water, at what temperature (°C) does your body begin to NOT adapt to the pain?"
The chart that was supposed to help answer that question looked like this:
The correct answer was 47, but the graph depicts a richer vision of the data, which shows that even at 47 degrees, some adaptation may be possible, and hints that eventually we might even adapt to 48-degree water. Perhaps people who viewed the graph didn't have time to process all that information, or perhaps they were focusing on a different aspect of the data.
The text-only slide looked like this:
The text oversimplifies the data, but also leads viewers to the "correct" answer. Though 97 percent of them got the right answer, people viewing only this slide would probably have difficulty sketching a graph that shows how pain adaptation works, and might not be able to explain the basic concept verbally.
Overall, I'd say this study doesn't shed a whole lot of light on the question of how best to present information, other than to say "it depends on what you want your audience to get out of it." Perhaps the most interesting result is that the crummy "pain" graphic did appear to cause more readers to hang around and listen to the whole presentation, so Chad's advice to never use distracting background graphics may not actually be correct.
On the other hand, this was a 2-minute presentation for a disinterested web audience, not a 50-minute lecture. Different advice will almost certainly apply in a different context. This, of course, was Chad's first piece of advice:
* Know your audience
- Log in to post comments
More like this
I suggest that there's a problem going in of a poor premise or two.
Simple (or nonexistent) backgrounds, sure.
Explain your graphics - well, the goal should be to produce graphics that do not need explanation. Not always possible, but I wish I had a buck for every time I've heard someone say, "Now, let me just explain this graph; it's a bit complicated." Sure it is ... it's a whole lot easier to produce complex graphics that require explanation than simple graphics that communicate effectively on their own. Or even worse, "Now, I only want to make one point from this graph; let me highlight it for you." Do me a favour - make a simpler graphic that only covers the relevant stuff.
Text is death - this is the one that I think is particularly off-base and grossly overstated. One of the things that leads to poor graphics is that the person really couldn't write down a short and coherent explanation of the point of the slide if we took up a collection to pay him.
A technique I use and recommend is to make a text-only slide and try it out on someone not that familiar with the presentation. If they can come close to catching the message, I'm in good shape. Next step - will a graphic of some sort enhance the slide and improve the overall communication? If so, go for it. But I'm still surprised at how often plain old words can do the job.
These are presenter problems. PowerPoint is blameless; it's a tool. And generally, the more powerful a tool is, the greater good ... and the greater damage ... it can do.
In Chad's favor, "text is death" is given more explanation in his blog post. The point is not to put too much text on the slide, not to never use text at all. And as we see from the data, if you want your audience to remember specifically that 47 degrees appears to be the temperature at which we begin to be unable to adapt to pain, then text is your best bet.
But if you want them to have a larger understanding of how we adapt to pain, then I would argue that the graph is more effective.
A fair point, and I agree with much of what he says. However, the notion of "graphic good/text bad" does seem to be abroad in the land, at least based on what I see.
I'd also suggest that his advice misses one important point. The reality that the audience can read your text faster than you can speak it is true but irrelevant. You should "talk away from" your text. Talk about the point in different words, for example. I would agree heartily with "Reading the text on your slide is death."
OK, but I didn't suggest that there is not an important place for graphs. However, in this example, I'd suggest that a bit of text annotation on the graph could do a lot to clarify the couple of key points that I think are imbedded there.
I'd suggest that a bit of text annotation on the graph could do a lot to clarify the couple of key points that I think are imbedded there.
Yes, I agree -- and I think this points to some limitations of cognitive testing of rhetorical methods. The problem instantly becomes so complex that it's difficult to parse what the significant factors are.
Of course, the key omission in this study is that viewers don't see the speaker at all, and I think the physical interaction between the speaker and his/her audience may trump nearly all the nuances we're discussing here.
Agree completely.
Not sure that I'd use the term "nuances," however; in my little corner of things, presentations often get read later on without the presenter on hand. Therefore, it's often key that they be "self-presenting." But you're right - the best set of slides still benefits enormously from a competent and enthusiastic presenter.
Interesting!
Though, as you pointed out, this was a 2-minute presentation rather than a 50-minute one. Probably most tips are designed to prevent hideously boring or confusing presentations -- I know many of mine were -- and both of those factors tend to "accumulate" over longer and longer talks.
So you might not see an effect of them simply because 2 minutes isn't long enough for either the audience to get so bored they zone out and miss large bits of the talk, or to have initial mild confusion escalate into extreme conclusion as the talk goes on and builds on previously presented material. (Also, there is the confusion that results from being so bored that you missed the important information!)
I definitely think the graph vs. text comparison is interesting, though. Kind of points out that if you just want your audience to remember key points, bullets are a great way to do that, but if you want a richer and more nuanced understanding of the data you should probably use a graph and spend more time describing it or highlighting its key points with text.
Since comments here don't support images, Scott emailed me an example of an annotated figure. Click here to see it.
"2 minutes isn't long enough for either the audience to get so bored they zone out and miss large bits of the talk."
Never underestimate the audience's capacity for boredom! I had the audio-only version, and I totally zoned out midway through. Powerpoint slides help to hold my attention or reorient me after doing that at a real talk. In fact, in order to keep myself from zoning out again, I started visualizing the Powerpoint slides I would make to go with the talk. . .
Dave, I'd be super interested in your line graph for the presentation with the pain graphic as a background. There's of course a lot of research out there on emotion and memory, and specifically for relevant material embedded within an emotional context. Perhaps for initial material (the first question addresses this? - I don't know as I haven't listened to the movie yet) there would be some sort of heightened memory due to arousal from the intro screen, before the pain graphic becomes muted. Then, memory for material later in the two minute period might be reduced when the pain graphic is muted and we may adapt to the emotional stimulation (how meta, I guess, as we may adapt to a presentation about adaptation). The overall effect then is that the pain graphic background results in roughly the same memory performance as the other manipulations.
Okay, I've made two graphs that include all the questions and all the conditions. The graph I initially posted kept the questions in the order they were asked, but I think you're interested in the how they correspond to the order of the presentation, so I've made another graph that changes the order to match where the relevant information was given in the presentation.
Questions in the order asked
Questions in the order the information was presented
Also, you may be interested in the questions themselves, so here they are:
1. When holding your hand in hot water, at what temperature (°C) does your body begin to NOT adapt to the pain? (47)
2. When two teeth are stimulated with mild discomfort that is NOT above the threshold of pain
The combined stimulus does not cross the threshold of pain
The combined stimulus always crosses the threshold of pain
* The combined stimulus may cross the threshold of pain
The combined stimulus is roughly equal to the linear sum of the two pains experienced separately
3. How is pain experienced across modalities?
It is always roughly equal to the linear sum of the separate pains involved
In the case of dental pain and hearing, it is roughly equal to the linear sum of the separate pains involved
* In the case of hearing and electric shock, it is roughly equal to the linear sum of the separate pains involved
In the case of hearing and sight, it is roughly equal to the linear sum of the separate pains involved
4. For what types of pain has it been shown that humans can adapt?
shock, sound, and needles
needles, heat, and sound
sight, heat, and dental pain
* dental pain, needles, heat, and cold
Thanks, Dave! My suggestion is addressed by your graph 2 - with the order of presentation on the x-axis. Clearly I was wrong! There seems to be no initial benefit from any arousal engendered by the pain picture.
Interesting results. In my experience, I've found that a good presenter is just a good presenter. I've seen great professors that literally wouldn't be able to lecture effectively without Powerpoint, and I've seen other great professors that simply stand up and start talking. I've also seen bad versions of both, and I suspect that it comes down to their investments in their students. The bad ones seem to view teaching as a burden on their fellowships, whereas the good ones seemed genuinely interested in getting the students to learn.
I also wrote some advice on public speaking a while back. Looking at it now, there are a few things I would change or phrase differently, but I still like it.