When you selected the college or university you planned to attend, how did you do it? Did you read narratives offered by college guides? Did you compare relevant statistics such as the student/teacher ratio and percent of students who went on to graduate school? Did you listen to the advice of older friends who had attended the school? Or did you visit the school in person? And which of these factors had the most impact on your decision?
For many students, the campus visit is the deal-maker (or breaker). Here at Davidson I've met many students who said they had been undecided, but when they visited the campus, they "fell in love." I've also met students who regretted their decision to come here. And, no doubt, there have been students who visited campus on a rainy day, attended a boring lecture, and found the Davidson students unwelcoming and uninspired, and so went elsewhere. Were they giving proper weight to the campus visit in making their decisions?
Many studies have addressed how people make important decisions like which college to attend, but one of the classics was conducted way back in the 1980s, by a team led by Richard Nisbett. Even at this time, psychologists knew that people often make decisions that aren't supported by seemingly obvious evidence. In the 1970s, Daniel Kahneman and Amos Tversky had asked people which of two hospitals was more likely to have a day with at least 60% male babies born, a hospital with 15 births per day, or one with 45 births per day. Most people said that the chances were the same in each hospital, but in fact there should be more variance in the male-female ratio in the smaller hospital, which means it would have more 60-percent days.
But do people really just not understand the statistics behind such a prediction, or do they not think to apply what they know? Nisbett's team asked 46 students to imagine they were an explorer who had landed on a remote South Pacific island. There they found a variety of things, including a new species of bird, a "shreeble," which was blue, and a native man, a member of the Barratos tribe, who was obese. Then they were asked what portion of all shreebles were blue, all Barratos were obese, and so on, for several different novel things found on the island. These questions were repeated for varying numbers of cases seen, like if they had seen 3 shreebles, all blue, or 20 Barratos, all obese. Here are some of the results:
For shreebles, the respondents assumed that one bird is a pretty good representation of the whole population. But one obese Barratos was not taken to be convincing evidence that all Barratos are obese. As the number of obese examples increases, so does the estimate of what portion of the entire population is obese. This is entirely in keeping with statistical logic (of course, there are other possible explanations: perhaps obese Barratos just have more trouble hiding from strange intruders on huge ships).
So if it's true that college students can use statistical reasoning in some cases, what determines whether they'll use it? In a new experiment, 157 students were given one of two scenarios describing one student's experiences deciding which of two colleges to attend.
In both scenarios, "Daniel" had been accepted at both Ivy U. and Liberal College, and several of his older friends were attending each. The friends at Ivy U. said they had many complaints about the education, personal, and social life there, while the friends at Liberal College said everything was great. Daniel decided to visit each campus for a day, and had a terrible time at Liberal College and a great time at Ivy U.
But the second scenario added a critical detail:
Before his visit, Daniel proceeded systematically to draw up a long list, for both colleges, of all the classes which might interest him and all the places and activities on campus that he wanted to see. From each list, he randomly selected several classes and activities to visit, and several spots to look at (by blindly dropping a pencil on each list of alternatives and seeing where the point landed).
When this detail was omitted, 74 percent of students in the study said Daniel should go to Ivy U., which his friends didn't like, but where he had a good time for a day. When it was included, only 56 percent of respondents said he should go to Ivy U. -- a significant difference. The researchers say the reason for the difference is that the second group had been led by the extra detail to think about the problem using statistical reasoning: his friends had been at each school much longer than he had, so their experience should probably carry more weight than his own brief visit.
In another experiment, they found that experienced athletes and actors were more likely to say a great try-out didn't necessarily imply a long-lasting talent in their respective activities (compared to non-athletes and actors), because they understood that it's statistically possible to have a single good try-out, which isn't necessarily representative of day-in, day-out ability.
Overall, the researchers found that a predisposition to look at data statistically (either because of hint given by the experimenters, the nature of the data, or the nature of the individual's experience) led to more statistical reasoning. In addition, people who had been trained in statistics -- both formally and informally in very brief training sessions -- were more likely to use statistical reasoning to solve problems.
Nisbett's team understood that statistical reasoning isn't the only relevant way to make decisions. In Daniel's case, he simply may have been better suited for Ivy U. than Liberal College, despite the experiences of his friends -- but it would be wrong to consider only his own brief experience in recommending what he should do.
Unfortunately, a quick look across the news pages reveals that statistical reasoning hasn't improved since 1983, and may even have declined: witness the widespread acceptance of pseudoscience like homeopathy, anti-vaccination, anti-global warming, and anti-evolution thinking. And college admissions offices will tell you that getting prospective students to visit campus is a key way to ensure they actually attend.
Nisbett, R.E., Krantz, D.H., Jepson, C., & Kunda, Z (1983). The use of statistical heuristics in everyday inductive reasoning Psychological Review, 90 (4), 339-363
- Log in to post comments
More like this
This article says, "in fact there should be more variance in the male-female ratio in the smaller hospital, which means it would have more 60-percent days. But do people really just not understand the statistics behind such a prediction..."
Quoi? I don't understand the statistics behind such a prediction. Please explain!
With large sample sizes (e.g. 45 births per day) the sample mean approaches the population mean. In this instance the population mean for male births would be 50% or very close to it. Therefore, when there are fewer daily births at a hospital, there is a higher likelihood there could be a day when all births were male (or 60% male), a very unlikely event at a larger hospital. If you use different sized hospitals, like one with 4 births per day and the other with 2,000 per day, it may be more clear.
The lack of reasoning drives me crazy. Even when presented with evidence people stick to their opinion contrary to the evidence. And we wonder why the world is as it is.
Tyler, I think the more relevant point is that with larger sample sizes, variance of the sample mean decreases. The sample mean is always expected to be the same as the population mean (assuming appropriate random sampling).
An increased acceptance of pseudoscience could easily be due to easier communications technology that allows people with similar (bad) opinions to gather together for group reinforcement, or for celebrities / authorities to spread their message to wider audiences. It doesn't help that certain ideas are stereotyped together: homeopathy w/ eating organic foods for instance.
The bigger question, really, is how much DO people apply what they learn throughout their educational careers to everyday experiences in life? My guess would be, not much, except for on-the-job training and learning of certain skills. I think people are just not used to (not expected to) apply their accumulated knowledge that much. (Well, learned knowledge from classes/reading as opposed to pop culture / social information.)
Leah:
I think the easiest way to understand this (if you've never taken a statistics class) is by thinking about coin flips. The odds of having a boy vs. girl is close to 50/50, like the chance of flipping a coin heads vs. tails.
If you're using a fair coin, you expect it to come up heads half of the time. So, take the extreme example since it makes the point easiest to understand, how often will you expect the coin to come up 100% heads? It depends how many times you're flipping it! If you only flip it twice, there's a very good chance (25%) it will come up heads both times. If you flip it three times, you still have a decent chance of getting 100% heads (1/8). If you flip it a million times, there's close to zero chance that 100% of the results will be heads.
@Hao- So, not to nit pick, but actually the variance itself increases with sample size, but it increases proportional to the square root of the sample size, assuming births are independent identically distributed events. Thus the variance as a proportion of the sample size does decrease as the sample size increases. Since the question is asked in terms of percentages, the variance as a proportion of the sample size is what we care about. If you tweaked the question to say "In which hospital is it more likely that there will be five more boys born that girls on a given day?", the correct answer would be the larger hospital.
i picked my college because it was close to my house 0_o
I wonder if there's not something more complex happening here as well, something that has nothing to do with statistics. People in conversation will very often re-interpet questions based on what they think the person means, ignoring complexity, and in other cases simply not pay enough attention to the question itself. The original question is unnecessarily complex, the addition of the condition of a single day being unusual, so people could be answering a subtly different question, as they are subconsciouly rephrasing what they think the questioner is trying to ascertain.
The question could also be interpreted here as meaning a single day at any point over an extended period of time, and not just a single day taken out of context. If so the likelihood would become 60% for either given a sufficiently long period of time of analysis.
Excuse me but Homeopathy is not a pseudo science and it is really getting tiresome to read this sort of innuendo against it from otherwise intelligent people.
The science behind Homeopathy remains under research by genuine scientists and the mechanism of its curative effect will eventually be uncovered.
The principal attacks against Homeopathy rest on its utilization of high dilution solutions in which all molecules of the curative substance have apparently been diluted away. Now here comes the tricky part - one must distinguish between the real research which seems to show that such high dilutions can and do have a biological effect, and the innuendo and "common sense" attacks against this, which revolve around letting everyone's high school "knowledge" of chemistry take precedence over what the researchers are finding.
This leads to a curious chicken - egg paradox in Homeopathic research - people complain that there is scanty or inadequate or insufficient research but then shout that their "common sense" or high school or college chemistry "knowledge" is sufficient so that the whole thing is impossible and therefore research funds should be denied!
If there were no experiments showing any biological effect from such high dilutions, then we would still have to deal with the numerous clinical and case history evidence in favor of homeopathy, without rationalizing it away as "placebo" which is the current fallacy used for accepting that the cures happened and then attempting to explain it away.
The experiments of M. Ennis (Inflammation Research, vol53, p181) which have been repeated multiple times in several labs, clearly show biological activity being stimulated by high dilution solutions in which all molecules of the stimulant have been diluted away.
So, insulting the genuine scientists who are researching this, as well as the practitioners of Homeopathy - genuine MD's and other professionals, by calling it a pseudo-science, is really unfair.
Thanks, Tyler & Samantha.
"principal attacks against Homeopathy rest on...high dilution solutions in which all molecules of the curative substance have apparently been diluted away"
Uhm...nope that's not what most of the peer reviewed scientific community has a problem with. Personally, I simply can't see repeatable results in large enough samples to justify taking it any more seriously than Phrenology. Just like Phrenology there were really smart, earnest scientists who got mixed up in the cult of the moment. So no offense, but the tenet of statistical reasoning rests on using the facts that you can demonstrate to draw conclusions on. Homeopathy is heavy on scientific sounding words but is really light on demonstrable facts.
Our daughter - now a sophomore and loving her college, the area+ used statistics to hone down her search and characteristics such as small classes, rural area to apply to and then picked among 6 that accepted her. The visit cinched the deal say that it is the users who decide if they use statistics. Some will, most won't - my seat of the pants opinion.
After all, statistics only apply to groups, not individuals. So it is not always easy to apply statistics to yourself.
Stats are very useful when addressing well defined questions (does a medicine work, or not).
In other areas of inquiry, though, there is a lot of other weighing that sits outside statistics. Statistics measure metrics (usually of elements that are easy to quantify) but there are lots of factors outside of standardized metrics. Assuming you've ruled out the seriously underperforming schools, it may be quite valid to use subjective or gut level feeling as a factor. After all you're going to have to spend a LOT of time there, and if you're happy, you will do better. By paying too much emphasis on statistics
Statistics is pretty much about plotting an 'average' experience. Yes I know it's not technically just average ... but it is a matter of combining the good and bad experiences of many individuals into a single value, which represents the actual experience of few if any of them. Statistically many new businesses fail, many marriages fail, statistically these are both bad bets; but in the real world there are many successes, and we'd be poorer if people chose to follow the statistically safe route.
Which College to attend??? Let me see. In which city does Man United play? Bingo!
No offense, but anyone who thinks that statistics (as a tool for learning/knowing about things) can only tell you about groups and central tendencies isn't too familiar with it. The starting point is often, but not always, at a group level, but in some cases it is possible to derive very accurate predictions for individuals.
Would you prefer counterknowledge?
@Hao- So, not to nit pick, but actually the variance itself increases with sample size, but it increases proportional to the square root of the sample size, assuming births are independent identically distributed events. Thus the variance as a proportion of the sample size does decrease as the sample size increases. Since the question is asked in terms of percentages, the variance as a proportion of the sample size is what we care about. If you tweaked the question to say "In which hospital is it more likely that there will be five more boys born that girls on a given day?", the correct answer would be the larger hospital.
As far as I know more male are born... Male-female balance is reached when they are in their fourties and after this age there is increase in female quantity camparing to male. So I guess in both hospitals there should be more male-birth.
"anti-global warming"
What statistical representive sample you do have of World temperatures? How many samples do you have even of today Ocean temperatures that make up 70% or Earth surface? And i am not even talking about 100 years ago.
Anti-science is here in this blog for political and herd behavior. Pseudo Science.
My stat teacher used to complain about how students learned everything correctly and then seemed to forget everything as soon as they stepped outside of class.
But it's probably wrong to assume people are rational agents, no? In the end I think most people make their decisions for reasons they have no clue about. Maybe it's better to not rely on statistics too much evolutationary speaking, it's taking risks that results in exceptional things (which is also the case for evolution itself?).
Re: Shreebles and Barratos:
I'm not sure that strict statistical reasoning is appropriate in this context. When considering the Barratos, the observer is likely to take into account all sorts of background information, such as that in most human societies (including -- so far -- the United States) the obese are a minority. Thus, one obese Barrato is not evidence of anything much, whereas 20 obese Barratos are surprising and evidence of something unusual going on.
By contrast, being a member of a bird species usually means being the same color as other members of that species (or at at a minimum, in a sexually dimorphous species, half the other members). Thus, one blue shreeble is strong evidence that either 50% or 100% of shreebles are blue. Three blue shreebles still point to either a 50% or a 100% figure.
I'm just surprised that, after seeing 20 blue shreebles, the respondents didn't conclude that 100% of shreebles are blue.
@Sparky: I'd still call that statistical reasoning, it's just a little more Bayesian. One has a prior probability on things like intraspecific color variation in birds..