"We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work." -Richard Feynman
Did you hear the news? A game-changing story about the Universe has just come out! Something is vastly, spectacularly different from the way we thought, and it will revolutionize the way we think about the most basic, fundamental properties of our very existence.
Image credit: U&I Software, http://www.uisoftware.com/.
Blah, blah, blah. Or, you know, not.
Extraordinary claims like this come out all the time: cosmological inflation is unnecessary, neutrinos can travel faster-than-light, our experiment has detected dark matter, the fundamental constants aren't really constant, and so on. If you keep your ear to the ground (or listen to the more speculative sources), you're bound to hear at least one or two of these a month.
And they're also almost always wrong. (And I hesitated there to put almost in that sentence.) Here's why.
Image credit: D. Horn et al., Phys. Rev. Lett. 36, 1236-1239 (1976); the Oops-Leon particle.
Let's say you've got a new piece of data, one that doesn't line up with the presently best-accepted theory. Maybe you made an observation that didn't line up with what that theory predicted. There's a first-line-of-defense that this data must pass: is this observation robust?
In other words, when you perform this experiment or make this observation over and over again, will you get the same results? Will you see the same signal?
Image credit: Null Result from the Center For Inquiry, re: Power Balance bracelets.
Assuming the answer to this is yes, and the experiment / observation is repeatable, there's another test you must do: do you understand where your errors and noise come from?
This one is hugely important, and one where practically all of the claims fail. Let me explain.
Image credit: J.K. Webb et al., Phys. Rev. Lett. 107, 191101 (2011).
This is the graph that shows how the fundamental constant, α, varies throughout space. The problem with this data, unfortunately, is that the variation is a signal superimposed on a giant amount of noise, and we don't understand what causes that noise.
In other words, we can't say "observation X shows that Y is happening," because Y doesn't fully explain observation X; nothing that we know does! Until you understand your background and your noise, you have no business drawing fundamental, scientific conclusions about what you've seen.
Image credit: R. Bernabei et al., (2008), from http://xxx.lanl.gov/abs/0804.2741.
This is the same story with DAMA or CoGeNT's claims that they've detected dark matter: they see a modulation in their signal as their detector moves about the Sun. But what the graph, above, doesn't show you is that this "signal" is pulled out of a background that's 50 times as large.
What is this background that we're looking at? They have no idea.
Is it irresponsible to conclude "therefore, dark matter?" You bet. Because if you don't understand the forest, you can't draw conclusions about the trees.
Image credit: CERN / LHC, via research.gov.
There's always a lot of hype floating around, because people get excited about new, revolutionary ideas. I can't blame them for that; I do, too. But this is science, and we have high standards of evidence here! For instance, if you're going to claim that the Universe is going to end soon, because the quantum vacuum is unstable, you'd better check that the quantum vacuum is actually unstable. (It's actually incredibly stable, as people have been saying for years.)
The Higgs Boson is exactly the mass it needed to be to stabilize the Standard Model all the way up to very high energies, well within the error bars of all measured particle masses. So you can read all the hype stories you want, but they're not right.
Image credit: ©2001 CERN.
The next thing coming down the hype-machine pipeline is surely going to be the Alpha Magnetic Spectrometer's results, due for release in a couple of weeks. Flying in orbit around the Earth, it's detecting cosmic rays, or high-energy particles originating from the Sun, the galaxy, and even extragalactic sources in the Universe!
It's conceivable that annihilating dark matter can cause an excess in cosmic rays. After all, one can easily design a scenario where that's exactly the case.
Image credit: MAGIC collaboration / U.A. de Barcelona.
Does this mean that we can look at the data, and know right away whether we've seen dark matter or not?
Almost definitely not! There are a ton of sources for cosmic rays, and they're very poorly understood. It would be shocking if there weren't an excess of cosmic rays in some locations at some energies; we don't fully understand where they come from or how they're produced.
But you can bet that won't be what the news outlets (and Sam Ting) will tell you. But remember.
Image credit: Center for Inquiry.
Always remember, extraordinary claims require extraordinary evidence.
For a scientific theory, that means:
- The new theory must be consistent with everything that came before,
- The new theory must explain this new observation, and
- It must lead to a new prediction of an observable phenomena which can go out and be tested.
Don't be fooled by these claims; they're a dime-a-dozen. But one that holds up to scrutiny?
Now, that's science. Not hype.
- Log in to post comments
More like this
Coincido es, arrogante y petulante hacer afirmaciones únicamente con un 1% de "conocimiento" de algo? bueno...
A laser beam strong enough to punch a hole into the Earth's core would be much more exciting as hype than any space travel.
Nice banner from extraordinary-claims.com.
I'll add it to my series of wallpapers after I've modified it to read: "Claims Require Evidence".
The word "Extraordinary" adds nothing at all relevant or useful.
Alan L, try googling "extraordinary claims require extraordinary evidence"
"A game-changing story about the Universe has just come out! Something is vastly, spectacularly different from the way we thought, and it will revolutionize the way we think about the most basic, fundamental properties of our very existence."
Absolutely. And if the cover of New Scientist doesn't say that, the editor will want to know why, because someone's not doing their job properly.
HA! I was hoping you'd talk about the story I saw on /. titled "Does the Higgs Boson Reveal Our Universe's Doomsday?"
We're DOOMED! DOOMED I say!
http://science.slashdot.org/story/13/02/19/2151238/does-the-higgs-boson…
Seems to me the worst kind of science hype is reporting that suggests a cure to a deadly disease is around the corner when it isn't. Sure, hype in the fundamental sciences is lamentable. But in reality, the stakes are relatively low. In the end, it does far less damage than stories that give false hope to people with sick kids.
Let's not beat up on the faster than light neutrino people.. they did just put it out there and ask for help. They said they didn't feel the results were correct, but they could not get rid of them.. asked others to review the experiment and maybe try themselves...
of course they were embarassed when it turned out to be a loose cable but they made no big claims.
Which brings up an important point about science journalism (or just regular journalism coverage about science, like what might appear on the front page of a local newspaper). Very often,the journalist doesn't write the headline. A more senior editor does that. And the editor is not thinking about accuracy so much as 'punch' and audience draw.
my hero!
Most scientific theories are wrong or not even wrong. Nevertheless, most scientist do their diligent best. e.g. the faster than light neutrino guys did their diligence best. ditto #9 Kevin Dowd above
To me the "The Worst Kind of Science Hype" is to make predictions that are extremely outside the realm of possible measurement verification.
"If we were born on a habitable planet 100 billion years from now, we’d conclude we were the only galaxy in the Universe." Ethan Siegel
In my opinion, such a naked prediction belongs in science fiction not science. Yes it is Mathematical Science Fiction, but still IMO science fiction because no problems, doubts, disclaimers, assumptions, margins of errors, discussion of theoretical problems, etc.. are discussed.
I prefer this Richard Feynman quote, "if you're doing an experiment, you should report everything that you think might make it invalid — not only what you think is right about it; other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked — to make sure the other fellow can tell they have been eliminated. Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can — if you know anything at all wrong, or possibly wrong — to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. "
On the other hand, the Feb 19, 2013 statement, "In two weeks scientists will publish the first set of results from the Alpha Magnetic Spectrometer, a $2billion particle collector mounted on the outside of the International Space Station in a bid to find dark matter." is science to look forward to.
Maybe even new evidence will require some significant modifications to the standard model of cosmology which will change little predictions like Where will we all be in 100 billion years?
Yes, yes, I know this is MY pet peeve, and neither Ethan Seigel, Kip Thorne, Briane Greene, nor many other scientists seem to care.
"Leading the experiment is Nobel Laureate Samuel Ting, a physicist with the Massachusetts Institute of Technology, who said: 'It will not be a minor paper.'" http://www.dailymail.co.uk/sciencetech/article-2280975/Have-dark-matter…
Yeah, the FTL neutrino thing can be placed squarely on the media. All the articles I saw had quotes from the researchers saying that they were not convinced and were seeking help to find out where they had gone wrong. The breathless headlines attached to those articles were the problem.
The AMS hype clearly is in some part being fueled by Ting. He's claiming big news while being extremely coy about what it actually is, and what it's degree of significance might be.
Though I tend to want to give the benefit of the doubt and assume it's still the headline writers jumping to conclusions.
After all not long ago John Grotzinger, lead scientist of the Curiosity program, fueled rampant speculation when he said data from the sample analysis instruments was "really good" and would be "one for the history books". Could MSL have detected alien life?! Turns out all he was really saying was that the data was of high quality, and that doing such analysis on another planet was historic.
Is Ting really referring to mind blowing conclusions his paper will draw, or does "it will not be a minor paper" simply refer to the significance of the first batch of data accumulated by a major instrument?
I guess we'll see. I look forward to Ethan's analysis.
@H #5
I use Bing, but I don't need to look up this pompous hyped up declaration popularized by, I believe, Carl Sagan.
Funny that I did not see climate change in your list of extraordinary claims requiring extraordinary evidence. That one would be at the top of my list.
Dear Dr. Siegel: I liked your article, but was surprised by your comments regarding Webb et al. and their work on varying alpha. I followed the link to your 11/2011 article about Webb. I thought your assessment there was fair: A tricky bit of work, done as well as could be expected, but probably wrong.
I am, therefore, astonished that you now link Webb to "the worst kind of science hype". That's really unfair and way over-the-top. I see Webb as doing good science. He has tackled an important question, involving very challenging technical issues, at genuine professional risk, and understands that his results may very well be either null or just wrong, upon further cross-checks.
BTW, just to be clear, I don't know Webb and am in no way affiliated with his team.
@ Alan L.
So does your claim that "extraordinary" adds nothing have any evidence?
And, should this evidence prove to be unconvincing, would it not be clear that the mere existence of some kind of evidence is not necessarily sufficient? That different situations require different degrees of evidence, and therefore adjectives applied to the term "evidence" or "claims" are not ipso-facto useless?
Or are you obviating the need for adjectives by defining "evidence" to mean "sufficient to demonstrate the claim in question" in all cases? Or that any evidence insufficient to demonstrate a claim has a high probability of being correct is not evidence? Because those would both be wrong. Evidence is evidence, sufficient or not. Ergo "extraordinary" evidence is meaningful for establishing evidence required for an extraordinary claim.
Steve in AZ - a couple more excellent examples mysteriously avoided:
"do you understand where your errors and noise come from?" - like trying to measure the temp of the planet and making predictions based on tenths of a degree F.
"Let’s say you’ve got a new piece of data, one that doesn’t line up with the presently best-accepted theory." - like all the AGW computer model being in conflict with the fact that the Earth hasn't warmed over the last 15 years.
Actually when you account for factors that would be expected to result in cooling like a prolonged solar minimum and the Souther Oscillation, you will find that the AGW component of earth's temperature has continued unabated and in very good agreement with the models.
Also, even if it was acceptable to ignore those factors, the statement "hasn't warmed over the last 15 years" is only true if you just take the temperature today and compare to the exceptionally hot year of 1998. It would be like using just the temperature of the year 1997, or 1992, and comparing to today and saying there has been extremely rapid warming. Which scientists don't do, because they understand you can't do that when there's a lot of year-on-year variance in temperature. So you or whoever told you that factoid was guilty of a very basic error.
But of course this demand for scientific rigour is always a one-way street with you folks.
"good agreement" - yeah, except that that AGW component is only 3% of 380ppm of the atmosphere, or less than 12ppm. Which highlights the fact that the errors in the measurements drown the signal. The idea that scientists can even take the "temperature" of the planet to a tenth of a degree is ridiculous. As for the selection of time frames, please explain why the great majority of warming the past century occured in the first half, before the largest part of the AGW component even got started.
And the claims of scientific rigour on the part of environmental scientists is pretty rich considering that NONE of the computer models predicted the 15 year temperature stabilization; quite the opposite. Which would lead someone who believes in scientific rigour to question the theory, or model or both, but not (to use your term) "you folks."
# 1 on the list of criteria for real science is misleading: new theory should agree with all the preceding EXPERIMENTAL data (that has not been debunked), not with preceding theories. Otherwise, theory of relativity would be wrong because it does not agree with that of the world ether.
When the human race is burning the equivalent of an Exxon Valdez every 5 minutes (plus even more coal), and has been doing for over a century, and we know CO2 traps heat, it would be quite amazing if the planet WASN'T warming.
Koch brothers and Berkeley had final say on that one anyway, sorry cowboys.
(You know, actual scientists with actual data.)
Mark A. - You mean like these kind of scientists?
"...the world’s leading climate modelers wrote in the NOAA’s State of the Climate report in 2008 that 15 years or more without warming would indicate a discrepancy between the models and measured reality."
From the Australian Sun: "Britain’s Met Office has revised down its forecast for a global temperature rise, predicting no further increase to 2017, which would extend the pause to 21 years.”
"The Hadley Centre/CRU records show no warming for 18 years (v.3) or 19 years (v.4), and the RSS satellite dataset shows no warming for 23 years."
They brainlessly cite blogs and newspaper articles for their Denialist claims instead of actual scientific papers on a blog post warning against doing such things and I wonder if that counts as irony or not.
William George - are you claiming that the 2008 State of the Climate report did not include the 15 year indication? Or perhaps that the British Met didn't revise down its forecast? Or that the Hadley Centre/CRU records actually show warming?
Ignoring inconvenient facts and calling "Denialist(s)" brainless is not an argument. Ironic, isn't it?
Not having read those papers, I can't say.
But having dealt with a number denialist dirty tricks, I'm quite familiar with practice of taking a graph showing a long term warming trend and only using the small segment of it that looks like the opposite happens. Sort of like how your gut size gets bigger over the years, but between September and December you managed to tighten your belt a notch and then claimed you were losing weight.
Then there's the habitual misquotes and mis-interpretation of the papers writers added to one blogger claiming something is true when isn't and the rest relaying that information as if it were and then pooing this all over every site that allows comment..
Like I said, I haven't read any papers you're pointing at. You may one of those rare precious denialists who aren't completely full of shit. In which case I apologize for doubting you and can't wait for the unicorn to inevitably show up and take me for a ride to Narnia.
Wow, a minimal, kinda, sorta concession wrapped in insults. Should I say, "Thank you, Sir! May I have another?" Or perhaps, kiss your ring? Regardless, apology accepted, but I think you'd have better luck talking to Al Gore about unicorns.
And I do share your distaste and distrust of so-called scientists who won't show their data (Michael Mann) or conspire to exclude the voices of those with whom they disagree (Michael Mann, again).
The models correctly predict the continued warming and are in excellent agreement when you account for the factors that are not intended to be "predicted" but rather in-put, like solar cycles. You have to do this.
And you also have to not cherry-pick your favorite year for declaring global warming to have ended. If you look at the longer term trend then it is clear it has continued and there is no "stabilization". But this is only what scientists have done, not you. You instead have cherry picked your favorite year, which is what scientists have not done. Accusations of a lack of rigor on their part are projecting, nothing more.
CB - You're not arguing with me, you're arguing with "…the world’s leading climate modelers wrote in the NOAA’s State of the Climate report in 2008 that 15 years or more without warming would indicate a discrepancy between the models and measured reality.”
I didn't cherry-pick anything - this is a criteria selected by the folks on the global warming side.
You seem to be saying that the "models correctly predict" except for things like solar cycles. Yet, solar cycles have been known and predicted for more than a century. And building a computer model that doesn't incorporate the fusion engine that drives everything in the Solar System seems a bit lacking, don't you think?
One wonders why you quote from something that mentions a report instead of the actual report. It's like you're trying to be less convincing.
I'm going to assume the quotes and talking points you've mentioned are from or inspired by the Financial Post opinion piece from a couple of months ago.
It and it's talking points are dealt with here
http://www.skepticalscience.com/latest-pre-bunked-denialist-letter-in-l…
Not that you've personally shown to be interested in learning about this.
Here are some personal observations from northern Sweden.
-The permafrost is f*cking thawing out, meaning forest colonise ground where the lichen eaten by reindeer used to grow.
-The winters here are distinctly warmer than in the 1970s/1980s, which is what you can expect, since arctic/sub-arctic regions are where small changes will manifest big effects.
-More extreme weather events.
.
But who knows, maybe I am a shill for The Big Conspiracy. Or maybe I have an evil alien brain parasite, controlling my typing telepathically. Or maybe The Illuminati got to me...
I forgot to add, the small glaciers that remain here are shrinking fast.
And even the Swedish conservaitves admit there is a problem, and is taking warming seriously. (But who knows, maybe they are all commie agent provocateurs)
Birger - I'll see you one Sweden and raise you a Germany.
http://www.powerlineblog.com/archives/2013/02/germans-are-catching-on-t…
surething - From your link:
"The main assertion, made several times in the letter, is a repeat of the myth that global warming stopped 16 years ago. This is one of the few arguments in the letter that has a shred of supporting evidence, as the contrarians claim:
"The U.K. Met Office recently released data showing that there has been no statistically significant global warming for almost 16 years."
This assertion quite clearly refers to the misleading Daily Mail article which was pre-bunked by Nuccitelli et al. (2012). While the measured global surface warming over the past 16 years is not statistically significant ...."
First it's "the main assertion", repeated several times, that's a "myth". Then it's one of the "few" that has a shred of evidence. Then they inplicitly acknowledge the accuracy and truth of the statement by repeating it almost exactly.
Wonder all you like, assumpton wrong, came from here - http://wattsupwiththat.com/2013/02/22/ipcc-railroad-engineer-pachauri-a…
I hate getting in a rush - implicitly and assumption
Birger - and from the land of my ancestors, Tromso, Norway (no swede jokes, I promise), data that indicates a drop of more than one degree C in both average and minimum temperature over the last decade.
http://www.tutiempo.net/en/Climate/TROMSO/10260.htm
Well just about everything that's wrong comes from wuwt at one point or another. Plus it has all be the same reworded article for years now.
It's not surprising that using sources like it, you still don't understand weather, climate or statistics. Either you stopped reading after the part you just quoted, didn't understand what you read, or are in fact dishonest. Warming stopping and short term trends not falling in a certain statistical range are entirely different things even though that ignores the actual warming of the planet instead of just the land surface.
From that same Met Office:
"Over the last 140 years global surface temperatures have risen by about 0.8ºC. However, within this record there have been several periods lasting a decade or more during which temperatures have risen very slowly or cooled. The current period of reduced warming is not unprecedented and 15 year long periods are not unusual."
You are going down the up escalator. Cherry picked by professional liars.
http://www.skepticalscience.com/pics/SkepticsvRealistsv3.gif
Weather variations like this are all quite easy to explain:
http://youtu.be/W705cOtOHJ4
(CNN) -- "Two decades of satellite readings back up what dramatic pictures have suggested in recent years: The mile-thick ice sheets that cover Greenland and most of Antarctica are melting at a faster rate in a warming world... The research released Thursday was backed by the European Union, NASA, the National Science Foundation and research councils in Britain and the Netherlands, with the findings published in this week's edition of the peer-reviewed journal Science." Nov 29, 2012
(BBC) "The dramatic recent decline in Arctic sea-ice cover is illustrated in new data from Europe's Cryosat mission. The spacecraft, which uses radar to estimate the thickness of marine floes, has observed a deep reduction in the volume of ice during autumn months. For the years 2010-2012, this is down a third compared with data for 2003-2008." Feb 13, 2013
(SFC) - "Broken down by political party, 83 percent of Democrats and 70 percent of Republicans say the world is getting warmer. And 77 percent of independents say temperatures are rising. Among scientists who write about the issue in peer-reviewed literature, the belief in global warming is about 97 percent." Dec 15, 2012
(abc)-"It May Be Too Late to Stop Global Warming... about the only thing that could stop it would be a complete economic collapse in China and the rest of the world's developing countries." Oct 26, 2012
So what to worry
As far as climate change goes, I see no harm in preparing for the worst and hoping for the best. Its only the very existence of human life that's on the line. But hey, god forbid you have to give up your Hummer, and actually have to sort the recyclables from regular trash. Fuck the future generations, right? So long as I'm not inconvenienced in any way/shape/form. If you don't think global warning is real then you must not believe in chemistry, physics and the planet Venus, so just wait a little longer because once it drops off William George that unicorn will be there to pick you up shortly.
The neo-con and libertarian are all for the rhetoric of "personal responsibility" and "clean up your own messes", but when it comes down to them taking responsibility and cleaning up, they scream like a spoiled child and DEMAND THEIR SWEETIES NOW!!!!!
Whut?
Michael Mann did share his data. You're getting your denier memes mixed up. You were programmed to say "CRU" there, though this again would be absolutely incorrect: they shared data all you had to do was show that this was for research. The CRU's work is paid out of MY taxes, thank you very much, so I don't want more work produced unnecessarily, requiring more staff and therefore more cost to me, the UK taxpayer.
Again, you're talking bollocks. Peer review means if your peers don't agree your work is scientifically sound, it doesn't get through review.
WRONG!
The trend over the last 16 years has been +0.14C.
What's extraordinary about it?
If it weren't for H2O and CO2, the earth would be 33C colder and have a temperature profile like that of the Moon, in the same orbit as us but without any greenhouse gasses.
We've increased CO2 40%. We've burned enough hydrocarbon to increase CO2 by over 80%. What's extraordinary about the CO2 we produced doing the same thing as the CO2 anything else produced?
Wow.
IcePilot, you are one stupid sack of shit.
IcePilot, of course solar cycles are included in the models, but the model they are using from stellar scientists is not perfect, and e.g. did not predict the extended solar minimum during the last decade. We don't know why that happened. But we can input what did happen into the models and see if it agrees better once you account for that. It does.
Whether predicted or not, you must account for the solar cycle before saying anything like "there's been no warming". If you have a factor that should cause cooling -- like a prolonged solar minimum -- then you have to account for that effect. What you're doing is like saying that if you have a pot of tap water on the burner, and you're dropping ice cubes in the water, if the temperature stays constant then the burner must be off. Which would be impossible if you were also dropping ice in the pot. So the ice must not exist either.
But I don't want to harsh your buzz from having someone say that one of your talking points "has a shred of evidence". You were so happy you didn't even get to the part where they pointed out it's just another case of cherry picking!
Not My Name,
Wow. That was some impressive argument you just made. Name calling. Ad Hominem is still a logical fallacy even when you do think you are right, and generally is considered an indication you are wrong or have no proper argument. Please be vulgar somewhere else.