The Autism Omnibus: The difference between real scientists and crank scientists

The Autism Omnibus trial continued last week, which was devoted primarily to the government's case. Consequently, there were a variety of real experts, as opposed to the pseudoexperts called by the prosecution last week. With only the occasional hiccup, they are taking serious bites out of the plaintiff's case, as documented on a near daily basis by Autism Diva and Kevin Leitch. Today, I want to focus on Day Eight of the testimony, not so much to beat up on those claiming that vaccines cause autism (although there's plenty of opportunity to apply some much deserved Respectful Insolence⢠to them). Rather, what interested me more about this day's testimony is how the testimony of Stephen Bustin illustrated beautifully (1) how technology can be abused in the service of quackery and (2) the difference between science and quackery. As Arthur Allen put it:

But if nothing else, the testimony by Bustin and other witnesses this week provided an opportunity to reflect on the extent to which the vaccines-cause-autism case is built upon the use--and abuse if you agree with the government's witnesses-- of new tools of biological discovery that have emerged over the past few decades. These tools--PCR, advanced flow cytometry, endoscopy and the Internet, to name four--are wonderful ways to observe small things. They are fabulous tools for science and medicine, and the public, but they are tricky.

Stephen Bustin is a British scientist who is a world expert, if not the world expert, on the polymerase chain reaction (PCR). Back when I was learning how to do quantitative real time PCR (QRT-PCR), I read his review articles on how to do it properly in a manner to get reproducible and accurate results. I also discovered for myself the pitfalls. The only other scientist in the world of PCR that I can think of who has nearly as much stature is Michael Pfaffl, who is a guru when it comes to normalization of measured gene products to standards and who also runs the Gene Quantification website, which is chock full of equations, protocols, and algorithms for doing quantitative PCR. Professor Bustin is regularly invited to speak at international conferences on PCR and is the author of A to Z of Quantitative PCR, which many scientists consider to be the Bible of QRT-PCR.

The reason Bustin's testimony was important is because one major prong of the test case being tried rests on a study that purports to show that measles virus from the MMR vaccine. The reason for this is that the Andrew Wakefield, he of the discredited paper claiming a link between MMR and bowel infection with measles, the concept being that the vaccine triggered a gut infection that contributed to autism, as well as "autistic entercolitis." The plaintiffs' main argument, which was put forth by pediatric neurologist Marcel Kinsbourne a few days earlier was that measles virus from the MMR vaccine infects the gut lining and enters the brain. From there, so the evidence-free hypothesis goes, it leads to dysfunction of astrocytes and other brain cells, which in turn leads to high levels of the neurotransmitter glutatmate, which in turn leads to a state of overstimulation, which produces the symptoms of autism. Key to this hypothesis was finding measles vaccine DNA in the gut. The other major prong is that thimerosal in vaccines causes immune suppression that allows this to happen, but Professor Bustin did not address that highly dubious claim.

Here's where Professor Bustin's expertise in PCR comes in. PCR is an amazing reaction. Basically, it depends on a special property of a DNA polymerase (an enzyme that makes DNA based on the template of its complementary strand) found in an organism that lives in thermal vents, namely Thermus aquaticus, hence the abbreviation of "Taq" polymerase. The reason that Taq polymerase is so insanely useful (and the reason that Kary Mullis was awarded the Nobel Prize in Chemistry for his development of PCR, is that it is able to withstand high temperatures. Indeed, at 95° C, a temperature at which most other enzymes would be reduced to an unfolded (denatured) and no longer functional mess, Taq polymerase keeps chugging along, adding nucleotides to a growing DNA strand. What Mullis realized is that, by cycling the reaction temperature rapidly from 95° to lower temperatures, he could set up conditions so that specific DNA sequences could be amplified.

Basically, two different primers (short stretches of DNA designed to be complementary to and thus bind to a sequence of interest, each of which are on opposite strands "aimed" at each other are mixed with the sample containing the DNA of interest and then run through different temperature steps. First is the denaturation step at 95° C, which guarantees that all the DNA of interest has been separated into its two complementary strands. Next, the temperature is dropped to a lower temperature designed to allow the primers to bind to their complementary sequences (the annealing phase). This temperature can't be too low, or nonspecific binding of primer to other DNA sequences that don't exactly match will occur, leading to amplification of DNA other than the intended sequence. It can't be too high either, or there won't be enough binding. Next, comes the elongation phase, where the temperature is increased to 72° or thereabouts, which is near the optimal temperature for Taq polymerase to do its thing. Finally comes another denaturation phase, where the temperature is briefly boosted to 95° to separate the strands, after which the temperature is dropped again to the annealing temperature and the cycle begins anew. Because the number of copies of the target DNA sequence roughly doubles every cycle, after 30-40 cycles even single DNA copies can be detected. Here's what it looks like in schematic form:

Back in the old days (the early 1990s, when I was doing my graduate work), PCR machines were big and bulky. Now they're little machines that take up very little space on the benchtop. Also, back in the old days, there was no good way to quantify the results. PCR was essentially a yes-no answer. Either you saw the proper-sized band on a gel or you didn't. Now, using special fluorescence techniques, it's possible to quantify how much of each DNA species is there. Finally, it's possible to use reverse transcriptase, which allows RNA to be reverse transcribed into DNA, and thus measure the levels of an RNA of interest. This latter technique is what I've used most in my lab, because now that PCR can be done quantitatively using quantitative real time PCR, where a fluorescence signal is measured at the end of each cycle and allows the amount of PCR product to be measured.

However, the greatest power of PCR also leads to the biggest pitfalls in its application. Basically, PCR is so sensitive that it's very easy for a PCR reaction to amplify a contaminant or for the primers anneal to sequences that are similar but not identical and thus amplify DNA other than that of interest. Indeed, I've encountered all of these pitfalls and more. That is why controls are of the utmost importance. There has to be a negative control to let the researcher know that his PCR conditions are not amplifying junk. Many are the times when I've gotten spurious signals in my negative control (usually water or, in the case of reverse transcriptase PCR, a reaction into which no reverse transcriptase was put, otherwise known as the "no template" control). I've spent days trying to chase down such contaminants and eliminate the spurious signals. Contamination of PCR reactions can be particularly problematic when you are using PCR to study a gene that you've also placed in a plasmid. During plasmid preps, it's very easy to spew your gene's sequence all over the lab. Careful segregation of functions is necessary, and even then that's no guarantee. Finally, if you've designed your primers poorly or if you anneal at too low a temperature, you can get nonspecific binding and amplify sequences that you don't want to. All in all, PCR can be a very tricky game, and it gets trickier the more cycles you use to amplify and the lower amounts of target sequence you are trying to amplify. PCR is an exponential amplification; small variations in early cycles can have enormous effects in later cycle, producing wide variation in results. (This was one of the biggest problems to be overcome before PCR could be made truly quantitative.) Here, Professor Bustin explains:

Q.What is a positive control?

Well, a positive control is an essential control that tells you whether your assay is working, so what you would do is you would take the target thatyou're interested in detecting and put it into a test tube and use your assay to detect it.

If you don't detect it, you know there's a problem with your assay because it's a positive control. If you do detect it, you know your assay is working. If you do this consistently each time, you know how efficient your assay is from day to day.

The positive control is simply something that tells you that your assay is okay.

Q And a negative control?

A A negative control is something very crucial. There you leave out your target, so if you don't detect it then that means that there's no amplification, which is what you want. If you do detect a positive in a negative control then you know there's a problem with your assay because it should not be there, and you always get suspicious of any assay that gives you a positive result in a negative control.

This is all very basic, and anyone who's ever done PCR in a serious way knows about it. This is one huge difference between science and pseudoscience, between legitimate medical research and quackery: Attention to controls and lining all one's ducks up in a line, so to speak. Real scientists do it; cranks can't be bothered do in their quest to find what they want to find. Now, here's where Dr. Bustin was so devastating to the claim that MMR causes autism. Andrew Wakefield and Arthur Krigsman, who claimed to have replicated Wakefield's work and found measles virus RNA in the guts of autistic children, both used Professor John O'Leary's Unigenetics Laboratory in Ireland, and it was the Unigenetics lab whose results led to a paper by Uhlman et al and an as yet unpublished poster presentation by Walker et al, both claiming to find measles virus in the guts of autistic children. As I mentioned before, these results are one of the major linchpins of the entire Cedillo case. Among the problems that Dr. Bustin found looking at the Unigenetics, well, why don't I let Dr. Bustin say, through select quotes:

1. Because I was doing all of the basic looking into the innards of the instrument, I was able to finger print the results and identify which run was done on which instrument, and what I found was that the instrument that was used on most of their runs had a huge variation in the heating and cooling characteristics across the block. What this means is that there is variability of your results, depending on where you place your tube on the instrument. In other words, Unigenetics PCR machines were crappy and didn't produce consistent temperatures.
2. So, obviously, if you have hundreds of millions, or thousands of millions, of bacteria, each containing tens of hundreds of copies of DNA, you've got a massive potential for DNA contamination. So you never want to have any plasmid DNA anywhere near your laboratory where you're doing the PCR. So it struck us as peculiar that they had a room labeled "Plasmid Room" next to the laboratory, and that plasmid room contained a shaker for growing up bugs. I asked Dr. Sheils several times, and she assured me that they did not use that plasmid room for growing up F gene target for their standards. So this may or may not be the source of contamination, but the DNA, again, as Dr. Ward mentioned this morning, is all-pervasive. Once you've got DNA contamination, it persists for years, and it gets in everything. If you're handling bacteria, if you're handling plasmids, it gets into your hair, on your hands, on your clothes, and you will carry it around with you, and that is the problem. In other words, the lab was sloppy and undoubtedly was contaminated with measles nucleotide sequences all over the place.
3. As I think I said one of the things about this paper is that it's fairly unique in my experience, and it's given no information at all about what actually was done. It actually tells you in outline what they did, where they got their samples from and that they prepared RNA, but it gives you no information whatsoever about, for example, the quality of the RNA the quantity of the RNA and how the different RNAs were extracted from different samples which they refer to. In other words, in the paper and the poster, there was no data about how the RNA was handled. In other parts of his testimony, Professor Bustin makes it clear that the quality of the RNA was probably poor. RNA, unlike DNA is very unstable, and it is said among scientists that it will degrade very easily if you just look at it funny.

But perhaps the most devastating part of Professor Bustin's testimony is when he reported the results of his own analysis. In essence, he showed that the results reported in the Unigenetics lab were almost certainly the result of contamination. He showed this most convincingly when he described experiments that he noted in which the reverse transcriptase (RT) was left out. Remember, RT is what makes DNA from an RNA template, and PCR amplifies DNA. Because of this, any signal that you get from a no-RT control can't be from your RNA. It has to be from contaminating DNA. Concludes Bustin:

A.Okay. Reverse transcription. We have been talking about real-time PCR all this time, but I think it is important after this point to say that the real time PCR really only refers to the DNA amplification side of things. In order to be able to amplify an RNA molecule you have to convert the RNA to DNA. I think that's been said several times over.

Now, this is crucial here because measles virus does not exist as a DNA molecule in nature, so you must identify the RNA. If you ever identify DNA then it has to be a contaminant. This is a very crucial point. Because measles virus does not exist as a DNA molecule you can't detect DNA. If you do it's a contaminant.

Q So you have to use the RT step to amplify?

A So you must use an RT step to detect the measles virus RNA. If you detect a target that is apparently measles virus in the absence of an RT step by definition it can't be measles virus because it has to be DNA. It's a very simple concept. At least it is to me. It's not to everyone else.

Using this principle and other observations, Bustin demonstrated that it was indeed DNA contamination that O'Leary and Uhlman were reporting, not a real signal from the RNA of the measles virus. He concludes:

So I could speculate all day, and I really don't want to speculate. It doesn't actually matter.

The fact is that I'm showing that they are getting DNA contamination. Where it comes from is another matter. What matters is we're getting DNA contamination, and, by definition, therefore, we're not detecting measles virus.

PCR is a very powerful technology. Properly used, it can detect as little as one copy of a DNA sequence of interest. Its very power, however, makes it easy to detect things that couldn't be detected before. It also, alas, makes it easy to detect things that aren't really there if the scientist using the technique is not aware of its vagaries and does not know how easily contamination, even minute contamination, can mess the assay up.. Such mistakes are fairly common in investigators new to PCR, and this was particularly true 15 years ago. However, PCR has been around a long time now. Few are the labs that don't have at least a benchtop PCR machine, and, as the price of this newer PCR technology falls, more and more labs have quantiative real time PCR machines in their labs, and almost all labs at least have access to a shared machine. What I'm saying is that scientists interested in subjects that might require PCR who trained any time in the last decade or so were almost certainly taught how to do PCR correctly. These days the only reason for screwing it up so royally without even being aware that the PCR might be screwed up is sheer carelessness. In the case of Unigenetics, that carelessness provoked an MMR scare, leading to decreased vaccination rates in the U.K. and the first deaths of children from measles in a long time. Such is the price of misuse (or careless use of) this technology.

If the evidence presented in the Autism Omnibus doesn't put the final stake in the heart of the misbegotten "MMR causes autism" pseudoscience, truly there is no hope for reason and science.