Last week, an antivaxer "challenged" me to look over a paper purporting to show that aluminum adjuvants in vaccines cause inflammation of the brain and therefore contribute to autism, a paper that she would be "citing frequently." Being someone who lives by the motto, "be careful what you wish for," I looked it over in detail. Not surprisingly, my conclusion was that the experiments were poorly done using obsolete and not very quantitative methodology and that the results do not support the conclusions made by the authors. I was not alone in this conclusion. Skeptical Raptor was, if anything, even harsher on the paper than I was.
The paper in question came out of the lab of Christopher Shaw and Lucija Tomljenovic in the Department of Ophthalmology at the University of British Columbia. As I note every time I examine a paper by these two warriors for antivaccine pseudoscience, both have a long history of publishing antivaccine “research,” mainly falsely blaming the aluminum adjuvants in vaccines for autism and, well, just about any health problem children have and blaming Gardasil for premature ovarian failure and all manner of woes up to and including death. Shaw was even prominently featured in the rabidly antivaccine movie The Greater Good. Not surprisingly, they’ve had a paper retracted, as well.
Given the authors' history and a paper that I and others found completely consistent with that history of publishing bad science in the service of antivaccine views, you might reasonably ask: Why am I writing about it again? It turns out that I was indeed far too kind the first time around. You see, I didn't look at all the DNA gels and Western blot films closely enough. I confess that sometimes I don't, particularly when the images provided by the journal online are relatively low resolution. Fortunately, however, there are others with a much sharper eye for photos of DNA gels and films of Western blots than I am, and, if what these people are saying is correct, I rather suspect that Shaw and Tomljenovic might well be cruising for their second retracted paper. Before I explain why, it's necessary for me to briefly explain two things for nonscientists not familiar with the methodology used.
In last week's post, I complained that the authors had basically ground up mouse brains and used semiquantitative PCR to measure the level of messenger RNA for each immune cytokine examined. There's no need for me to go into how this method is only roughly quantitative or how there are much better methods available now. I did that last time. What I do need to point out is that, after the PCR reaction is run, the PCR products (DNA fragments amplified by the PCR reaction) are separated by placing them in an agarose gel and running an electrical current through it. This gel electrophoresis works because DNA migrates towards the positive electrode and, once it solidifies, agarose forms a gel that separates the DNA fragments by size. The gel can then be stained with ethidium bromide, whose fluorescence allows visualization of the bands, which can be assessed for size and purity. Photos of the gel can be taken and subjected to densitometry to estimate how much DNA is in each band relative to the other bands.
To measure protein, Western blots work a little differently. Basically isolated cell extracts or protein mixtures are subjected to polyacrylamide gel electrophoresis (PAGE) with a denaturing agent (SDS). Again, like DNA, protein migrates towards the positive electrode, and the gel forms pores that impeded the process, allowing separation by size and charge. The proteins are then transferred to a membrane (the Western blot) and visualized by using primary antibodies to the desired protein, followed by a secondary antibody with some sort of label. In the old days, we often used radioactivity. These days, we mostly use chemiluminescence. Blots are then exposed to film or, more frequently today, to a phosphoimager plate, which provides a much larger linear range for detecting the chemiluminescence than old-fashioned film. Just like DNA gels, the bands can be quantified using densitometry. In both cases, it's very important not to "burn" (overexpose) the film, which pushes the band intensity out of the linear ranger) or to underexpose them (noise can cause problems). It's also important how the lines are drawn around the bands using the densitometry software and how the background is calculated. More modern software can do it fairly automatically, but there is almost always a need to tweak the outlines chosen, which is why I consider it important that whoever is doing the densitometry should be blinded to experimental group, as bias can be introduced in how the bands are traced.
So why did I go through all this? Hang on, I'll get to it. First, however, I like to point out to our antivaccine "friends" that peer review doesn't end when a paper is published. Moreover, social media and the web have made it easier than ever to see what other scientists think of published papers. In particular, there is a website called PubPeer, which represents itself as an "online journal club." More importantly, for our purposes, PubPeer is a site where a lot of geeky scientists with sharp eyes for anomalies in published figures discuss papers and figures that seem, well, not entirely kosher. It turns out that some scientists with sharp eyes have been going over Shaw and Tomljenovic's paper, and guess what? They've been finding stuff. In fact, they've been finding stuff that to me (and them) looks rather...suspicious.
One, for instance, took figure 1C of the paper and adjusted the background and contrast to accentuate differences in tones:
It was immediately noted:
- A clear and deliberate removal of the Male 3 Control TNF result. This isn't an unacknowledged splice, as there is no background pattern from a gel contiguous with either band, left or right.
- Removal of the left half of the Male 1 Control IFN-g. Dubious also about Male 3 Control IFN-g, as the contrast highlight shows boxing around the band.
- What appears like an unacknowledged splice in ACHE blot, between AI Animal 2, Control Animal 3
Comparing this representative blot to the densitometry accompanying it, they score from 5 independent experiments IFN-g fold change from control to AI, relative to actin, as on average 4.5, with an SEM ranging from ~2.7 to 6.5. This seems too good to be true.
Look at the band. It's the second from last band. It looks as though the band has been digitally removed. There is an obvious square there. The edges are clear. Now, this could be a JPEG compression artifact. Indeed, one of the commenters is very insistent about reminding everyone that compression artifacts can look like a square and fool the unwary into thinking that some sort of Photoshopping had occurred.. However, I do agree with another of the PubPeer discussants this is enough of a problem that the journal should demand the original blot.
On this one, I'll give Shaw and Tomljenovic the benefit of the doubt. (Whether they deserve it or not, you can judge for yourself.) That might be a compression artifact. Other problems discovered in the gels are not so easily dismissed. For instance, there definitely appears to be the ol' duplicated and flipped gel bands trick going on in Figure 2A:
Spotting these takes a little bit of skill, but look for distinctive parts of bands and then look to see if they show up elsewhere. It's also necessary to realize that there could be multiple different exposures of the same band, such that the same band can appear more or less intense and mirror-imaged. You have to know what to look for, and I fear that some readers not familiar with looking at blots like these might not see the suspicious similarities, even when pointed out. Still, let's take a look. There are more examples, for instance, these two bands in Figure 4C:
And Figures 4B and 4D, where bubbles on the gels serve as markers:
You can look at the rest of the PubPeer images for yourself and decide if you agree that something fishy is going on here. I've seen enough that I think there is, as is pointed out near the end:
Great to see such rapid progress being made: Band duplications firmly established for gels in Figs. 2 and 4. Perhaps we can add some RT-PCR from Fig. 1 too? In Fig. 2, seek out the band marked above that looks like a sailing boat with mast and forestay. Now look for it in Fig. 1A. And then perhaps check for any other duplications?
Others note that Shaw and Tomljenovic have engaged in a bit of self-plagiarism, too. Figure 1 in the 2017 paper is identical (and I do mean identical, except that the bars in the older paper are blue) to a paper they published in 2014. Basically, they threw a little primary data into one of their crappy review articles trying to blame "environment" (i.e., vaccines) for autism, this one published in 2014 in OA Autism. Don't take my word for it. Both articles are open-access, and you can judge for yourself.
Some comments from PubPeer:
As far as I can see figure 1 is identical in the two papers? But in the 2014 paper hisograns are described as means +/- SEM from three independent experiments and in 2017 as means +/- SEM of five independent experiments? http://www.oapublishinglondon.com/article/1368
And:
Brazen self-plagiarism of the open access 2014 paper’s Fig. 1 is a key find by the human commentator. Especially since it is not in PubMed (though it is Ref. 166 here). This means that they have used certain elements of a single gel four times in three years: Nice work if you can get it.
Here is the direct link to 2014 Fig. 1
http://www.oapublishinglondon.com/images/html_figures/1368_346.png
The licence for the 2014 paper states “Creative Commons Attribution License (CC-BY)”. Unfortunately, the 2017 recycling of Fig. 1 is neither creative nor is it attributed.
What this means is that Elsevier were misled regarding the copyright situation and the originality of the work. So this finding surely gives the 2017 publisher a get out of jail card. If they choose to play it, they can now unilaterally withdraw this embarrassing Anti-vaxxer concoction on these grounds alone.
Don’t forget to archive the two papers for your records: They might disappear from the publishers’ web sites at some point.
And that's still not all. Let's take a visit to our scaly friend, Skeptical Raptor, where he notes that The Mad Virologist and the Blood-Brain Barrier Scientist jointly analyzed the paper and found:
But there are six other key points that limit what conclusions can be drawn from this paper:
- They selected genes based on old literature and ignored newer publications.
- The method for PCR quantification is imprecise and cannot be used as an absolute quantification of expression of the selected genes.
- They used inappropriate statistical tests that are more prone to giving significant results which is possibly why they were selected.
- Their dosing regime for the mice makes assumptions on the development of mice that are not correct.
- They gave the mice far more aluminum sooner than the vaccine schedule exposes children to.
- There are irregularities in both the semi-quantitative RT-PCR and Western blot data that strongly suggests that these images were fabricated. This is probably the most damning thing about the paper. If the data were manipulated and images fabricated, then the paper needs to be retracted and UBC needs to do an investigation into research misconduct by the Shaw lab.
Taken together, we cannot trust Shaw’s work here and if we were the people funding this work, we’d be incredibly ticked off because they just threw away money that could have done some good but was instead wasted frivolously. Maybe there’s a benign explanation for the irregularities that we’ve observed, but until these concerns are addressed this paper cannot be trusted.
I note that they go into even more detail about the problems with the images that have led me (and others) to be suspicious of image manipulation, concluding:
These are some serious concerns that raise the credibility of this study and can only be addressed by providing a full-resolution (300 dpi) of the original blots (X-ray films or the original picture file generated by the gel acquisition camera).
There has been a lot of chatter on PubPeer discussing this paper and many duplicated bands and other irregularities have been identified by the users there. If anyone is unsure of how accurate the results are, we strongly suggest looking at what has been identified on PubPeer as it suggests that the results are not entirely accurate and until the original gels and Western blots have been provided, it looks like the results were manufactured in Photoshop.
I agree. Oh, and I agree with their criticism of the use of statistics. I even brought up their failure to control for multiple comparisons, but Shaw and Tomljenovic also used a test that is appropriate for a normal distribution when their data obviously did not follow a normal distribution.
So, my dear readers, it turns out that Orac, as Insolent as he can be when slapping down bad science by antivaxers, was not nearly Insolent enough in this case. Mea culpa. I should have known better, given Shaw and Tomljenovic's history. Not only do we have poorly done and analyzed experiments, but we also have self-plagiarism and, quite possibly, scientific fraud. Only releasing the full resolution original images from the original experiments (which are now probably four years old) can put these questions to rest.
Science matters. I hate to see it abused like this, particularly when experimental animals are killed in the service of such awful science.
- Log in to post comments
More like this
Well, for all we know now, they may have never touched a mouse, but just copied their gels together from something they found on the web. This looks really bad.
Oh man. I used to check images like these for a journal, and this, uh, looks suspicious. I can't believe no one queried these images before publication.
So not only are the methods and statistical analysis flawed but there might be evidence for outright fraud (sounds like another retracted paper we all know). Yet places like the National Vaccine (Mis)Information Archive are spreading this as definitive evidence.
A. If there was manipulation of the images, the university should respond.
B. Shaw, Meehan - is there any link between ophthalmology and being anti vaccine? I expect that's unfair to most ophthalmologists, but wonder.
Orac writes,
Not only do we have poorly done and analyzed experiments, but we also have self-plagiarism and, quite possibly, scientific fraud.
MJD says,
This is, quite possibly, Orac's best re-review!
In summary, when discovered and placed in the public domain, circumstantial evidence (e.g., questionable DNA gels and Western blot films) of scientistic fraud can damage or end the careers of researchers who, without a doubt, have been shown to engage in it.
It's my understanding that Shaw is a tenured full professor at UBC. It's also my understanding that one of the grounds for revoking tenure is academic fraud.
Just sayin"...
It tends to get pinned on the most junior researcher. (I note that Yongling Li seems to have departed Shaw's group, as well.)
I don't know who Meehan is, but Shaw does not have an MD and is in no respects an ophthalmologist. Anyway, antivaccinosis can strike indiscriminately. For instance: pediatrics (Sears, Mendelsohn), Gastoenterology (Wakefield), Neurosurgery (Blaylock), Pathology (Sin Hang Lee).
One would think that a succession of junior researchers pinned with academic fraud after having collaborated with the same senior researcher would be a great big red flag for any junior researcher who actually wants to have a future in the field.
Given Shaw's research interests, I'm tempted to think that perhaps he was bestowed upon the Ophthalmology and Visual Sciences department by means of a lateral transfer from another department that didn't want to keep him.
When it comes to poor quality images, particularly when compression artifacts are quite apparent, I'm perfectly happy to wish a pox on both houses - the authors for submitting crap or failing to take the publisher to task for wrecking that which was good when submitted, and the publisher for doing the latter or failing to take the author to task for submitting crap. I'da thunk someone in a dep't with "Visual Sciences" in the name might be a little hyper-vigilant regarding such matters.
We seem to have made a transition from the days when "white space" was regarded as sacred and actual useful content profane to an age of just plain incompetence and/or indifference.
While poor images are set in stone (or a polymer coating a sheet of aluminum) for printed publications, there is simply no excuse for allowing them to persist in electronic publications. Decent digital images can make rather large files, but I doubt that many journal articles are delivered via dial-up.
Of course fabulous images don't make up for poor research.
"2.The method for PCR quantification is imprecise and cannot be used as an absolute quantification of expression of the selected genes."
That is the first thing I thought. 20 years ago, there might be a reason for doing gels (heck, 10 years ago I was doing syber green in an academic lab, because probes were too expensive but gels are too imprecise), but these days, qRT-PCR machines have been around for a long time, and probes are absurdly cheap ($1.20 a reaction for a basic Taqman). Most if not all of those genes are off-the-shelf assays.
Not that I'd trust these fooks to do qRT-PCR correctly.
Or run a negative control. If they did it in this paper, they didn't mention it.
Or use actin as a housekeeper, ffs.
And using a paired T-test for so dizam many comparisons.
Yeah, lousy paper even from just skimming it.
I should have said "when compression artifacts are detrimental" instead of "... apparent" - some a just ugly but don't mask important content.
Orac,
Thanks
Just plain thanks for all you do
R
Yongling Li coauthored the 2013 paper with behavioural results for the same mice, but wasn't involved in the 2014 "OA Autism" paper. OTOH, Dan Li was coauthor in 2014 and 2017... also yet another review-shaped self-plagiarism:
https://www.ncbi.nlm.nih.gov/pubmed/25428645
"Are there negative CNS impacts of aluminum adjuvants used in vaccines and immunotherapy?"
[Contains Figure 1 again]
The Acknowledgements could be read as a CYA positioning of the lab staff to take any blame:
It's on Researchgate if anyone wants to quantify the degree of self-plagiarism:
https://www.researchgate.net/publication/268880246_Are_there_negative_C…
That band removed is not a compression artefact. Compression artefacts don't quite look like that. That is a clear case of photoshop. Photoshopping gels bands should lead to immediate retraction. Let's see what the journal editor does.
But what gets me is why bother with all the copying and pasting of bands. If they were going to just make it up, they could go the Judy Mikovits approach and just rename the bands as they wished.
In response to T. Bruce: Dr. Meehan is an Oklahoma doc trained in ophthalmology whose antivax views got taken down here:
https://www.skepticalraptor.com/skepticalraptorblog.php/jim-meehan-anti…
Let's not forget the most prominent ophthalmologist who's encouraged antivax idiocy - Sen. Rand Paul of Kentucky.
I wondered the same thing, Chris. Maybe they did both?
The Photoshop is easy to see (once you start to look for it), but other that trying to replicate the study, how could you prove or disprove something like renaming (or, to call it what it is, flat out lying)?
I'm glad to know I wasn't imagining things when I looked at the bands and thought, "They look like the same sample to me."
Orac is too kind when he calls this science. This isn't science. It's science fiction.
Off topic, so you can delete this when you see it, but I know you will be most pleased with this news from Australia:
"Fake wellness blogger Belle Gibson has been ordered to pay a fine of $410,000 after being found guilty of misleading and deceptive conduct earlier this year.
The Federal Court in Melbourne found she misled her readers when she claimed her brain cancer was cured through alternative therapies and nutrition.
It was later revealed she never had the disease."
http://www.abc.net.au/news/2017-09-28/disgraced-wellness-blogger-belle-…
Well it was all made up, but given how easy detecting photoshopping of gels has become, why risk that. I suppose if you are inventing stuff already, you don't think of these things.
The renaming of lanes is in fact really hard to work out unless you have expertise with those specific genes/proteins. Mikovits' paper didn't pass the smell test when I read it, but it was really hard to put your finger on where the fraud was. A clear indication of a problem was that numbers of individuals in the paper and the supplementary information didn't match, and didn't match with Judy's public statements, but what did that mean?
It was only when Judy decided to use the same photo in a conference presentation with different labels on the lanes that it became obvious the sort of chicanery in operation and that the photo in the paper wasn't what they claimed it was. In the end it turned out that the blot was something else again and had been repurposed twice.
Meehan is an anti-vaccine loon who deserves to lose his medical license for the voluminous amount of dangerous stupid he spews. On the bright side, he overstepped his big mouth last year attacking physicians who promote vaccines and Meehan now has a libel case pending against him brought by a physician he slandered.
Shaw et al made stuff up about their control mice, too. In Table 1 they state that control mice received their saline injections on Days 3, 5, 6, 10, 12, 17 -- same as the Al-exposed group. The trouble is that in Table 2 of the earlier 2013 report of the same experiment, the controls only received five injections, on 2, 4,, 9, 11, 16.
What is a day between friends?
Oh and Dan of Vaccine Papers tells me Shaw is going to retract the paper.
Perhaps it has got too embarrassing.
Elsevier management were already talking to the journal editors about minimising damage to their reputation, so one way or the other, the paper will not be around for long,
Even without the fakery it was a crap paper.
It just doesn't stop. In the original 2013 description:
In the 2017 account:
If you don't know whether your mice were 16 weeks or 34 weeks old when you killed them, it could be time to change careers.
I'd like to think that maybe not as many mice were needlessly killed for Shaw's current paper, but more likely they killed at least as many as stated in the paper before deciding to fake it all up.
@ #28 Chris Preston
One can only hope a withdrawn publication full of falsified data and even self-plagiarising still counts as fraud.
Well, maybe they went the whole hog fraud-wise, and they didn't even use let alone kill mice in the first place, but just made things up over a few beers kale smoothies or so...
Anyway, if this is a fraud (and so far, it sure smells like one), it once again emphasizes the sheer stupidity of antivaccine people, even the ones with a sort of higher education. Did they really think that they would get this kind of clumsy mess past the science crowd unnoticed? And about this particular subject? They may as well put on bullseye pants and hand out shotguns...
Once again a study shows that vaccines improve health:
Contact: CDC Media Relations
(404) 639-3286
New study shows Tdap vaccination during pregnancy can prevent whooping cough in babies Less than half of pregnant women in the United States take advantage of vaccination
A new CDC study published today in Clinical Infectious Diseases reported that vaccination with whooping cough vaccine, Tdap, during the third trimester of pregnancy prevented more than three out of four (78 percent) cases of whooping cough (also known as pertussis) in babies younger than two months. However, only 49 percent of pregnant women who delivered between fall 2015 and spring 2016 received the vaccine. CDC recommends women get Tdap during each pregnancy to provide critical short-term protection to babies when they are most at risk for this life-threatening illness.
The study used data from 2011 through 2014 on babies younger than two months from six states. It found that mothers whose babies had whooping cough were less likely to have received Tdap during pregnancy. The study reported that, in addition to being 78 percent effective at preventing whooping cough, Tdap vaccination during the third trimester was 90 percent effective at preventing serious cases of whooping cough that require hospitalization.
“Women have such a great opportunity to help protect their babies before they enter the world by getting Tdap vaccine while pregnant,” said Nancy Messonnier, M.D., director of CDC’s National Center for Immunization and Respiratory Diseases. “This study highlights how babies can benefit when their mothers get the vaccine and reinforces CDC’s recommendation for women to get Tdap vaccine in the third trimester of each pregnancy.”
Read More Here
Orac is correct. The paper was a fraud.
Of course, there should be an investigation ASAP.
I can't believe anyone actually takes anything written by Shaw and Tomljenovic seriously anymore.
VP@36, That is a laudable admission. Will you now scrutinise Shaw et al.'s other studies and consider the criticism leveled at those?
In April 2016 the 10th Int.Congress on Autoimmunity took place in Leipzig -Germany.
http://autoimmunity.kenes.com/2016#.Wc-yO9FpGM8
Within this event the 4th International Symposium on Vaccines took place- sponsored by the antii-vaccine Dwoskin-Family Foundation and their CMSRI -"research" Institute.
The programm of this symposium can be found here
http://autoimmunity.kenes.com/Documents/4th%20International%20Symposium…
Chaired by C Dwoskin and Chris Exley (The Aluminium "expert" ) they also had a panel discussion with Lucia Tomljenovic and C Exley with the following questions:
"1.Why do you think that the authorities are trying to hide the truth?
2.What is the objective evidence that HPV vaccination is not justified?
3.What do you think, are the mechanisms by which vaccine can induce autoimmunity?"
So all authorities on this globe are hiding the truth- such wording is typical for any conspiracy theory in this case it is the anti-vaccine movement including their fraudulent "publications"
Will you now scrutinise Shaw et al.’s other studies and consider the criticism leveled at those?
In particular, his two 2014 publications -- in OA Autism and Immunotherapy -- which used the same Figure 1 with all its problematic features (such as the statistical absurdity of adding standard-error bars and applying t-tests to samples involving only three measurements). Do you find them convincing? Would you include them in the Vaccine Papers archives? Would you advise Shaw to retract them?
In April 2016 the 10th Int.Congress on Autoimmunity took place in Leipzig -Germany.
And Mark Geier was there, having abandoned his mercury / chelation grift in order to climb aboard the aluminium scamwagon. I would have thought that the other panelists would have felt slightly abashed to be sharing the podium with such an out-and-out charlatan, but evidently one must abandon all sense of shame to batten onto Dwoskin money.
Just received an email:
"The paper by Shaw and co-workers is being retracted jointly by the authors and the editor.
John Dawson, JIB Editor"
Pleased it'll be retracted; frankly gutted it was ever published...
Cheers!
The Orbital Laser Cannon of Pubpeer Correction is firm but fair.
#42 Magdalen -
Thanks.
I 1st saw news of the retraction via RtAVM:
https://www.facebook.com/RtAVM/posts/1715728805163788
who cited a guest post by Smut Clyde at Leonid Schneider's blog:
https://forbetterscience.com/2017/09/29/the-rise-and-fall-of-an-antivax…
.
At the very bottom is this update:
Update 8.10.2017. The Editor-in-Chief John Dawson wrote me back, with this signed one-liner (no “Dear Dr Schneider”, or “Hi Leonid”):
- “The paper by Shaw and co-workers is being retracted jointly by the authors and the editor.
— John Dawson, JIB Editor”"
.
Now the question remains as to what is going to happen to the authors of this scam. I predict UBC will defend their right to publish alternative facts.
Good news on the retraction and here is hoping a certain anti vac tenured pseudoscientist at ubc gets sacked
I wonder how they will explain this particular retraction.
It is gratifying to see that Clare Dwoskin et al. are getting exactly what they paid for. Although it's a shame that benefactors would rather fund sloppy, fraudulent science than bestow their largesse on worthy autism causes.
Per RW: "We immediately did our own analysis. Indeed, some images have been altered. How that happened, we don’t know."
Don't worry, Christopher, you'll "figure it out" in short order. And republish in OMICS or something.
Someone here said the post-doc would be thrown under-the-bus:
http://retractionwatch.com/2017/10/09/journal-retract-paper-called-anti…
Worth the read as Shaw's Pollyanna routine is maddening.
It would be xmas come early if it turned out Dan Li absconded with the original data (that wasn't fraudulent) to CYA for the two studies that were published with fraudulent data by Shaw and Tomljenovic.
I didn't know where to put this.
http://www.cracked.com/article_25057_6-conspiracy-theorists-who-got-own…