Worthless grant review comments

We've all had that R21 or R03 come back with completely useless comments. Months and months of work, hours or weeks spent in the lab collecting that preliminary data (which is supposedly unnecessary for those R21s). More time spent waiting and waiting. Revisions. Resubmissions. The same useless comments back to you.

Come on. We all know it's a racket. In a tight funding climate, nobody in charge of the purse strings wants to fund a competitor. But they gotta find a way to reject your grant in a way that is completely noncommittal. Hence, weasel words.

Here's my favorite: "The proposed studies are not unique". With this simple, vague statement, any hopes of funding a decent or important project are quashed. Think that study will fill in a crucial hole in the literature? Screw you. Your project isn't unique enough.

"Unique" or "innovative", or other similar words have pretty much whatever meaning the reviewer wants them to have. Or needs them to have. But what does uniqueness matter, really? Some of the most informative developmental neurobiology work, for example, still relies heavily on chick embryo limb bud removal; a technique developed about a century ago, requiring little more than a tungsten needle and a microscope. Apply some simple histochemical procedures (which date back even farther) and a few molecular biology techniques (which are about as ubiquitous as you can get), and you can potentially rewrite our understanding of the developing nervous system. Yet on the surface, standard fare. Sorry bud, your project just got pigeon-holed. I've seen some great grants go down in flames this way, grants that were either conceptual genius or exceedingly relevant to a health-related issue. The most egregious example I saw was an R01 dealing with the potential for soy phytoestrogens (as an unregulated dietary supplement) to affect behavior and pathology in a model of aging and dementia. This grant-- beautifully designed to address a number of questions relevant to the health of postmenopausal women-- scored right at the payline on the first review, just missing funding, but then subsequently triaged on both resubmissions. The payline shifted, the grant got rerouted to another, much more competitive study section, and suddenly the grant was "not innovative".

The best part? You revise and resubmit according to the reviewer's useless comments, and you get the exact same comments back again.

So what's your favorite useless grant criticism?

Tags

More like this

It looks as though I've been tagged by Drug Monkey, who apparently thinks that I might have something worth saying about the state of the NIH and its peer review system, about which the NIH is presently soliciting comments, as pointed out to me by Medical Writing, Editing, & Grantsmanship. Why…
Over at DrugMonkey, ScienceBlogling PhysioProf comments on the lower funding rates for R03 (Small Grants) and R21 (Exploratory/Developmental Research Grants) NIH grants: What the fuck is the deal with using the traditional study section peer review mechanism for piddly ass little chunks of change…
Back at the beginning of the month, I boldly announced my intentions to finish all the reviewer comments on a revise-and-resubmit paper. In that post, I calculated that with ~25 comments to address " if I just average one a day, it should be easily manageable." Now look over at the left hand…
One of the favorite targets of pseudoscientists is the peer review system. After all, it's the system through which scientists submit their manuscripts describing their scientific findings or their grant proposals to their peers for an evaluation to determine whether they are scientifically…

"The experiments are descriptive". I once took part in a PPG application that would have determined whether the sex chromosomal complement or the gonadal hormones contributed to a sex difference in hypertension. The grant was highly mechanistic to us, but it got the "the aims are descriptive in nature" kiss of death. What exactly does this criticism mean? Since when is descriptive science useless science??

the grant got rerouted to another, much more competitive study section

How did this happen?

From the frau's R01 renewal
Reviewer 1

Insights gleaned from these analyses will be important for understanding NOUN development, should inform the understanding of GENE2 gene function in other contexts as well. Overall significance is excellent.

Reviewer 2

That being said, however, the impact of this proposal is not especially high. How much do we care whether GENE1 acts upstream of GENE2 genes, as well as downstream? Analysis of the differential function of specific GENE1 genes is useful information, but not groundbreaking.

The only quibble here is that the prevailing paradigm is that the two genes are co-factors. If as the frau's data shows, GENE1 is actually a master regulator that controls GENE2, then this work is not only groundbreaking, it will change the way we look at development biology. If I left the magic letters in there, the uproar from the audience would be deafening. The researchers in GENE2 are notable and numerous; however, they cannot bear with the thought that their particular gene is controlled by another gene.
Despite Reader #1's enthusiastic response about this renewal, this did not make it past the second reviewer. Unfortunately, the politically connected and sophists control the direction of science.

By Onkel Bob (not verified) on 23 Jul 2008 #permalink

1. Project is too narrow.
2. Project is too broad.
Surprisingly, you can hear both of these comments for THE SAME GRANT.

Also:

Reviewer #1: Techniques are highly innovative.
Reviewer #2: Techniques are not innovative.
Same grant, same review, same round.

It occurs to me that this kind of disagreement is similar to what happened with Sizzle: different scientists have completely different takes on the same science. Except I'm guessing your grant was probably much better than Sizzle. ;)

Seriously. . . I know it's very, very frustrating. Sorry. :(

Is it common knowledge what this "Sizzle" thing is? Can someone please clue me in?

Is it common knowledge what this "Sizzle" thing is? Can someone please clue me in?

The first rule of Sizzle is: there is no Sizzle.

By cashmoney (not verified) on 23 Jul 2008 #permalink

" The PI is studying a highly significant human pathogen, ****, and has exciting preliminary observations that suggest a novel vaccine strategy. This proposal will test the *** as vaccines in order to further describe the system. No mechanistic studies are planned as these are part of another application. The identification of the **** is high risk but high impact. Taken together, there is enthusiasm for the system and the investigators, but it is dampened by the likelihood that when it is all said and done, we will not have any better understanding of how this works or does not."

A quote from my recent A2 review. Maybe best described as sizzle to fizzle in under a single paragraph?

Maybe this is a stupid question, but is there an online resource that explains the entire grant review process (NIH, NSF, whatever)? I'm going into grad school and while I've helped two PIs with writing and revising their grants I don't really know what the process is like outside reading a few posts (like this and those on DrugMonkey). I figure learning about the process early will hopefully give me an edge over other students once in grad school.

Maybe this is a stupid question, but is there an online resource that explains the entire grant review process (NIH, NSF, whatever)

The NIH review - study section has been described to me as a clusterf*ck ala your worst nightmare's grading session. Usually a study section is assigned a pile of grants and asked to evaluate them. Two or three reviewers take a set, another two or three take another set, and so on. Depending on the study section anywhere from 4 to 15 reviewers participate. The selected reviewers study the ones they are assigned intently and draw a conclusion and skim the rest. They then all get together and "discuss." If a reviewer really likes a application and is charismatic, that person can get the rest to score the grant on the high side. The contrary is also true if the person dislikes the grant (or doesn't understand it) and is persuasive, that person can sink the grant.
This all occurs over a couple of days. (The frau sat on study section for 4 days) The other wild card is the grant officer - that person can shepherd - push - guide a grant to a favorable conclusion by placing it in a friendly study section or one where they have some influence. Again, the reverse is also true, a grant officer that is a frustrated and jealous scientist (not uncommon) can be a real hinderance to getting funded.
To paraphrase the Borg: "Existence is futile, your grant will be triaged."

By Onkel Bob (not verified) on 23 Jul 2008 #permalink

Steve,

NIAID has an excellent grants tutorial page:
http://www.niaid.nih.gov/ncn/grants/default.htm

You may find the annotated R01 (an actual grant that fared very well in review with detailed comments to explain what was good about it) and the mock study section video especially useful.