Eliminate peer-review of baseline grants entirely?

I think it would be an excellent idea.

I've suggested it to my colleagues, however, and the reception has been less than enthusiastic -- among those who are receiving large grants. It gets a favorable response among those from small schools, folks who apply for small grants, early career scientists, and the like.

I don't envision a permanent gift, so to speak. If an investigator signs up for the grants, then they should also, in reasonable time frame, show that they've done something constructive with the previous funding. Not as a matter that takes major peer-review, but more on the level of did they publish in the professional literature.

And like the Canadian suggestion, I envision it as a small grant -- sufficient to cover travel and publication costs but not putting up buildings on campus, etc.. As such, it won't suffice in my field for, e.g., tenure at a research institution (average annual grants for that group being $500,000/year as of a few years ago).

I participated in the NSF panel reviews a couple of times. The costs of the review process in the US could be markedly higher than suggested in Canada. In addition to the time spent by the scientist preparing the grant proposal, there were the (average) 3 lengthy reviews per proposal, and then the 5 of us on the panel doing our own reviews of all (call it 50) proposals, then NSF flying everybody to the panel meeting and the solid day we spent in discussion, plus the NSF program manager and staff handling of the proposals. And this for proposals where the largest single annual amount was no more than $500k, and possibly only $300k. And, as about 75% of proposals were not funded, most of those expenses were ... expensive.

That was more than a decade ago, and already it was clear that there was a move to 'safe', or as you suggest, band-waggony, proposals. That trend can't have stopped.

You can see what will happen if that's implemented, though.

I am glad to see that Larry thinks this is a good idea.

I have been talking to a variety of funders about this recently. A few definitely think that if a granting program is costing more to run than it would cost to give every applicant funds, then something is wrong. (I discussed this a bit in this post ). However the issue will be if this were announced then more people would apply for such a program and you might not have enough money anymore. So it depends on the program how well this could work. The other issue (which was raised by Steve Salzberg in response to my post I ref. above) is that peer review keeps people in line in their projects and thus funding everybody to do whatever they want may not be a great idea.

I think we could streamline many review panels and save money for more science while at the same time having some peer review.

I am with Jonathan on this. If it is given with no review at all, people will turn in all sorts of crappu proposals (or why even write proposals, just stick your hand out for a handout). But the review could be streamlined greatly and save a lot of money. How about public, digg-like, review on a website? Where the author can defend the proposal in public, take in criticisms, improve the proposal in real time, etc. It will (like digg) have problems, but having something out in the open may be a thought to ponder.

agreed. I would be afraid of funding crapola. Tho it is something to think about.....

I'd invite each of you to request a copy of:

Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer review system exceeds the cost of giving every qualified researcher a baseline grant. Accountability in Research: Policies and Quality Assurance 16(1), 1-28.

and then continue the discussion.

For a reprint contact me at: gordonr@cc.umanitoba.ca

I started the whole thing by sending the article to Bora (I got it from a historian friend of mine from U of Alberta). I would urge everyone to actually read the article. There are many other very good arguments against peer review specifically at the level of smallish baseline discovery grants besides the cost. It is a very nuanced argument and the proposals are nuanced too (not all or nothing).

I want to address something I have seen in the comments here (as well as at Lawrence Moranâs Sandwalk blog) that I think is highly pernicious. I find it really sad that scientists themselves would interiorize the highly insulting view that were we not forced to write grants we would just slip into crappy, useless research. This is precisely the attitude implied by the granting agencies: if we donât slap ridiculous amount of self-justification and scrutiny onto you, you will take the money and run to the Bahamas. I think it is highly damaging to science to consider its practitioners as extrinsically motivated (some sort of Skinnerâs pigeons who perform only when given reinforcement).

I would say that science types are by and large a self-selected bunch who obviously are not motivated by mere monetary rewards (if they were they would have turned their supposedly powerful brains to more lucrative endeavors such as business, law, etc.). Second, they pass through a long and arduous apprenticeship that culminates in a PhD. By then they should have soaked up the scientific ethos thoroughly. In addition, throughout our career trajectories we are scrutinized most carefully -- when we get the job, when we are promoted, when we publish, etc. The article points out, and I think quite rightly, that scientists are judged more rigorously at all these points and that the one area where they should be trusted is precisely when they are embarking on new research.

Somewhere along the line you have to allow for what Michael Polanyi called the unaccountable element in science. A system that doesnât allow for that element is stiffling science as a âspontaneous coordination of Independent Initiativesâ (see this). There will be plenty of âquality controlâ later on in the process (publishing, etc.). We as scholars/researchers/scientists are intrinsically motivated to search for truth, or the best approximation to it, with all we have and in the best way we could conceive of. We are the types who would do it even if we had just a pen and a piece of paper. We are like Picasso who supposedly said that if he were put in a solitary cell with nothing to draw with he would paint by spitting on the walls.
Now this is what Rosie Redfield said on Sandwalk:

I think that's a terrible idea. Everyone gripes about preparing grant proposals, but that's the one time we're forced to think rigorously about our research BEFORE we do it. If I didn't have to justify what I want to do in order to get the funding, I'd waste a lot of the taxpayers' money doing what I would only later realize to be useless experiments.

Oh our masters (you who hold the purse-strings) force us to think rigorously because if not FORCED we wonât do it. This is very PERNICIOUS for science. Instead of sending this message, which is just parroting what politicians are telling us Joe sixpack is thinking about science (and academics) anyway, we should be sending the message that we are to be TRUSTED to determine the direction of our own research entirely on our own. We have to defend our judgement and our intrinsic motivation. We are devotees of truth and need no FORCE (or behaviorist manipulation) to be doing our very best to seek for it.

See also this post and this.

I think this is a great idea, though it's doubtful the economic logic would also apply for the United States (but it would be interesting to see a similar financial breakdown). I completely disagree that the lack of peer review for small grants would result in a dilution of good science. Those studies that didn't produce results would simply stop after this initial funding and, in Canada at least, there would be no financial loss to the federal government. A lot of grants today are denied, not because they are shoddy proposals (though that is sometimes the case too) but because they are asking questions that may not have a lot of prior research to back up what they want to do. Through this kind of approach, preliminary research that seeks to push the boundaries could get a foothold and the next really innovative studies might get their start in this way.

A comment I saw elsewhere was, "But what can you do with $30K?" (average NSERC grant, according to the Gordon and Poulin article). I think that the grants are small compared to what many American researchers aspire to is important.

A letter in Science last year pointed out that the U.S. funding system actively selects for expensive projects, not productive projects, largely due to institutions getting overhead. (Unless things have changed at NSERC, every penny goes to the researchers, making the total grant costs lower.)

I've argued for a streamlined system for very small grants. Micro-loans (cf. Grameen Bank) have been very successful at developing economic wealth; micro-grants could be very successful at developing scientific wealth.

For many, many kinds of science, it doesn't take much money to grease a lot of wheels. While not reviewing such grants at all makes me uncomfortable, there could be much to be gained from small grants with high probability of funding that go through an extremely streamlined review process. "Have you published X papers in the last X years? Did the money from your previous micro-grant get spent on bona fide research supplies? Here's a small pot of money, go do science."

As I remember, there were something call " Small Grant " administered by either NSF-USA or NIH-USA long time ago. The review procedures were simple. And there were small grants from home-universities to encourage new faculty to do research. In my days, i.e. 40 years ago, 30,000 was big big. Nowadays 30,000 is not much but good enough to start. A good researcher can be productive with that amount.

This discussion continues here