More about NIH grant reviews than you want to know

Today I got several emails, each asking for my views on a proposed change to the format for National Institutes of Health grant proposals. This may seem of only parochial interest except to those of us who make our living applying for NIH grants, but how health research is funded is of interest to the public in general so here's some background.

The NIH grant review process is one of the most admired in the world and a strong argument can be made it is responsible for the high quality of NIH supported science. I'll deal here with only the most important of the grant mechanisms at NIH, the so-called R01s (pronounced R -Oh - one), or Research Project Grant. These can either be investigator initiated or in response to a particular program announcement asking for proposals in a specific area of research. In either case the scientist writes a proposal according to a rigid format and sends it to NIH according to be considered on a fixed calendar cycle. Deadlines are currently February 1, June 1, and October 1. Many a proposal has been sped at midnight by car to the airport mail or Fed Ex facility to get postmarked by the deadline date. I know because I've done it myself, on more than one occasion. NIH is now moving to electronic submission, and so far it has been a rough ride. I'm sure they'll get it sorted out. Someday.

Anyway, once the proposal gets to NIH someone triages it to an appropriate NIH Initial Review Group (colloquially called the Study Section). [NB: I'm leaving out the part where someone checks to make sure all the page limits are adhered to, the correct margins aren't exceeded and you have used the right fonts and font sizes. You don't want to know about that.] Study Sections are panels of volunteer scientists who sit for periods of three to six years and meet several times a year at NIH for two days at a time to review the proposals assigned to them. I've been on quite a few grant review panels and chaired some of them. It is hard work. Very hard work. It is also one of the best ways to learn how to write a good grant and I recommend accepting an invitation to Study Section to anyone who submits NIH grants as part of their job.

Members are assigned to be primary, secondary or third reviewer on several grants. Prior to the meeting, the primary and secondary reviewers prepare detailed, multipage reports going over the proposal. At the Study Section meeting the primary and secondary reviewers summarize their reviews and each gives a "priority score," essentially a grade that goes from 100 to 500, with the lower numbers being the best. This is followed by discussion among all members. The primary and secondary reviewers then restate their priority scores, which are often changed after hearing arguments from colleagues. Then all members write down their own priority scores on a ballot, without announcing it aloud. The scores are then averaged and sent back to the proposing scientist along with a revised summary of the written reviews. These scored summaries are usually referred to as "pink sheets" although they aren't pink. I assume they were at some time in the distant past. Receipt of the information on the pink sheet doesn't tell whether you have been funded or not, but usually you know from the score what your chances are.

The scientific review is not the final word. The assembled grant proposal scores also go to a higher Council at each Institute where priorities may be changed for programmatic (or sometimes political) reasons. By and large, however, grant funding rises or falls on two things: your priority score relative to other proposals and the available money. In tight years, like this one, less than 10% of grants may be funded in some institutes.

So that's the system. It's like what Winston Churchill said about democracy. A lousy system but better than all the others. But its imperfections are still serious. For one thing, the psychology of the process puts a premium on fault finding. The convention is to write a very detailed report of several single spaced pages. "This is a great grant" doesn't quite work for this. You have to have things to say. Those things are predominantly criticisms. Grant reviewers are like dogs. They pee on things to mark their territories. The result is that "out of the box" thinking or more daring ideas are penalized. The safe project proposed by an investigator with a track record is rewarded. This means a lot of pretty good science gets done. It also means that some innovative science and nw ideas have a hard time.

If there are fifteen or twenty applications, most members only read their assigned proposals with any care and often they don't read the others at all, getting a feel for them from the discussion and answers to questions they put to the primary and secondary reviewers. This also means that a scientist can be done in by an incompetent, malevolent or just plain "out to lunch" reviewer they got assigned more or less by chance because the primary and secondary reviewers have tremendous leverage. the result is a bit like college admissions. Lots of good applicants with not enough room for all of them. Not exactly a crap shoot, but lots of room for chance to play a part.

As I've said, reviewing a grant is arduous and takes many hours, even before study section meets. So it's no wonder NIH is having a hard time getting reviewers. Hence the new proposal: to cut down the portion of the grant that actually describes the research from 25 pages to 15 pages. The email referred to at the outset was NIH's attempt to find out how such a change would affect a scientist's ability to present his or her ideas fully and accurately, and how the change would affect the ease and accuracy of review. Notice I say "the portion of the grant that actually describes the research" because they are not proposing to cut down any of the other paperwork, like budgets, resumes, other support pages, compliance statements and the like. My last research grant proposal (not an R01 but a complex multiproject program with many core facilities) ran almost 900 pages. Of that, I think only about 450 were describing the science.

I'm not sure how I feel about this new page limitation as a grant writer. For most of us, it is all about the science, and like many researchers I've often struggled to get all my science into the 25 page limit. Like undergraduates who have developed tricks to make their papers look longer, we used to employ lots of tricks to squeeze an extra couple of sentences into our proposals, like font changes, line spacing and so forth. NIH has cracked down on this in recent years.

Now doing the science in 15 pages seems daunting. But you can get used to anything, even a stone in your shoe, as someone's grandmother once told me. But as a reviewer, I like the idea.

A lot.