Changes to NIH Peer Review

NIH has agreed to some suggestions from advisory panels about how to change the grant peer review process:

One year ago, NIH Director Elias Zerhouni asked external and internal advisory panels for advice on how to cope with a record number of applications, a flat NIH budget, and a shortage of quality reviewers. The two panels issued recommendations this winter (Science, 29 February, p. 1169). NIH's response was presented today to the Advisory Committee to the Director by National Institute of Dental and Craniofacial Research Director Lawrence Tabak.

NIH agreed with the panels on the need to shorten the application--by more than half, from 25 pages to 12--and to emphasize the anticipated impact of the research over methods and other details. Applicants will also be given more explicit feedback on their proposals. To attract more reviewers, NIH will allow them to serve over 6 years rather than 4 years, test out online reviews to reduce travel (review committees currently meet in person), and give those who attend at least 18 meetings a grant supplement of up to $250,000, about as much as 1 year of an average grant. NIH also plans to have reviewers segregate the applications of young investigators from the rest of the pool and assign a different NIH-wide cutoff point for funding them so that at least 1500 a year are funded.

However, NIH officials nixed the panels' recommendation to jettison a system in which unsuccessful applicants can resubmit their proposal two more times. Reviewers tend to favor these amended applications (A1s and A2s) over first-time awards (A0s) because the applicant responded to reviewers' comments or out of sympathy, the advisory committee found. Since the doubling of NIH's budget ended in 2003, the percentage of first-time applications funded has shrunk from 60% of the total pool to about 30%.

To level the playing field, the advisory committee recommended that all proposals be considered "new." The committee also urged that the weakest proposals be marked "not recommended for resubmission." Its goal was fewer resubmissions and a lighter workload on reviewers.

I think that the bit about shortening grants and paying people to be regular attendants at study section (the meeting of peer reviewers where the assign scores to each of the grants) will certainly improve the caliber of reviewers. It would also be a great idea if you could do these reviews from afar rather than heading down to Washington. (Do they do them in places besides Washington? I don't actually know.)

On the other hand, I do understand the resistance to labeling all grants as "new." It really depends on why you think most grants aren't funded the first time. If you think that is because they weren't very good to begin with, you should be in favor of labeling all grants as new so as to more rapidly identify the best grants and get rid of the rest. On the other hand, if you think that a lot of grants aren't funded the first go because they A) weren't written well but have fundamentally good ideas or B) because funding is so limited, you need that "resubmitted" label to help your chances for the second or third go-around.

They are attempting to balance the benefits of identifying great grants on the first go with the benefits of making sure everyone who deserves funding eventually gets it. That is a tough line to walk.

Thoughts?