Biochemistry and Molecular Biology

August 2007

     New investigators and the co-PI trap – Before addressing this month's main topic, I'd like to warn young investigators without an R01 about the potential danger in accepting a formal role as Co-PI on a more senior investigator's R01, or as Project Leader on a Program Project grant. Either would have the advantage of giving you considerable credit for the grant's research income, and presumably increase your perceived expertise for your own future applications, but would also deny those applications the substantial benefits of "New Investigator" status. For most Institutes these include a 2-5 percentile point break on the payline, and being subject to little if any cut from the funding level/project length recommended by your Study Section. Some Institutes (notably NHLBI) even give similar breaks to young investigators' first renewal applications. In general, I'd advise that the benefits of retaining eligibility as a new investigator outweigh those of Co-PI status, so you should stick to being just a co-Investigator. Note, however, that neither being a Core Director (vs. a Project Leader) on a Program Project or Center grant nor being the PI or co-PI of an R03 or R21 grant would adversely affect your New Investigator status.

As promised in my July Column, below are my ruminations in response to the recent Request for Information on NIH's research support mechanisms and peer review processes. My overall assessment parallels Winston Churchill's view of democracy – it's the worst possible system, excepting all others. Finally, what follows was written as a column, not merely my response to NIH, so before submission to the RFI site it will be revised to remove extraneous links, side comments, etc. (note that the deadline for responses has been extended to 4pm on September 7).

1. Specific challenges presented by NIH's current system of support for biomedical and behavioral research such as the current array of grant mechanisms, number of grants awarded per investigator, and the duration of grants.

If properly used and funded by the various Institutes and Centers, the extensive array of existing grant mechanisms is more than sufficient to cover all phases of individual research projects and a researcher's career. However, there are three (frequently related) situations in which promising researchers and/or their novel ideas are in danger of slipping through the cracks between current mechanisms. The first is the gap that frequently exists in a junior investigator's funding a couple of years after (s)he takes his/her first independent faculty position. If the institutional startup package runs out before the first NIH grant is obtained, a young faculty member's independent research career may be over before it ever really begins. This unfortunate outcome is particularly likely for those working in fields where the most pertinent NIH Institute does not support investigator-initiated R03 or R21 small grant applications. The standard NIH advice to new investigators is typified by a Program Director's presentation quoted in a recent blog: " concentrate on crafting and submitting a very competitive R01 rather than divert[ing]… effort to R21 or R03 proposals. Neither of the latter are renewable, and neither are appropriate “starter” grants on the road to independence." However, this advice presupposes that applicants have sufficient high-quality (preferably published) preliminary data to compete successfully for an R01; the typical 2-5 percentile point boost for new investigators often isn't enough to overcome this obstacle for those who must start their independent research careers essentially from scratch because continuing on from where their postdoctoral work left off would place them in direct competition with their mentors' well-established labs. A similar problem exists for more experienced investigators branching out into new areas in which the relevant Institute doesn't support investigator-initiated pilot grants. Indeed, their plight is often worse, as many private funding agencies that provide small grants for young investigators don't support more senior applicants. Finally, there's a particular problem for X-ray crystallographers who have successfully purified an interesting protein, but must still conduct the crystallization trials. Unfortunately, most NIH Study Sections and Program Officers don't consider these a suitable topic for pilot grants, even though for an important protein or family such trials would seem a quintessential example of "high risk, high reward."

2. Challenges of NIH Review Process. The current problems are by now well-known – given the increasing time pressure from their own institutions and the need to maintain their own funding levels, many potential reviewers are now unwilling or unable to devote the time required for thorough reviews and Study Section meetings under the current application formats and review processes. This often leads to uneven reviews by inexperienced reviewers, and a lack of consistency from one funding round to the next as Study Sections must increasingly rely on ad hoc reviewers. Resubmissions properly addressing the initial reviewers' concerns thus often face an entirely new set of criticisms, when previously they would have been funded.

3. Solutions to Challenges in Item 1. I believe that the best cure for the frequent funding gap between new investigators' institutional startup funds and their initial NIH grant would be revival of the R29 FIRST award, but with an increased funding maximum of seven modules for each of five years. To overcome the Study Sections' understandable reluctance to fund grants for five years with less than the usual amount of preliminary data, perhaps there could be a somewhat more rigorous than usual programmatic review of progress after year 3 of the project, before years 4 and 5 are approved for funding. To solve the other major problem previously experienced with this mechanism, the rules should be changed to allow renewal of an R29 as an R01 with the same grant number. I would also suggest that applicants with New Investigator status be allowed to submit a third revision of their grant proposals, as they are usually less adept at the mechanics of grant-writing than are more established investigators. The other two problems I discussed under Item 1 could easily be solved by ensuring that all Institutes support the investigator-initiated R03 and R21 funding mechanisms, and make a policy statement explicitly supporting the suitability of R03 grants to fund crystallization trials for novel proteins or protein families of particular import to that Institute. The shared difficulty in implementing all these suggestions is the autonomy of the individual Institutes and Centers in setting their own funding policies and priorities – if persuaded of their wisdom, Director Zerhouni and the CSR can strongly suggest, but not mandate these changes.

Solutions to Challenges in Item 2. Given the long-standing success of the current peer review process, and its clear contribution to American dominance of biomedicine and biotechnology, I think it premature to undertake a total revamp of the present system before determining whether a less drastic approach can achieve the desired results. Two changes already being either planned or implemented (decreasing the page limits for grant applications, and allowing ad hoc reviewers to "attend" Study Section meetings via videoconferencing when necessary) should substantially increase participation in the existing peer review process. [For an interesting discussion of some more drastic changes to the current peer review process, see Michele Pagano's provocative letter to Cell "American Idol and NIH Grant Review " and the responses it engendered (the latter link is to the first of five related letters; you can easily access them all online via the Table of Contents for that issue of Cell)].

4. Core Values of the NIH Peer Review Process. The most important core values that must be maintained are those of the NIH itself, which can be summarized as "improving human health by advancing scientific knowledge." These two primary goals may in some ways seem conflicting, especially given today's inadequate budgetary resources. But while improving human health may appear the more "applied" goal, and advancing knowledge to be more "basic" (with "translational" research spanning the intervening gap), the overarching core value must be the linkage between the two, for without an adequate basic foundation, we have no way of knowing how to improve human health most effectively. The risk is that by focusing too much on the first goal, NIH may interrupt and damage the translational pipeline that must be engendered by the second. As for the peer review process itself, the first core value is encompassed by the name – proposals should be reviewed by the applicant's true peers in terms of knowledge and expertise. Other core values here are fairness, impartiality, confidentiality, and the highest ethical standards of conduct for both reviewers and applicants. Finally, NIH has three other core values that may sometimes conflict with the desire to fund only the very best science – promoting diversity in the research community, advancing the missions of all the various Institutes and Centers, and fostering the next generation of scientists.

5. Peer Review Criteria and Scoring. For the most part, I think the current review criteria are appropriate and well written; however, to decrease potential attention to relatively trivial issues, under "Approach" I'd modify the second question to read "Does the applicant acknowledge likely potential problem areas and consider alternative tactics?" (emphasis added here to highlight the change). The major problem with the existing review criteria is not what's written, but rather the unwritten criteria that are part of the "culture of peer review." The first of these is the amount (and form) of preliminary data required to convince the Study Section of a project's "feasibility." All too often reviewers demand so much preliminary data as to create a nearly insurmountable barrier for a researcher's initial application in a given area, and justify the complaint that "they want me to do most of the project before they'll fund it!" Indeed, reviewers often tell colleagues of the "unwritten rule" that an R01 application should be supported by at least one publication "in a good journal," while reviews of R21 applications often ask for more preliminary data even though the parent R21 Program Announcement says " Preliminary data are not required but may be included if available. " A similar situation exists for the amount of methodological detail required in the Research Design and Methods Section. The latter two issues can also raise questions as to the impartiality of the review process, as it often appears that more preliminary data, methodological details, or inclusion of consultants for new techniques are required of new investigators than of established researchers perceived as "having a good track record." Finally, some reviews give the impression of criticizing the applicant for "not being a member of the club," perhaps with an eye to "turf protection;" such instances seem more frequent when the NIH budget is tight. The solution for all these issues would seem to be more emphatic enforcement of the written review criteria by the Study Section's SRA, with pointed reminders that certain considerations are inappropriate.

As for the scoring process, I think the biggest issue is the relative weight that should be given to the different review criteria in arriving at the overall priority score. While I suspect that most reviewers give less weight to the "environment" than to the other four areas, I would suggest that except for R21 applications or those in response to FOAs that emphasize a high risk, high reward rubric, the "innovation" category should also be given lower weight. It is frustrating to read a summary statement for an application that the reviewers agree presents a state-of-the-art research plan from a talented investigator seeking to address a very important problem, yet whose priority score is merely in the "excellent" (rarely funded) range because one or more reviewers finds nothing innovative about the project. The problem here is again often cultural, as until the most recent revision of the review criteria, "innovation" was usually taken to mean "using a novel approach or technique;" while the current criteria have clearly broadened that definition, not all reviewers appear to have gotten the message. Intervention by the SRA would again seem the best solution in such situations, as the only obvious alternative (separate scores for the five criteria, with a weighted formula to calculate the final score) would be much more cumbersome and time-consuming than the current method. I do like the idea (suggested in a recent CSR workshop) of having reviewers provide a second "best case" score to help applicants evaluate the chances of a revision being funded. However, for this to be a uniformly valuable addition to current procedures, all (especially triaged) applications would have to receive this "what if" score.

6. Career Pathways. My thoughts in this area are incorporated under question 1.

Happy August – it's hard to believe that classes will be starting in another three weeks. Those planning submissions for the October/November application cycle should already be well into writing – let's hope a storm doesn't interrupt the creative process!

-- Dave Konkel x24074; E-mail: dkonkel@utmb.edu (copyright 2007).