Email updates

Keep up to date with the latest news and content from Genome Biology and BioMed Central.

Highly Accessed Comment

Risky business

Gregory A Petsko

Author Affiliations

Rosenstiel Basic Medical Sciences Research Center, Brandeis University, Waltham, MA 02454-9110, USA

Genome Biology 2011, 12:119  doi:10.1186/gb-2011-12-6-119

The electronic version of this article is the complete one and can be found online at: http://genomebiology.com/2011/12/6/119


Published:30 June 2011

© 2011 BioMed Central Ltd.

Comment

My parents came of age during the Great Depression of the 1930s - you know, the economic catastrophe that was caused by the financial industry. (Oh yes, now that you mention it, the financial industry caused the latest one too. And most of the ones in between. Good thing we finally reformed the rules regulating that industry and put the worst crooks behind bars - what do you mean, we didn’t?)

Well, one thing that is certain is that my parents - and their entire generation - were shaped by that experience as by nothing else in their lives. Even World War II didn’t have the effect that ten years of worrying about the essentials (food, clothing, shelter) did. Sixty years after the post-war recovery (and both of them lived to see the millennium turn), they were still ultra-conservative with money, wasted not a scrap of food, and never believed that good fortune was anything but a temporary phenomenon.

The lesson I learned from this is that times of scarcity make you risk-averse, and if you are risk-averse to begin with, they make you riskophobic, which is not a real word but ought to be, since ‘risk’ is actually derived from the Greek word for ‘cliff’, which anyone with a fear of heights (batophobia, which I know sounds like it should mean fear of bats, but it doesn’t; fear of bats is chiroptophobia) will tell you is worth being phobic about. While there is no official phobia for fear of risk-taking, there is one for fear of failure, which I would argue is pretty much the same thing. That word is atychiphobia, and you will find it in a list of known phobias somewhere between arachibutyrophobia (fear of peanut butter sticking to the roof of the mouth) and aulophobia (fear of flutes). (Personally, I would have thought that anyone with peanut butter stuck to the roof of their mouth had good reason to fear flutes, but a discussion of that is probably best saved for a future column.)

Atychiphobia is rampant in many quarters in this potentially Lost Economic Decade we’re living through, but nowhere will you find it more in evidence than in the government agencies that fund biomedical research. They won’t admit it, of course (rhabdophobia - fear of being criticized), but they have become so conservative about how they give their money away, you’d swear they had harpaxophobia (fear of having one’s valuables stolen).

We’ve all encountered this, I suspect - I know I have. It’s gotten to the point where, in order to get a research proposal funded, you need to have already done nearly all the experiments you propose to do, so as to assure the people reviewing the grant that there is no possibility you won’t succeed. Incremental science tends to be favored in times like this because it’s safe. New ideas and difficult or ground-breaking experiments, on the other hand, have a much harder time being supported.

Now, this may sound like I’m being critical of the bureaucrats who run the funding agencies, but actually, I’m not. For the most part, they don’t make funding decisions on individual investigator-initiated grants. As Cassius might have put it had he been planning to submit a grant to the National Institutes of Health, ‘The fault, dear Brutus, lies not in our grant administrators, but in ourselves.’ Because, of course, the ranking of submitted research proposals is typically done by peer review, and if science funding has become a game that only conservative research can win, we have no one to blame but us. When we criticize those who eschew risk in favor of safe science, we should be looking in a mirror.

The problem, of course, is that once things have gotten this way, it’s very hard to reverse the trend. Reviewing panels are already filled with people who have no memory of how the system is supposed to work. It doesn’t take more than 5 to 10 years of tight funding for risk-aversion to become ingrained in the culture.

Which is why a soon-to-be-released report from economists at the Sloane School at MIT ought to be must reading for everyone in Washington, because it suggests that there may be a structural way around the problem. The study, which will appear in the RAND Journal of Economics, asserts that if scientists are given more time and latitude in their research - especially the freedom to fail and try again before their work is evaluated for renewal - they will produce a significantly higher number of high impact papers. Pierre Azoulay, one of the coauthors for the study, says that good bosses understand this and make it clear to their employees that short-term failure is tolerable.

The report points out that high-pressure environments that focus on short-term success - exactly the sort of culture we now find in scientific funding - often hinder creativity. To find out what sort of situations foster it, the authors looked at two groups of biologists (including some genome biologists). One group consisted of investigators funded by the Howard Hughes Medical Institutes - a foundation whose stated approach is to fund people, not projects. Hughes Investigators are given five years of support at a time, but their renewal is based on how innovative they have been and the potential of the work they plan to do to propel the field forward, not whether they have adhered to the specific goals they listed at their last review. Reviewers are discouraged from critiquing the details of the proposal, and riskier projects are more likely to meet with favor than incremental science, which is the surest way to get an Investigator discontinued. Failure to meet lofty goals is tolerated, in the short term, if the Investigator has a track record of getting the job done, or is trying to do something that is deemed of extraordinary significance if he or she can pull it off.

The second group was chosen from scientists whose career accomplishments were deemed similar, but who were primarily supported by the National Institutes of Health (NIH) through peer-reviewed proposals - that is, as part of the risk-averse culture we’ve just talked about. These biologists also typically receive five years of funding at a time, but their renewal is based on how well they meet the specific aims they proposed when the project was funded, and failure to do so by the end of the first funding period usually leads to the project not being renewed.

The study found that the first group produced twice as many high-impact papers as the members of the second group. ('High impact' was defined as being in the top 1% by citation index.) Interestingly enough, the first group was also more likely to produce papers that were of low impact, suggesting that they were failing about as often as they were succeeding, which was interpreted as indicating that they were taking many more risks.

We can criticize some aspects of this study, such as the use of citation frequency as a measure of impact (at least the report’s authors didn’t just count the numbers of papers published in Nature, Science or Cell), and we might wonder whether a truly comparable group of non-Hughes Investigators was selected, but it certainly seems to make sense, doesn’t it? And I’m not the only one who thinks so. The Wellcome Trust, the largest supporter of biomedical research in Great Britain, launched a program two years ago of giving flexible grants to researchers, rather than supporting specific projects.

If NIH is serious about encouraging innovation; if the agency really wants high-risk/high-return research to be supported; if people who manage scientific funding realize that we scientists ourselves are the cause of the excessively conservative culture that pervades biomedical research, then the administrators in Washington should consider making structural changes that foster those aims. They can start by forcing senior scientists who receive NIH support to serve on grant review panels, thereby providing some perspective on the way peer review is supposed to function. Furthermore, they can, for people with a demonstrated track record of high quality science, eliminate the need to provide detailed descriptions of how research is to be accomplished or even require that specific aims be met as a condition for renewal. They can provide funding for up to seven years, not just five. And they can change the way renewal of such projects is evaluated. The only criteria that should matter ought to be: how important will the results of this work be if it is successful, and does this investigator have a past history of making important discoveries and/or doing things that one would have thought were difficult or impossible to do. Among other benefits, such a system would encourage successful scientists to strike out into new areas, since preliminary data would be much less important than the likely significance of the work, and false starts would be tolerated if the eventual goal was transformative for the field.

Some may argue that NIH already has a system like this in place, the so-called Merit Awards, in which an investigator with a long record of good scores on a grant is given ten years of funding at a time, supposedly to encourage risks and provide leeway for false starts. I used to think that program was a great step in the right direction, until I learned of several recent cases in which Merit awardees had their next year's funding held up until they had shown that they were making significant progress towards their original specific aims. Clearly, the culture of risk-aversion is spreading from the scientists who review one another’s proposals to the program officers who administer the grants. Nothing but radical changes in the structure of funding at least some proposals will reverse this slide towards mediocrity.

I'm not proposing that NIH, or any other agency, put all its resources into this funding model, but right now, because of atychiphobia, it’s hardly putting any. I think that at least a quarter of all grants should be funded the way I suggested; a third would be better. I could discuss this in more detail, but kopophobia (fear of fatigue) prompts me to stop writing, before I develop ergasiophobia (fear of work) and you develop hippopotomonstrosesquipedaliophobia, which I’m sure I don’t have to tell you, is fear of long words.