NIMH’s Portfolio Balance: Quality Science Comes First
By on
As I mentioned in my Sophomore Year message in November, one of the topics I have often been asked about is the balance of research investments NIMH makes. Early on in my tenure as NIMH director I noted that we need to first fund excellent science, and within the context of excellent science, we need to maintain a diverse portfolio of short-, medium-, and long-term investments if we are to maximize our impact on public mental health. This need for diversity of timeframes reflects the dilemma we face in trying to relieve the burden for those who suffer from mental disorders now, while also supporting research that leads to more effective treatment and prevention programs in the future.
Naturally, I wanted to know: what does our current balance look like? So, I asked. And it turns out, it was not such an easy question to answer, for many reasons. Not the least of those reasons is that it is quite challenging to estimate how soon a research project will impact the practice of mental health providers, or otherwise make a difference for those suffering from mental illnesses. Nonetheless, a team of dedicated NIMH employees worked hard to try to answer this question. While I can’t say we have a definitive, exact answer, we have come up with several ways to look at our portfolio of research investments that get at how much we invest in different timescales of opportunity.
Poring over these data, I’ve made three observations. First, no matter how you measure it, our balance has changed over the past 10 years. Second, changing scientific priorities likely played a role in this shift. And third, there are early signs of a reversal in this shift. I’ll show you the data behind each of these observations, and then conclude with a discussion of some of the implications.
- 1. No matter how you measure it, our balance has shifted.
There are lots of ways to tally up how much we invest in a particular area, or in a particular timeframe. We’ve tried multiple ways, but they all show pretty much the same thing. Here I’m showing you data from two methods focused on defining short-, medium-, and long-term timeframes.
In Figure 1, we rely on data from each of the divisions of NIMH that fund research. These include extramural funding divisions like the Division of Neuroscience and Basic Behavioral Science (DNBBS), which focuses on increasing our understanding of how the brain works and what goes wrong in disease; the Division of Translational Research (DTR), which funds research aimed at understanding the pathophysiology of, and developing novel treatments for, mental illnesses; and, the Division of Services and Intervention Research (DSIR), which funds efforts to test interventions and enhance the utilization of successful treatments. Roughly speaking, DNBBS programs are mostly long-term investments, DTR programs are mostly medium-term investments, and DSIR programs are mostly short-term investments. As you can see, over the past decade, the DNBBS portfolio has grown slightly, with corresponding decreases in DTR and DSIR. Note also the increase in expenditures labeled “OD/Offices,” which comprises a mixed portfolio of long-term investments in genetics and technology, as well as short-term investments in research on global mental health, health disparities, and other NIMH priorities.
Figure 1
| Year | DNBBS | DTR | DSIR | DAR | Office/OD | IRP |
|---|---|---|---|---|---|---|
| 2007 | 28.7 | 31.8 | 15.2 | 12.3 | 0.5 | 11.5 |
| 2008 | 30.2 | 32.5 | 13.1 | 12.3 | 0.3 | 11.6 |
| 2009 | 31.9 | 30.8 | 13.7 | 12.1 | 0.1 | 11.5 |
| 2010 | 32.8 | 31.2 | 12.5 | 12.2 | 0.1 | 11.3 |
| 2011 | 33.9 | 30.2 | 12.7 | 11.8 | 0.3 | 11.2 |
| 2012 | 33.3 | 30.3 | 11.8 | 12.0 | 1.5 | 11.0 |
| 2013 | 35.0 | 29.4 | 10.3 | 11.9 | 2.2 | 11.2 |
| 2014 | 33.2 | 29.0 | 11.2 | 11.0 | 4.4 | 11.3 |
| 2015 | 31.8 | 28.5 | 10.8 | 11.1 | 6.4 | 11.5 |
| 2016 | 33.1 | 27.4 | 9.8 | 11.0 | 7.3 | 11.5 |
Figure 1. Percent of NIMH research funding presented for funding offices within the Office of the Director (OD) and for each funding division over time (FY 2007 – FY 2016). See Methods for additional information.
Figure 2 reflects our attempt to better define research projects corresponding to different timeframes. We did a deeper dive into the specific projects supported by each of these various divisions and offices, sorting them into three categories: Therapeutics Development and Services research, Disease-related Basic research, and Fundamental Basic research. The first category includes things like clinical trials and implementation research, and represents primarily short-term investments. The second category includes research in humans and experimental systems aimed at understanding the mechanisms of illness, akin to medium-term investments. The final category comprises basic research aimed at understanding how the brain works, how it is influenced by its environment and how it guides behavior, representing long-term investments. There are two takeaways here. First, purely basic science represents a relatively small but growing part of the NIMH portfolio. And second, from 2011-2014 there was a modest but noticeable shift away from Therapeutics Development and Services research that has since stabilized.
Figure 2
| Year | Therapeutics Development and Services | Disease-related Basic | Fundamental Basic |
|---|---|---|---|
| 2007 | 48.0 | 40.7 | 11.3 |
| 2008 | 49.1 | 39.3 | 11.6 |
| 2009 | 49.6 | 38.1 | 12.4 |
| 2010 | 49.2 | 38.3 | 12.5 |
| 2011 | 48.4 | 38.7 | 12.9 |
| 2012 | 45.3 | 40.9 | 13.9 |
| 2013 | 40.1 | 44.0 | 15.9 |
| 2014 | 38.5 | 46.0 | 15.6 |
| 2015 | 38.5 | 46.3 | 15.2 |
| 2016 | 38.7 | 45.0 | 16.4 |
Figure 2. Percent of NIMH research funding presented by research area over time (FY 2007 – FY 2016). See Methods for additional information.
- 2. Changing scientific priorities likely played a role in this shift.
What happened that explains this shift? Was there a change in how we made funding decisions? Or a change in scientific priorities? We can get some clues by looking at application numbers and success rates over time, which we can do for the division data. Given that a majority of our categorized portfolio comprises research project grants (RPGs), we specifically reviewed the RPG data to determine if there are any drivers for this change. As shown in Figure 3 (which shows RPGs only), comparing these statistics for each of DNBBS (Figure 3a), DTR (Figure 3b), and DSIR (Figure 3c) reveals part of the answer. Success rates, which vary from year to year based on the overall NIMH budget and other factors, have not changed dramatically. This suggests constancy in how the divisions select grant applications for funding. What has changed, however, is the number of applications being submitted in the areas covered by these divisions. DTR and DSIR, which fund translational and services research, respectively, have seen significant drops in the number of submitted applications over time. These data suggest that for the most part, the drop in short-term investments may be due to a decrease in applications in these areas.
Figure 3a
| Fiscal Year | Applications | Awards | Success Rate |
|---|---|---|---|
| 2007 | 975 | 236 | 24 |
| 2008 | 1,063 | 230 | 22 |
| 2009 | 987 | 223 | 23 |
| 2010 | 925 | 264 | 29 |
| 2011 | 1,013 | 180 | 18 |
| 2012 | 1,050 | 261 | 25 |
| 2013 | 1,051 | 230 | 22 |
| 2014 | 1,042 | 216 | 21 |
| 2015 | 1,000 | 215 | 22 |
| 2016 | 1,119 | 264 | 24 |
Figure 3b
| Fiscal Year | Applications | Awards | Success Rate |
|---|---|---|---|
| 2007 | 1,111 | 228 | 21 |
| 2008 | 966 | 192 | 20 |
| 2009 | 964 | 186 | 19 |
| 2010 | 923 | 158 | 17 |
| 2011 | 971 | 146 | 15 |
| 2012 | 947 | 184 | 19 |
| 2013 | 956 | 159 | 17 |
| 2014 | 866 | 172 | 20 |
| 2015 | 721 | 118 | 16 |
| 2016 | 644 | 152 | 24 |
Figure 3c
| Fiscal Year | Applications | Awards | Success Rate |
|---|---|---|---|
| 2007 | 471 | 81 | 17 |
| 2008 | 390 | 79 | 20 |
| 2009 | 376 | 73 | 19 |
| 2010 | 411 | 66 | 16 |
| 2011 | 422 | 62 | 15 |
| 2012 | 344 | 71 | 21 |
| 2013 | 398 | 55 | 14 |
| 2014 | 398 | 78 | 20 |
| 2015 | 223 | 39 | 17 |
| 2016 | 220 | 46 | 21 |
Figures 3a-c. Number of RPG applications and awards, and percent success rates, for DNBBS (a), DTR (b), and DSIR (c) over time (FY 2007 – FY 2016). See Methods for additional information.
But why should applications have decreased? Here I have only circumstantial evidence, in terms of changes in NIMH scientific priorities that occurred over this time period. In the 2000s, NIMH concluded several large clinical trials, including STAR*D, Step-BD, and CATIE, real-world studies of the effectiveness of antidepressants, mood-stabilizers, and antipsychotics, respectively. These were succeeded by large-scale investments in ARMY STARRS, ED-SAFE, ED-STARS, and RAISE programs aimed squarely at suicide-related behaviors in Army and civilian populations, and at first episode psychosis. As these programs continued into the 2010s, the National Advisory Mental Health Council recommended that NIMH re-evaluate its investments in clinical research and shift attention toward the elucidation of pathophysiological mechanisms. The Research Domain Criteria (RDoC) project was launched in 2011, and the experimental therapeutics platform for clinical trials was announced around the same time. These two initiatives posed challenges for those engaged in pragmatic clinical trials and services and implementation research, the mainstays of our short-term investment portfolio. Based on data in Figures 3 b-c, it seems as if these challenges may have led to the reductions in application numbers.
It should be noted that the relative decrease in applications to DTR and DSIR is more dramatic than the relative changes in funding seen in Figures 1 and 2; this difference is due to pronounced increases in the average costs of DTR and DSIR grants, which have risen much faster (34-54%) than the costs of DNBBS grants (19%). These increased costs reflect, at least in part, NIMH-imposed requirements aimed at increasing rigor and reproducibility in clinical research.
- 3. There are signs of a reversal.
Although Figures 1 and 2 suggest that the portfolio shifts have stabilized, Figure 4 suggests that short-term research may be increasing of late. Because most NIMH RPGs are 4-5 year projects, the graphs of total expenditures (Figure 4a) are dominated by out-year expenses on non-competing grants awarded 4 or 5 years ago (Figure 4b). If instead we focus only on competing grants (Figure 4c), we find that the stabilization in application numbers for DTR and DSIR (Figures 3 b-c), coupled with an increase in success rates, resulted in a relative increase in the applied (short-term) expenditures in 2015 and 2016. Because the number of competing grants is relatively low and varies from year to year, we can see this most clearly by combining Disease-focused and Fundamental Basic research into one pot (“Basic”) and comparing this to Therapeutics Development and Services research (“Applied”). The uptick in relative expenditures in the Applied area in new RPGs suggests we may be seeing increases in our short-term expenditure portfolio after a nadir in 2013.
Figure 4a
| Year | Applied | Basic |
|---|---|---|
| 2007 | 383,003,340 | 382,364,971 |
| 2008 | 407,473,594 | 382,858,164 |
| 2009 | 435,149,694 | 390,846,849 |
| 2010 | 443,064,893 | 424,182,585 |
| 2011 | 423,363,982 | 427,847,757 |
| 2012 | 407,912,900 | 471,816,273 |
| 2013 | 330,104,691 | 499,214,163 |
| 2014 | 320,898,455 | 529,065,600 |
| 2015 | 324,234,566 | 525,741,704 |
| 2016 | 356,411,637 | 550,782,949 |
Figure 4b
| Year | Applied | Basic |
|---|---|---|
| 2007 | 267,266,316 | 283,204,324 |
| 2008 | 278,380,510 | 297,319,720 |
| 2009 | 335,170,414 | 288,268,430 |
| 2010 | 340,482,982 | 302,320,708 |
| 2011 | 337,626,924 | 335,225,378 |
| 2012 | 327,427,139 | 331,321,254 |
| 2013 | 263,142,035 | 361,447,367 |
| 2014 | 232,849,910 | 393,664,462 |
| 2015 | 239,244,849 | 390,998,856 |
| 2016 | 242,775,367 | 413,525,750 |
Figure 4c
| Year | Applied | Basic |
|---|---|---|
| 2007 | 115,737,024 | 99,160,647 |
| 2008 | 129,093,084 | 85,538,444 |
| 2009 | 99,979,280 | 102,578,419 |
| 2010 | 102,581,911 | 121,861,877 |
| 2011 | 85,737,058 | 92,622,379 |
| 2012 | 80,485,761 | 140,495,019 |
| 2013 | 66,962,656 | 137,766,796 |
| 2014 | 88,048,545 | 135,401,138 |
| 2015 | 84,989,717 | 134,742,848 |
| 2016 | 113,636,270 | 137,257,199 |
Figures 4a-c. NIMH-funded competing and non-competing RPGs (a), non-competing RPGs (b), and competing RPGs (c) by research area over time (FY 2007 – FY 2016). See Methods for additional information.
Conclusions
To best interpret these findings, we must first remind ourselves that the NIMH mission is not to fund grants per se, but rather to transform the understanding and treatment of mental illnesses. From this perspective, it is crucial to recognize that the strategic shifts in scientific priorities initiated by NIMH in the past decade were motivated by the recognition that what we had been doing wasn’t working. Our clinical trials portfolio was failing to develop truly novel therapies, our emphasis on DSM-based diagnoses was making little headway into understanding the heterogeneity and neurobiological bases of mental illnesses, and the large effectiveness trials we had been conducting did little to change clinical practice. We made some radical changes to improve the quality and impact of the science we were funding because we were focused on our mission, as we should be.
There are plenty of other factors to consider that have affected the balance of investments over time. Clinical and implementation scientists have begun to adjust to our strategic shifts. Meanwhile, program staff in DSIR and DTR have worked hard to communicate with investigators about new opportunities and to encourage cutting-edge research programs in high-priority areas through funding announcements and requests for applications. These factors are likely playing an important role in stabilizing and perhaps reversing downward trends in these areas by ensuring mission-driven, high-quality applications.
It is also important to recognize that increases in research expenditures in basic science (both Fundamental and Disease-focused) are driven by an incredible wealth of new neuroscience tools as well as gains in the understanding of the neurobiological bases of behavior and of mental illnesses. These developments have created a growing cadre of talented young scientists with the potential to revolutionize our understanding of the brain in both health and disease. Both our focus on high-priority research on short- and medium-term goals, and the emergence of outstanding young neuroscientists using powerful new techniques to address long-term objectives, enhance the quality of the NIMH portfolio across all timescales.
Indeed, ensuring the research is of the highest quality and impact is my primary goal as NIMH director. I think it truly important that we be as transparent as possible about the kinds of research we fund and the balance of investments we maintain. We must also continuously monitor our short-, medium-, and long-term investments for scientific and public health impact. Nonetheless, quality scientific opportunities will continue to be the primary driver behind our funding decisions, rather than any particular idea of what our balance should be. There is no optimal portfolio balance if the science underlying it lacks impact. Accordingly, I will do all I can to support and encourage investigators to engage in the highest quality science across all areas of our portfolio.
No hay comentarios:
Publicar un comentario