viernes, 2 de marzo de 2018

NIMH » NIMH’s Portfolio Balance: Quality Science Comes First

NIMH » NIMH’s Portfolio Balance: Quality Science Comes First

Logo for the National Institute of Mental Health (NIMH).

NIMH Director's Message

NIMH’s Portfolio Balance: Quality Science Comes First

By  on 
As I mentioned in my Sophomore Year message in November, one of the topics I have often been asked about is the balance of research investments NIMH makes. Early on in my tenure as NIMH director I noted that we need to first fund excellent science, and within the context of excellent science, we need to maintain a diverse portfolio of short-, medium-, and long-term investments if we are to maximize our impact on public mental health. This need for diversity of timeframes reflects the dilemma we face in trying to relieve the burden for those who suffer from mental disorders now, while also supporting research that leads to more effective treatment and prevention programs in the future.
Naturally, I wanted to know: what does our current balance look like? So, I asked. And it turns out, it was not such an easy question to answer, for many reasons. Not the least of those reasons is that it is quite challenging to estimate how soon a research project will impact the practice of mental health providers, or otherwise make a difference for those suffering from mental illnesses. Nonetheless, a team of dedicated NIMH employees worked hard to try to answer this question. While I can’t say we have a definitive, exact answer, we have come up with several ways to look at our portfolio of research investments that get at how much we invest in different timescales of opportunity.
Poring over these data, I’ve made three observations. First, no matter how you measure it, our balance has changed over the past 10 years. Second, changing scientific priorities likely played a role in this shift. And third, there are early signs of a reversal in this shift. I’ll show you the data behind each of these observations, and then conclude with a discussion of some of the implications.
  • 1. No matter how you measure it, our balance has shifted.
There are lots of ways to tally up how much we invest in a particular area, or in a particular timeframe. We’ve tried multiple ways, but they all show pretty much the same thing. Here I’m showing you data from two methods focused on defining short-, medium-, and long-term timeframes.
In Figure 1, we rely on data from each of the divisions of NIMH that fund research. These include extramural funding divisions like the Division of Neuroscience and Basic Behavioral Science (DNBBS), which focuses on increasing our understanding of how the brain works and what goes wrong in disease; the Division of Translational Research (DTR), which funds research aimed at understanding the pathophysiology of, and developing novel treatments for, mental illnesses; and, the Division of Services and Intervention Research (DSIR), which funds efforts to test interventions and enhance the utilization of successful treatments. Roughly speaking, DNBBS programs are mostly long-term investments, DTR programs are mostly medium-term investments, and DSIR programs are mostly short-term investments. As you can see, over the past decade, the DNBBS portfolio has grown slightly, with corresponding decreases in DTR and DSIR. Note also the increase in expenditures labeled “OD/Offices,” which comprises a mixed portfolio of long-term investments in genetics and technology, as well as short-term investments in research on global mental health, health disparities, and other NIMH priorities.
Figure 1
Fiscal YearPercent of FundingNIMH Funding by Division and Office/Office of the DirectorDNBBSDTRDSIRDAROffice/ODIRP20072008200920102011201220132014201520160510152025303540
NIMH Funding by Division and Office/Offices of the Director 
(Percent of Funding)
YearDNBBSDTRDSIRDAROffice/ODIRP
200728.731.815.212.30.511.5
200830.232.513.112.30.311.6
200931.930.813.712.10.111.5
201032.831.212.512.20.111.3
201133.930.212.711.80.311.2
201233.330.311.812.01.511.0
201335.029.410.311.92.211.2
201433.229.011.211.04.411.3
201531.828.510.811.16.411.5
201633.127.49.811.07.311.5
Figure 1. Percent of NIMH research funding presented for funding offices within the Office of the Director (OD) and for each funding division over time (FY 2007 – FY 2016). See Methods for additional information.
Figure 2 reflects our attempt to better define research projects corresponding to different timeframes. We did a deeper dive into the specific projects supported by each of these various divisions and offices, sorting them into three categories: Therapeutics Development and Services research, Disease-related Basic research, and Fundamental Basic research. The first category includes things like clinical trials and implementation research, and represents primarily short-term investments. The second category includes research in humans and experimental systems aimed at understanding the mechanisms of illness, akin to medium-term investments. The final category comprises basic research aimed at understanding how the brain works, how it is influenced by its environment and how it guides behavior, representing long-term investments. There are two takeaways here. First, purely basic science represents a relatively small but growing part of the NIMH portfolio. And second, from 2011-2014 there was a modest but noticeable shift away from Therapeutics Development and Services research that has since stabilized.
Figure 2
Fiscal YearPercent of FundingNIMH Funding by Research AreaTherapeutics Development and ServicesDisease-related BasicFundamental Basic2007200820092010201120122013201420152016051015202530354045505560
NIMH Funding by Research Area 
(Percent of Funding)
YearTherapeutics Development and ServicesDisease-related BasicFundamental Basic
200748.040.711.3
200849.139.311.6
200949.638.112.4
201049.238.312.5
201148.438.712.9
201245.340.913.9
201340.144.015.9
201438.546.015.6
201538.546.315.2
201638.745.016.4
Figure 2. Percent of NIMH research funding presented by research area over time (FY 2007 – FY 2016). See Methods for additional information.
  • 2. Changing scientific priorities likely played a role in this shift.
What happened that explains this shift? Was there a change in how we made funding decisions? Or a change in scientific priorities? We can get some clues by looking at application numbers and success rates over time, which we can do for the division data. Given that a majority of our categorized portfolio comprises research project grants (RPGs), we specifically reviewed the RPG data to determine if there are any drivers for this change. As shown in Figure 3 (which shows RPGs only), comparing these statistics for each of DNBBS (Figure 3a), DTR (Figure 3b), and DSIR (Figure 3c) reveals part of the answer. Success rates, which vary from year to year based on the overall NIMH budget and other factors, have not changed dramatically. This suggests constancy in how the divisions select grant applications for funding. What has changed, however, is the number of applications being submitted in the areas covered by these divisions. DTR and DSIR, which fund translational and services research, respectively, have seen significant drops in the number of submitted applications over time. These data suggest that for the most part, the drop in short-term investments may be due to a decrease in applications in these areas.
Figure 3a
Fiscal YearNumber of ApplicationsSuccess Rate (Percent)Division of Neuroscience and Basic Behavioral Science (DNBBS)Research Project Grant (RPG) Applications, Awards, and Success Rates9759751,0631,0639879879259251,0131,0131,0501,0501,0511,0511,0421,0421,0001,0001,1191,11923623623023022322326426418018026126123023021621621521526426424%24%22%22%23%23%29%29%18%18%25%25%22%22%21%21%22%22%24%24%ApplicationsAwardsSuccess Rate200720082009201020112012201320142015201602004006008001,0001,2001,4001,6001,8002,000051015202530
Division of Neuroscience and Basic Behavioral Science (DNBBS) 
Research Project Grant (RPG) Applications, Awards, and Success Rates
Fiscal YearApplicationsAwardsSuccess Rate
200797523624
20081,06323022
200998722323
201092526429
20111,01318018
20121,05026125
20131,05123022
20141,04221621
20151,00021522
20161,11926424
Figure 3b
Fiscal YearNumber of ApplicationsSuccess Rate (Percent)Division of Translational Research (DTR)Research Project Grant (RPG) Applications, Awards, and Success Rates1,1111,11196696696496492392397197194794795695686686672172164464422822819219218618615815814614618418415915917217211811815215221%21%20%20%19%19%17%17%15%15%19%19%17%17%20%20%16%16%24%24%ApplicationsAwardsSuccess Rate200720082009201020112012201320142015201602004006008001,0001,2001,4001,6001,8002,0000510152025
Division of Translational Research (DTR) 
Research Project Grant (RPG) Applications, Awards, and Success Rates
Fiscal YearApplicationsAwardsSuccess Rate
20071,11122821
200896619220
200996418619
201092315817
201197114615
201294718419
201395615917
201486617220
201572111816
201664415224
Figure 3c
Fiscal YearNumber of ApplicationsSuccess Rate (Percent)Division of Services and Intervention Research (DSIR)Research Project Grant (RPG) Applications, Awards, and Success Rates471471390390376376411411422422344344398398398398223223220220818179797373666662627171555578783939464617%17%20%20%19%19%16%16%15%15%21%21%14%14%20%20%17%17%21%21%ApplicationsAwardsSuccess Rate200720082009201020112012201320142015201601002003004005006007008009001,0000510152025
Division of Services and Intervention Research (DSIR) 
Research Project Grant (RPG) Applications, Awards, and Success Rates
Fiscal YearApplicationsAwardsSuccess Rate
20074718117
20083907920
20093767319
20104116616
20114226215
20123447121
20133985514
20143987820
20152233917
20162204621
Figures 3a-c. Number of RPG applications and awards, and percent success rates, for DNBBS (a), DTR (b), and DSIR (c) over time (FY 2007 – FY 2016). See Methods for additional information.
But why should applications have decreased? Here I have only circumstantial evidence, in terms of changes in NIMH scientific priorities that occurred over this time period. In the 2000s, NIMH concluded several large clinical trials, including STAR*DStep-BD, and CATIE, real-world studies of the effectiveness of antidepressants, mood-stabilizers, and antipsychotics, respectively. These were succeeded by large-scale investments in ARMY STARRSED-SAFEED-STARS, and RAISE programs aimed squarely at suicide-related behaviors in Army and civilian populations, and at first episode psychosis. As these programs continued into the 2010s, the National Advisory Mental Health Council recommended that NIMH re-evaluate its investments in clinical research and shift attention toward the elucidation of pathophysiological mechanisms. The Research Domain Criteria (RDoC) project was launched in 2011, and the experimental therapeutics platform for clinical trials was announced around the same time. These two initiatives posed challenges for those engaged in pragmatic clinical trials and services and implementation research, the mainstays of our short-term investment portfolio. Based on data in Figures 3 b-c, it seems as if these challenges may have led to the reductions in application numbers.
It should be noted that the relative decrease in applications to DTR and DSIR is more dramatic than the relative changes in funding seen in Figures 1 and 2; this difference is due to pronounced increases in the average costs of DTR and DSIR grants, which have risen much faster (34-54%) than the costs of DNBBS grants (19%). These increased costs reflect, at least in part, NIMH-imposed requirements aimed at increasing rigor and reproducibility in clinical research.
  • 3. There are signs of a reversal.
Although Figures 1 and 2 suggest that the portfolio shifts have stabilized, Figure 4 suggests that short-term research may be increasing of late. Because most NIMH RPGs are 4-5 year projects, the graphs of total expenditures (Figure 4a) are dominated by out-year expenses on non-competing grants awarded 4 or 5 years ago (Figure 4b). If instead we focus only on competing grants (Figure 4c), we find that the stabilization in application numbers for DTR and DSIR (Figures 3 b-c), coupled with an increase in success rates, resulted in a relative increase in the applied (short-term) expenditures in 2015 and 2016. Because the number of competing grants is relatively low and varies from year to year, we can see this most clearly by combining Disease-focused and Fundamental Basic research into one pot (“Basic”) and comparing this to Therapeutics Development and Services research (“Applied”). The uptick in relative expenditures in the Applied area in new RPGs suggests we may be seeing increases in our short-term expenditure portfolio after a nadir in 2013.
Figure 4a
 
Fiscal YearDollars (Millions)NIMH-funded (Non-competing and Competing) Research Project Grants (RPGs)by Research AreaAppliedBasic20072008200920102011201220132014201520160100200300400500600
NIMH-funded (Non-competing and Competing) Research Project Grants (RPGs) by Research Area 
In Dollars
YearAppliedBasic
2007383,003,340382,364,971
2008407,473,594382,858,164
2009435,149,694390,846,849
2010443,064,893424,182,585
2011423,363,982427,847,757
2012407,912,900471,816,273
2013330,104,691499,214,163
2014320,898,455529,065,600
2015324,234,566525,741,704
2016356,411,637550,782,949
Figure 4b
 
Fiscal YearDollars (Millions)NIMH-funded Non-competing Research Project Grants (RPGs)by Research AreaAppliedBasic2007200820092010201120122013201420152016050100150200250300350400450
NIMH-funded Non-competing Research Project Grants (RPGs) by Research Area 
In Dollars
YearAppliedBasic
2007267,266,316283,204,324
2008278,380,510297,319,720
2009335,170,414288,268,430
2010340,482,982302,320,708
2011337,626,924335,225,378
2012327,427,139331,321,254
2013263,142,035361,447,367
2014232,849,910393,664,462
2015239,244,849390,998,856
2016242,775,367413,525,750
Figure 4c
Fiscal YearDollars (Millions)NIMH-funded Competing Research Project Grants (RPGs) by Research AreaAppliedBasic2007200820092010201120122013201420152016020406080100120140160
NIMH-funded Competing Research Project Grants (RPGs) by Research Area 
In Dollars
YearAppliedBasic
2007115,737,02499,160,647
2008129,093,08485,538,444
200999,979,280102,578,419
2010102,581,911121,861,877
201185,737,05892,622,379
201280,485,761140,495,019
201366,962,656137,766,796
201488,048,545135,401,138
201584,989,717134,742,848
2016113,636,270137,257,199
Figures 4a-c. NIMH-funded competing and non-competing RPGs (a), non-competing RPGs (b), and competing RPGs (c) by research area over time (FY 2007 – FY 2016). See Methods for additional information.

Conclusions

To best interpret these findings, we must first remind ourselves that the NIMH mission is not to fund grants per se, but rather to transform the understanding and treatment of mental illnesses. From this perspective, it is crucial to recognize that the strategic shifts in scientific priorities initiated by NIMH in the past decade were motivated by the recognition that what we had been doing wasn’t working. Our clinical trials portfolio was failing to develop truly novel therapies, our emphasis on DSM-based diagnoses was making little headway into understanding the heterogeneity and neurobiological bases of mental illnesses, and the large effectiveness trials we had been conducting did little to change clinical practice. We made some radical changes to improve the quality and impact of the science we were funding because we were focused on our mission, as we should be.
There are plenty of other factors to consider that have affected the balance of investments over time. Clinical and implementation scientists have begun to adjust to our strategic shifts. Meanwhile, program staff in DSIR and DTR have worked hard to communicate with investigators about new opportunities and to encourage cutting-edge research programs in high-priority areas through funding announcements and requests for applications. These factors are likely playing an important role in stabilizing and perhaps reversing downward trends in these areas by ensuring mission-driven, high-quality applications.
It is also important to recognize that increases in research expenditures in basic science (both Fundamental and Disease-focused) are driven by an incredible wealth of new neuroscience tools as well as gains in the understanding of the neurobiological bases of behavior and of mental illnesses. These developments have created a growing cadre of talented young scientists with the potential to revolutionize our understanding of the brain in both health and disease. Both our focus on high-priority research on short- and medium-term goals, and the emergence of outstanding young neuroscientists using powerful new techniques to address long-term objectives, enhance the quality of the NIMH portfolio across all timescales.
Indeed, ensuring the research is of the highest quality and impact is my primary goal as NIMH director. I think it truly important that we be as transparent as possible about the kinds of research we fund and the balance of investments we maintain. We must also continuously monitor our short-, medium-, and long-term investments for scientific and public health impact. Nonetheless, quality scientific opportunities will continue to be the primary driver behind our funding decisions, rather than any particular idea of what our balance should be. There is no optimal portfolio balance if the science underlying it lacks impact. Accordingly, I will do all I can to support and encourage investigators to engage in the highest quality science across all areas of our portfolio.

No hay comentarios:

Publicar un comentario