Back to Basics: A call for fundamental neuroscience research 11

Posted by Story Landis

NINDS supports a broad range of research projects, from basic studies of the nervous system to large Phase III clinical trials. Several years ago, we embarked on an institute-wide planning process to analyze and optimize our investments in basic, translational, and clinical research. Triggered by the observation that between 2003 and 2008, NINDS funding for R01s decreased by 10%, we extended our analyses to determine how our extramural funds are distributed across the spectrum of basic and applied research, and whether that distribution has changed over time.

To perform the analysis, we developed simple definitions of basic and applied research (listed at the end of this post) that could be applied as unambiguously and reproducibly as possible. We also divided each of these broad categories into two subcategories—basic/basic, basic/disease-focused, applied/translational, and applied/clinical. Expert neuroscientists, including program directors, scientific review officers, and other members of our staff then assigned funded projects to these subcategories based on careful reading of abstracts, specific aims, and, when necessary, additional sections of the grant application. Because a single application often proposed research in more than one subcategory, we assigned percentages of a grant to each subcategory as appropriate; for example, a grant could be described as 75% basic/basic and 25% basic/disease-focused.

Our analysis covered the period between 1997 and 2012 to ensure that any trends we observed did not reflect a short-term response to a particularly good or bad funding year. This analysis included most of the new and competing continuation grants issued each year. The specific funding mechanisms that we included are described below. Since this was an extremely labor-intensive task (and our staff have day jobs!), we selected eight years within this period for review.

Our first finding was that between 1997 and 2012, NINDS expenditures on applied research as a fraction of total competing research budget increased from 13% to 29% while the proportion of basic research declined from 87% to 71% (Figure 1).

Figure 1: Percentages of the NINDS “competing budget” (see definition below) devoted to basic research (basic/basic and basic/disease-focused categories combined) and applied research (applied/translational and applied/clinical categories combined)

Figure 1: Percentages of the NINDS “competing budget” (see definition below) devoted to basic research (basic/basic and basic/disease-focused categories combined) and applied research (applied/translational and applied/clinical categories combined)

When we divided basic research into basic/basic and basic/disease-focused subcategories, we observed a striking decline in the funding of basic/basic research, which decreased from 52% to 27% of our competing budget during this time period (Figure 2).

Figure 2:  Percentages of the NINDS competing budget in the four subcategories

Figure 2: Percentages of the NINDS competing budget in the four subcategories

One possible explanation for our increased spending on disease-focused research is that the cost of disease-focused research may have increased at a greater rate than that of fundamental basic research. We therefore examined the numbers of awards (as opposed to the dollars spent) in each broad category and subcategory, and found similar trends to those shown in Figures 1 and 2 (data not shown).

We also looked at the funding of basic/basic and disease-focused research specifically within the portion of the NINDS portfolio represented by investigator-initiated grants that were not solicited by the institute through targeted funding opportunity announcements (~70% in FY2012). Targeted solicitations are most often focused on filling scientific gaps in the Institute’s research portfolio related to understanding, preventing, or treating particular disorders or addressing public health concerns. By excluding applications submitted in response to these solicitations, we could assess the extent to which NINDS was driving the changes in relative funding of basic/basic and disease-focused grants. Figure 3 is analogous to Figure 2, but includes only investigator-initiated grants. The shift towards disease-focused research is less pronounced but still evident. While applications funded in response to NINDS solicitations tend to be disease-related or applied research, these data suggest that NINDS initiatives are not the major reason for the decline in basic/basic research funding.

Figure 3: Percentage of the competing budget spent on unsolicited investigator-initiated grants in the four subcategories

Figure 3: Percentage of the competing budget spent on unsolicited investigator-initiated grants in the four subcategories

What, then, is driving the overall NINDS shift from basic to applied research, and more specifically, the sharp decrease in fundamental basic research? If changes in the scientific landscape are driving this shift, then it could reflect a natural progression of the field, and NINDS certainly welcomes opportunities to translate basic discoveries into clinical applications. However, basic and applied research are necessarily intertwined and interdependent – basic discoveries lay the groundwork for clinical applications and applied investigations uncover new fundamental questions. We are therefore concerned by the alternative possibility that many investigators falsely believe that NINDS is no longer interested in supporting research into the normal function of the brain and nervous system, and that their chances of obtaining NIH funding are better if they propose disease-focused basic or applied studies, rather than fundamental basic science research. The current challenging budget environment and the growing number of NIH programs intended to accelerate translational research may be contributing to this belief. Comments from grant reviewers that question the relevance of proposed fundamental basic research projects to human disease could further reinforce the idea that a translational emphasis is advisable, if not required.

As a first step towards exploring this issue, we analyzed applications that were submitted during two specific years (2008 and 2011) for funding mechanisms included in the original analysis. We were interested in determining whether the trends shown above could be explained by changes in the numbers of applications submitted in the four subcategories or by differences in success rates among them. We found a striking decrease (- 21%) in dollars requested by PIs to support basic-basic research projects. In contrast, disease-focused basic requests increased 23%, applied-translational requests increased 42%, and applied-clinical requests increased 38%. However, Table 1 shows that despite these changes in application numbers, funding rates (Number of Funded Applications/Number of Total Applications) across categories remained quite stable over the same time period. Notably, fundamental basic science applications continue to perform as well, or better, in peer review than other applications.

Basic/
Basic
Basic/
Disease-Related
Applied/
Translational
Applied/
Clinical
2008 26% 20% 18% 22%
2011 26% 22% 14% 22%
Table 1: Funding rates of the four subcategories in 2008 and 2011, showing that Basic/Basic research applications perform well in review

My concern is that the decrease in the number of basic/basic applications reflects the perception that NINDS is only interested in disease-oriented research. NINDS is committed to maintaining a vibrant basic research enterprise. In fact, once we recognized the declining investment in basic/basic research, we began to emphasize fundamental basic science research in selecting high program priority grants for funding beyond the payline in 2012.  As is evident from the last data point, our actions arrested its decline, at least temporarily.

We would like to understand better the decline in basic science applications. For example, why aren’t researchers submitting more fundamental basic research applications to NINDS? Are researchers sending these applications to other NIH neuroscience institutes, for example the National Institute of Mental Health (NIMH) or the National Institute on Drug Abuse (NIDA)? Are most of the newly-funded disease researchers former basic researchers, or do they represent a new cohort? We plan to continue to examine the causes for the decline, informed by the input received from this post and additional ongoing analyses.

Fundamental basic research is the engine of discovery; it generates new knowledge, drives innovation, and underlies all past and future breakthroughs. Gaps in our understanding of how the healthy brain and nervous system function can form roadblocks to understanding dysfunction in disease. Supporting the basic studies to fill those gaps is a critical piece of the NIH and NINDS mission – and an area unlikely to find sustained investment from the private sector.

I believe that the most effective strategy for advancing scientific knowledge and translating that knowledge into therapies that benefit patients is to maintain a robust and balanced pipeline across the research spectrum – from fundamental basic research to large phase III clinical trials. This principle was articulated by Congress in the 2014 NIH appropriations bill. To that end, we recently revised our Institute’s mission statement to recognize the critical role of fundamental research while continuing to emphasize our commitment to enhancing health and reducing disability across the spectrum of neurological diseases and disorders. It now states: “The mission of NINDS is to seek fundamental knowledge about the brain and nervous system and to use that knowledge to reduce the burden of neurological disease”.

I welcome any thoughts you have about how NINDS can work towards this goal.

Story

Definitions:

The definitions of the basic and applied categories that we used are different from those officially used by NIH. The more general coding definitions used by NIH (for example, see OMB Circular A-11 (2011) pg. 262 for the definitions of basic and applied research and http://grants.nih.gov/grants/glossary.htm#C for the definition of clinical research) were less useful for our particular purposes, specifically the division into basic/basic, basic/disease-focused, applied/translational and applied/clinical. As an example of the differences between the NINDS and NIH definitions, all studies involving identified humans or identified human materials are coded as applied clinical research by NIH. In contrast, by the NINDS definition, a human brain imaging study investigating normal brain structure or function would be considered basic/basic research. By the NIH definition, a genome-wide association study would be considered applied/clinical research. By our definition, it would be categorized as basic/disease-focused research. We don’t consider our system of categorization to be “better” than NIH’s, just different. In addition, NIH coding methodologies changed significantly during the years included in our analysis.

  1. Basic Research: aimed at understanding the structure and function of the nervous system. Can involve studies performed in vitro, in animals, or in humans.
    1. Basic/Basic: focused on understanding the normal nervous system.
    2. Basic/Disease-Focused: focused on understanding disease mechanisms.
  2. Applied Research: aimed at developing or testing diagnostics, therapeutic agents, or preventive interventions. Can involve studies performed in vitro, in animals, or in humans.
    1. Applied/Translational: up to, but not including, first in human studies
    2. Applied/Clinical: first in human studies through phase III clinical trials.
  3. Competing budget: $ spent on the first years of all the grants awarded in a particular year. The out-years of these grants are referred to as the non-competing years. Monitoring the competing budget gives a real-time snapshot of funding decisions made on a year-by-year basis.
  4. Investigator-initiated research: Includes all funded unsolicited applications, including: applications submitted in response to parent announcements and program announcements (PAs); applications submitted by new investigators with percentiles just beyond our payline; a small number of “high program priority” applications selected by NINDS and approved by the NINDS Council; and clinical trial applications initiated by investigators.  This category excludes funded applications submitted in response to parent announcements with special receipt, referral and/or review considerations (PARs), parent announcements that includes specific set-aside funds (PASs), and requests for applications (RFAs).

Funding mechanisms included in the analysis: P01, P20, P30, P50, R01, R15, R21, R25, R29, R34, R37, R55, R56, S11, SC1, SC2, U01, U10, U24, U44, U54

Funding mechanisms not included: All Fellowship (F’s), Career (K’s), and Training (T’s) awards; All SBIR (R41/R42, R43/44) awards; All conference grants (R13’s); and Loan Repayment awards (L30, L40).

11 comments

  1. Story: You and your team are to be congratulated for performing and sharing this thoughtful analysis. The most sensible policy decisions can only be made after careful analysis and your analysis confirms what I and others suspected, namely that members of the scientific community may be over-reacting to the call more translation and are abandoning (or at least disguising) fundamental research that is essential for translation to be possible in the long run. Now we have some real data to quantify and refine our views.

  2. The data confirm what most of us in basic research has known for a long time, and, what is worse, increasingly experienced. It is really only a symptom of our societies’ trend to demand shorter time for return of investment, be it financially, politically,educationally, militarily etc.

  3. Thank you, Story, for uncovering and presenting these important, distressing findings.

    Two aspects of your Figure 2 deserve comment. First, expenditures for disease-focused basic research were essentially flat across your study period while “basic-basic” declined precipitously. That is, allocations to basic-basic research migrated to applied research, rather than to “basic-disease”.

    Second, your study period includes both dynamic (the NIH budget doubling, 1998-2003) and static (2004-2012) phases of overall NIH funding. If we assume that the NINDS budget has tracked roughly with the full NIH budget, relative basic-basic allocations fell dramatically during the doubling, (that it, basic science funding was least advantaged by the doubling) and “applied-translational” increased dramatically, while basic-disease and “applied-clinical” were relatively unchanged. After the budget flattened in 2004, basic-basic has continued to decline (that is, fewer dollars are being allocated) while applied-clinical has increased. That is, both in times of plenty and in times of budgetary stress, basic-basic research has been relatively disadvantaged.

    I suggest it would be helpful if you could present the Figure 2 and Figure 3 data as number of investigators and as actual (rather than relative) dollars. These vantage points may reveal how the investigator community has responded to increasing vs. static funding, and to perceived messages, from NIH leadership and from study sections, about the relative value of basic and applied research.

    Finally, you ask for thoughts on how NINDS can re-establish and “maintain a robust and balanced pipeline across the research spectrum”– precisely the correct goal. Let me venture a silly-sounding approach. You point out that NINDS uses targeted solicitations to fill “scientific gaps in the Institute’s portfolio” related to diseases or public health. Your new data reveal a growing scientific gap in your portfolio related to fundamental neuroscience knowledge. What if NINDS crafted targeted solicitations for proposals to pursue some broad, critical unknowns in basic neuroscience? Here, the “targeted” part of the solicitation is to attack curiosity-driven questions, so the solicitations would need to be written so broadly that investigators feel unconstrained.

    I am aware, of course, that the entire “unsolicited” portion of your portfolio is meant to invite just such unconstrained proposals. Thus, a targeted solicitation for untargeted research sounds silly. However, investigators seem to have lost confidence and trust that NIH and its study sections truly value basic science. My proposed targeted solicitations, which importantly should be reviewed in regular Center for Scientific Review study sections, would stand as an explicit message from NINDS, both to the applicant community and to study sections, that you actively seek and intend to reward proposals to uncover fundamental new knowledge in neuroscience.

  4. I would like to thank NINDS for evaluating the devastating effects that the push for “bench-to-bedside” priorities in grant funding have had on basic neuroscience research. As indicated in several other comments, many neuroscientists attempting to conduct research into basic scientific questions related to how the brain develops and functions in the adult and in aging individuals feel that it is extremely difficult to receive a sufficient priority score from a scientific review group that is within the range for funding if the proposal is “basic-basic”. Part of this appears to be a “mind” shift in reviewers (partly fostered by NIH) that if it is not obviously “translatable” to clinical use then it really is not important or significant research. If NINDS is serious about promoting more basic-basic research then it must get this message to reviewers on the study sections. As most scientists are aware, many very significant advances in neuroscience have not come from narrowly-focused clinically directed research but from studies exploring basic biological properties of how the brain is put together and functions on the microscopic and macroscopic level. I would applaud NINCDS if it actually would clearly identify funds specifically to support basic neuroscience research and not feel pressure to only focus on disease related research projects.

  5. Thank you, Story, for this very clearly presented piece addressing what I think is a fundamental shift in how researchers perceive how to “sell” their research. My research has been very basic over the past two decades, studying regulation of renal phosphate transport; however, recently, I have been receiving criticisms regarding the absence of disease relevance, application to patient care, etc. Furthermore, the “common knowledge” promulgated by individuals in positions to guide younger grant submitters is that the research needs to be disease-focused. I am not sure how this perception occurred because if you ask individuals at the NIH and other granting agencies, they do not seem to be wed to disease-focused research. Perhaps it is the members of the study sections who promote this agenda?

    • Thank you for your feedback. We have received a number of comments through the blog and directly to NINDS leadership about the possibility that study section members are placing particular emphasis on disease-focused research, and we have passed these on to the Center for Scientific Review (CSR) leadership.

      In response to these concerns and those that CSR has heard independently, CSR is working to incorporate language in their guidance to reviewers emphasizing the importance of every component of the research spectrum ¬ basic, translational, and clinical. Scientific Review Officers have also been encouraged to intervene if reviewers object to an otherwise meritorious fundamental basic research application because of its lack of direct disease relevance.

      We take the concerns about review seriously, but want to remind applicants that NINDS fundamental basic research applications do better overall in review than other types of NINDS applications.

  6. Regarding supporting the basic neuroscience research: “Neuroscience has discovered changes in the brain that underlie learning as well as the boundaries between swatches of cortex devoted to different body parts,
    talents, and even physical senses that can be adjusted by learning and
    practice. Neuroscientists already know that the brain is not malleable
    by experience. It is also already known that damaged lobes can not serve as inhibitory brakes on the limbic system via the stria terminalis. Connections
    bet. the frontal lobe in each hemisphere and the limbic system provide a lever by which a person’s knowledge and goals can override other mechanisms, and
    among those mechanisms appears to be one designed to generate heavier that harms other people”. So just what basic research is NINCDS supporting?
    Ch. 3: “The Last Wall to Fall” S. Pinker

  7. Pingback: Minutes April 7 | NIH Assembly of Scientists

  8. Felicitaciones y perdón. Yo no soy cientifica pero soy paciente
    He leído sobre las investigaciones que se estan haciendo sobre las enfermedades del sistema nervioso, para mi como paciente de esta enfermedad como la de parkinson que tengo, es muy importante ya que desearia que nadie pase lo que yo estoy pasando.Hace 4 años me dio la herpes zoster y me ha regresado, mi resistencia y voluntad hacen que pueda sobrellevar los inmensos dolores que tengo. Además tengo problemas muy serio con la columna.Espero que puedan leer este mensaja y que Dios ilumine y ayude a todos esos cientificos para que puedan ayudar a muchos seres humanos.

  9. Dear Story,
    This is a very interesting analysis. From talking to people, I too had the impression that study sections had shifted towards favoring disease-oriented research. I find the difference between the NINDS and NIH definitions of basic/basic research interesting because it suggests, at least in some quarters at the NIH, that work in humans should always be considered applied. This highlights a worry that I have had for a while that basic studies of human behavior are harder to place in the NIH research spectrum. The very small space put aside for behavior in the recent brain initiative is an example of this. I very much hope that NINDS will make it clear moving forward that humans should be considered a model system for basic research, just like any other animal. I also hope that in our excitement about all the new tools becoming available, that we do not forget they are ultimately secondary to what they have been developed to measure: behavior.

  10. I want to add my voice to the chorus of thanks. I would add that the review criteria of significance and impact contribute to this trend. Evaluating the impact of basic science requires the perspective of time, not the opinion of a reviewer.

    it is important to base the debate on the future of science in the USA on evidence. It is not the responsibility of NIDS but perhaps we, the investigator community, should ask the NIH to perform similar analysis across all institutes. It will also be important to correlate outcomes (positive outcome will include breakthroughs, therapies, products; negative – retractions) with the 4 categories of research outlined above.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s