When someone says “scientific tools”, you may think of laboratories with microscopes and mass spectrometers. But it is now also software and digital tools that are changing the nature of science and health research.
One example is Nextstrain, the open-source bioinformatics toolkit that is at the heart of efforts to sequence and track variants of the Sars-CoV-2 virus. Another was the Observational Medical Outcomes Partnership (OMOP), which created a common language for clinical data regardless of where, when and by whom it was collected.
Unfortunately, because of the way science is typically funded, we often neglect this sort of crucial digital infrastructure and undervalue the people who build and maintain it.
In the past year, one of the goals of the Data for Science and Health team at the Wellcome Trust has been to reimagine this particular aspect of research culture and scientific funding. Here’s what we’ve learned about the problem and how to fix it.
Every month, we see various proposals for funding from teams around the world developing digital tools. What is becoming clear is that funders aren’t asking these teams the right questions. Application forms and norms put more emphasis on publication records, research questions and potential long-term impact, which in turn encourages applicants to hide software development in conventional research proposals, or make grand claims about how the research enabled by their tools may play out.
What applicants often don’t realise is that reviewers may even end up penalising them because these speculative claims of impact come at the opportunity cost of using the limited space to provide detail on the implementation of the vital underpinning software, which would have been much more compelling.
However, it is not the impact agenda itself that’s problematic. Rather, it is the one-size-fits-all conceptualisation of success that we use. For hypothesis-driven research, a high-impact or hugely successful study conjures thoughts of field-building or paradigm-shifting discoveries. Some software tools and digital infrastructure projects may well achieve such things, but most are more about facilitating the efforts of a community of practice and supporting more efficient and effective research.
In truth, funding such projects isn’t that hard. It just requires us to create processes that reflect what we are trying to achieve. It all comes back to the questions we ask, and how we ask them. In essence, where we are currently asking about publication records, we should instead be asking for information on things such as the quality, usability and performance of tools, their usage and adoption, and resources needed to address maintenance needs.
Fortunately, philanthropic foundations such as the Chan Zuckerberg Initiative and the Sloan Foundation have pioneered ways to run open competitive processes to provide maintenance funding for software tools. These processes are a big step in the right direction: recognising that digital infrastructure requires a different approach to funding and a new shared understanding for what “good” looks like. But we cannot expect them to do this alone.
While new digital technologies might be sexy, developing thoughtful processes for funding is not. Yet the science and health research landscapes continue to rapidly change, so funders will increasingly need to engage with digital infrastructure funding. If we don’t, the cost is high: if some crucial piece of digital infrastructure either isn’t built to a sufficient standard or falls into disrepair, the result could be huge efficiency losses or large swathes of the academic literature becoming entirely unreproducible, further entrenching the reproducibility crisis.
The link from infrastructure to outcome may be complex, but the impact of getting it wrong is surprisingly linear. The Wellcome Trust is committed to changing this. We will be building sustainability plans with every new grantee developing digital tools. We will also do our part to ensure that maintenance funding for core digital research tools and infrastructure is available.
However, an enduring solution requires cooperation between government funding agencies, philanthropic foundations and the research software engineering community at large. We have to work with others who share our view of this problem to solve it – consider this an open invitation to start a conversation with us.
Bilal A. Mateen is clinical technology lead and Tariq Khokhar is head of data for science and health at the Wellcome Trust.
后记
Print headline: Foundations for progress
