NIH ban on AI in grant review ‘lacks technological understanding’

Scientists are challenging a major funder’s automatic ban on using artificial intelligence in its grant review process by claiming that it could also apply to googling research topics.

The US National Institutes of Health ordered the ban on the grounds that all grant submissions must remain confidential, and that AI services are unacceptable tools because they store and reuse whatever information is submitted to them.

“No guarantee exists explaining where AI tools send, save, view, or use grant application, contract proposal, or critique data at any time,” the NIH said in a recent advisory. “Thus, using them absolutely violates our peer review confidentiality expectations.”

That position, however, has been attracting widespread criticism from the academic community, for making no clear distinction about reviewers submitting to online services even minor queries related to topics discussed in grant applications, and for failing to recognise that even common search tools such as Google also store and reuse information entered into online queries.

“Every query you make to Google goes into a database,” said one of the critics, Jared Roach, a senior research scientist at the Institute for Systems Biology in Seattle.

The NIH needs to be a lot clearer about what behaviours it is actually trying to ban, Dr Roach said. “Obviously it can’t get to zero risk, but what level of risk is OK?” he said.

The NIH is the nation’s largest provider of basic research funding, with most of its $48 billion (£38 billion) annual budget distributed through tens of thousands of grants. Those grant awards are typically decided through a process in which about a dozen volunteer scholars are assembled on a proposed area of research, and they jointly offer the NIH advice on the novelty and value of a set of grant applications in their areas of expertise.

It’s typical for such experts to assess those applications by using the internet to conduct background research on the topics, Dr Roach said. As a hypothetical example, he suggested that a grant application proposing that cayenne pepper can cure Alzheimer's disease would lead many reviewers to make a Google search asking that question. Technically, however, that search would be publicly revealing the confidential topic of the application, he noted.

The federal government’s next-biggest research funding agency, the National Science Foundation, with an $11 billion annual budget, has not yet taken action on the matter. The NSF “has a working group evaluating the use of generative AI by reviewers and PIs and is expected to release guidance in the coming months,” an agency spokesperson said.

NIH officials, asked about the criticisms from Dr Roach and dozens of others, said that standard web searches “differ from using generative AI in that web and database searches don’t use inputs for potential fodder for future outputs.”

Mohammad Hosseini, a postdoctoral scholar at Northwestern University whose work involves augmented intelligence in medicine, backed up this view.

He said that queries to search engines such as Google typically involve a few words, while AI systems invite the expectation of a reviewer entering the entire text of a grant proposal.

“So the NIH is spot on in insisting that there is meaningful difference between AI systems and Google searches,” said Dr Hosseini, associate editor at the journal Accountability in Research.

“Especially in regard to grants,” Dr Hosseini said, “it is important to note that the private sector and tech giants in particular love to get their hands on new ideas as early as possible – even before they are funded – and so the NIH has a responsibility to protect researchers and their ideas from being scooped.”

paul.basken@timeshighereducation.com