AI chatbot can conduct research interviews on unprecedented scale

Research interviews conducted by artificial intelligence can help academics conduct studies on an unprecedented scale and produce more nuanced results, it has been claimed.

Two London School of Economics scholars have developed a chatbot powered by a large language model that, they say, can complete interviews with thousands of participants in a matter of hours.

Rather than having a standard set of multiple-choice and open text questions, as has typically been the case with online surveys, the chatbot takes a conversational approach, collecting interviewees’ responses and using them to generate new questions within a broad set of parameters.

Its creators, Friedrich Geiecke, an assistant professor of computational social science, and Xavier Jaravel, a professor of economics, say the tool employs best practice from academic literature – for example, encouraging participants to freely express their views, and then posing follow-up questions to ensure clarity.

They also claim that the chatbot demonstrates “cognitive empathy”, using its follow-up questions to seek to understand an interviewee’s perspective as closely as they understand it themselves.

The tool is being made freely available for other researchers to download, adapt and use.

In trials, it demonstrated some impressive results. A team of sociology PhD students from Harvard University and the LSE that was asked to assess the quality of the interviews based on transcripts rated them as being broadly comparable to interviews conducted by human experts.

And when the almost 1,000 study participants were asked to evaluate their interaction with the chatbot, the majority said they had enjoyed it and preferred this mode of interview over open text fields. Only 15 per cent of respondents said they would have preferred the interview to have been conducted by a person.

Respondents also tended to provide more detailed responses than they did with traditional open text boxes, with a 142 per cent increase in the number of words written.

These advantages were particularly evident in relation to political questions. Here, the researchers found that participants preferred interacting with the chatbot because they viewed it as a “non-judgemental entity”, allowing them “to feel more comfortable and to freely express their views”.

Could AI manage your research project?

When combined with platforms that allow scholars to recruit large survey samples, such as Prolific, the tool could allow for research on a significant scale, at a cost of about £3 to £4 per participant.

Professor Jaravel said they were “close” to incorporating voice interactions into the chatbot and were also looking at how AI could be used to analyse the responses collected, which would helpful because the method can lead to researchers having “millions” of sentences to sift through.

While the tool might not be adopted immediately by the likes of anthropologists and sociologists, who tend to conduct very lengthy interviews with relatively small samples, he said the tool might serve to bridge the gap between qualitative and quantitative research for scholars such as economists and political scientists.

“I think this will probably transform the way economists and political scientists do surveys,” Professor Jaravel told Times Higher Education. “As we add more features, we will get increasingly closer to qualitative interviewing, and that might be on the horizon, but that’s not the primary goal.

“The goal is that we can do something that’s really low cost, and not any more expensive than what political scientists and economists currently do, that’s going to have much more traction to extract information and keep the respondents engaged.”

juliette.rowsell@timeshighereducation.com