OpenAI shelves chatbot text detector over accuracy concerns

ChatGPT’s creator has quietly taken its artificial intelligence-generated text detection tool offline owing to concerns about its “low rate of accuracy”.

OpenAI’s “classifier” launched just six months ago – one of a number of new products that claimed to be able to tell a user when ChatGPT or other chatbots had been used to write a piece of text.

Such tools have been hugely embraced by universities since their launch, with Turnitin recently announcing that its version has been used to check 65 million essays in three months.

When it was first announced, OpenAI said in a blog that its classifier could be useful when checking for “academic dishonesty” as it was forced to respond to a wave of concern that ChatGPT could be used to help students cheat on assignments.

Campus resource: AI text detectors: a stairway to heaven or hell?

The company has always admitted its tool was not fully reliable, stating from the start that it could only confidently identify 26 per cent of AI written text. 

It had hoped its early launch would enable it to gather feedback to improve the tool but in an update to the original blog posted with little fanfare, OpenAI said its tool was “no longer available due to its low rate of accuracy”.

“We are working to incorporate feedback and are currently researching more effective provenance techniques for text and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated,” the post says.

Academics have warned that the rush to use AI detection software comes with a high risk that students will be accused of misconduct with little means of defending themselves.

Unlike the plagiarism checkers offered by Turnitin and others, there is no source material that can be used to confirm whether a student has used AI.

Coupled with concerns about false positives and claims about accuracy not being independently verified, many have cautioned universities against using such AI detection – a warning that has mostly been ignored, with Turnitin saying that 98 per cent of its customers have adopted its detector.

Toby Walsh, Laureate fellow and Scientia professor of artificial intelligence at UNSW Sydney, tweeted: “If the company that builds these chatbots give up on detecting chatbots (with all their inside information on weights, guardrails…) then there’s probably no hope for outsiders like Turnitin to detect real v fake text reliably.”

tom.williams@timeshighereducation.com