Turnitin announces AI detector with ‘97 per cent accuracy’

Turnitin will make its artificial intelligence writing detector available “as early as April”, claiming it can identify ChatGPT-authored writing 97 per cent of the time.

The Silicon Valley-based company will offer the tool alongside its ubiquitous plagiarism checker in a move commentators said could heighten calls for more universities to ban students from using AI.

Companies have been scrambling to offer a tool capable of judging if a student has used ChatGPT since its hugely successful launch by the company OpenAI last year. In January, the software became one of the fastest-growing technological tools of all time when it reached 100 million active users after just two months.

Turnitin said its detector “identifies 97 per cent of ChatGPT and GPT3 authored writing, with a very low less than 1/100 false positive rate”.

“Based on how our detection technology is performing in our lab and with a significant number of test samples, we are confident that Turnitin’s AI writing detection capabilities will give educators information to help them decide how to best handle work that may have been influenced by AI writing tools,” said Annie Chechitelli, chief product officer at Turnitin.

“Equally important to our confidence in the technology is making the information usable and helpful and in a format that educators can use.”

Turnitin has faced competition from start-ups to develop such a tool, but academic integrity experts have warned that many are highly inaccurate, and those students who are falsely accused of using AI on tests will struggle to prove otherwise.

Ms Chechitelli said that the detector had been “deliberately” trained on a large dataset of academic writing because it was “essential that our detector and any others limit false positives that may impact student engagement or motivation”.

Experts have said that the emergence of smarter, more accessible AI tools such as ChatGPT should lead educators to reassess their assessment methods, including utilising such tools in learning where possible.

Few have openly announced plans to ban the use of the technology outright but Mike Sharples, emeritus professor at the Open University’s Institute of Educational Technology, said he felt that the emergence of detectors would fuel more calls for AI essay bans.

Writing on Twitter, Professor Sharples said the tool was needed by Turnitin “to survive” but its claims about the false positive rate had not been independently verified. “Can it be defeated with minor text tweaks? Or by running text through another AI to rephrase?” he asked.

Eric Wang, Turnitin’s vice-president for AI, said the company was “very happy to see productive conversations taking place across the education community about academic integrity and tools to ensure the authenticity of authorship”.

“Teachers should use Turnitin’s detector to have fulsome conversations with students about this technology,” he said. “We will continue to develop and refine Turnitin’s detector and are confident that as AI writing evolves, our technology will keep pace with new developments and capabilities.”

tom.williams@timeshighereducation.com