OpenAI has announced (opens in new tab) a new tool that it says can tell the difference between text written by a human and that of an AI writer – some of the time.
The Microsoft-backed company says the new classifier, as it is called, has been developed to combat the malicious use of AI content generators, such as its very own and very popular ChatGPT, in “running automated misinformation campaigns (opens in new tab), … academic dishonesty, and positioning an AI chatbot as a human.”
So far, it claims that the classifier has a success rate of 26% in identifying AI-generated content, correctly labelling it as being ‘likely AI-written’, and a 9% false positive rate in mislabeling the work of humans as being artificially created.
Spot the difference
OpenAI notes that the classifier performs better the longer the text, and that compared to previous versions, the newer version is “significantly more reliable” at detecting autogenerated text from more recent AI tools.
The classifier is now publicly available, and OpenAI will use the feedback it gets to determine the usefulness of it and to help improve further developments of AI detection tools going forward.
OpenAI is keen to point out that it has its limitations and should not be relied upon as a “primary decision-making tool”, a sentiment shared by most involved in all fields of AI.
As mentioned, the length of the text is important for the classifier’s success, with OpenAI stating that it is “very unreliable” on pieces with less than a thousand characters.
Even longer texts can be incorrectly identified, and human written content can be “incorrectly but confidently labeled as AI-written”. Also, it performs worse on text in written in non-English languages as well as computer code.
Predictable text where the content can only realistically be written one way is also unable to be labelled reliably, such as a list of the first one thousand prime numbers, to give OpenAI’s example.
What’s more, OpenAI points out that AI text can be edited to fool the classifier, and although the classifier can be updated and learn from being tricked like this, interestingly, the company says it is “unclear whether detection has an advantage in the long-term.”
Text that is also very different from that which it has been trained on can cause the classifier issues too, with it “sometimes [being] extremely confident in a wrong prediction.”
On this training data, OpenAI says that it used pairs of written text on the same topic, one AI-produced and the other it believed to be written by a human – some gathered from human responses to prompts used to train InstructGPT, the AI model from the company that is primarily used by researchers and developers.
The development of the classifier comes amid numerous concerns and debates surrounding the use of AI chatbots, such as OpenAI’s own ChatGPT, in academic institutions such as high schools and universities.
Accusations of cheating are mounting, as students are using the chatbot to write their assignments for them. Essay submission platform Turnitin has even developed its own AI-writing detection system (opens in new tab) in response.
OpenAI acknowledges this fact, and has even produced its own set of guidelines for educators (opens in new tab) to understand the uses and limitations of ChatGPT. It hopes its new classifier will not only be of benefit to this institution, but also “journalists, mis/dis-information researchers, and other groups.”
The company wants to engage with educators to hear about their experiences with ChatGPT in the classroom, and they can use this form (opens in new tab) to submit feedback to OpenAI.
AI writing tools have been causing a stir elsewhere too. Tech site CNET recently came under fire for using an AI tool to write articles (opens in new tab)as part of an experiment, but was accused of failing to distinguish theses articles from those written by actual people. Such articles were also found to contain some basic factual errors.
