Education
Plagiarism researcher: Detectors for AI texts are not a solution
There is detector software that is supposed to detect AI use in text work – but it is not reliable. photo
© Frank Rumpenhorst/dpa
“ChatGPT, write me a bachelor’s thesis” – such requests have been possible since the new hype about AI. Universities have so far found it difficult to find out whether the work was written by a person or a machine.
An undeclared use of AI text generators such as From an expert perspective, ChatGPT in theses, for example, remains difficult to prove for the time being. “The hope that there is a simple software solution for unmasking AI texts will not be fulfilled,” said Berlin plagiarism researcher Debora Weber-Wulff (University of Technology and Economics) to the German Press Agency.
“There is a lot of supposed detector software out there, but it doesn’t do what it’s supposed to do.” Some of the manufacturers themselves have acknowledged shortcomings and pointed out the limits of reliability.
ChatGPT is a chatbot that, in response to user commands, can formulate texts at a human language level and summarize information. The publication at the end of 2022 triggered a hype about artificial intelligence (AI) – and in the higher education sector also concerns about uncritical use or even deception by students. There are now also programs that can be used to find out about texts created in this way.
Weber-Wulff worked on a study that tested 14 alleged AI detectors. Accordingly, these tools did not provide reliable results when it came to the question of whether a human or a machine wrote a text. The research team reported on this in the “International Journal for Educational Integrity” at the end of 2023. The study identifies the core problem as being that around one in five texts generated with AI were not recognized as such.