OpenAI sitting on tool to watermark AI-generated content
OpenAI, the creator of ChatGPT, has developed a text watermarking tool that could help it comply with the Artificial Intelligence (AI) Act, but the company has yet to release it, reportedly because it fears losing users.
The EU AI Act requires providers of AI systems, like ChatGPT, that generate synthetic audio, image, video, or text content to mark the system’s outputs as detectable as artificially generated or manipulated.
The Act entered into force on 1 August, but this requirement applies from 2 August 2026.
On Sunday (4 August), the Wall Street Journal (WSJ) quoted sources who said OpenAI has been ready to deploy a text watermarking tool for a year but is hesitant to do so because it fears losing users.
In a blog post update on Sunday, OpenAI confirmed that it has developed a “highly accurate” text watermarking tool that they “continue to consider as we research alternatives”.
However, the company said they are currently weighing the risks of such a tool, for example “our research suggests the text watermarking method has the potential to disproportionately impact some groups,” such as non-native English speakers.
They also doubt its efficacy against globalised tampering, such as translation systems and rewording with another generative model, “making it trivial to circumvention by bad actors”.
According to the WSJ, OpenAI has been debating and conducting surveys since November 2022 to decide whether to deploy the watermarking tool.
In a survey, nearly 30% of loyal ChatGPT users said they would use the chatbot less if OpenAI deployed watermarks and a rival did not. However, another survey conducted by OpenAI found that, in the abstract, four out of five people worldwide supported the idea of an AI detection tool.
Watermarking as a tool
The tool would digitally stamp any content created by ChatGPT, making it harder for people to misuse AI-generated content or use it to cheat. The misuse of such content has caused big problems with employers, teachers, and professors, who have called for ways to clamp down on the issue.
The WSJ reported that the watermarking tool does not leave visible traces in written text, but when that text is checked by an AI detection tool, it is flagged as AI-generated content.
Current AI detection tools are somewhat unreliable and can give mixed results, making it harder for teachers and professionals to flag AI-generated content.
The AI Act
Watermarking is mentioned in the draft AI Pact, a set of voluntary commitments companies can sign up for to prepare for compliance with the AI Act. The Commission hopes to have a pledge-signing event in September where companies publicly commit to the Pact.
“The organisations may commit to […] to the extent feasible, design generative AI systems so that AI-generated content is marked through technical solutions, such as watermarks and metadata identifiers,” the draft pact reads.
As recently as last week, OpenAI publicly stated its commitment to comply with the EU’s AI Act, which requires providers of generative AI models like ChatGPT to “ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated.”
But as for specifics on how the watermarking tool can help OpenAI with AI Act compliance, a company spokesperson pointed Euractiv to the 4 August blog post.
Read more similar news:
Comments:
comments powered by Disqus
