EU's AI rules put pressure on OpenAI TextGrain

Summary

The EU's new AI Act mandates machine-readable watermarks in AI-generated text. However, tests on OpenAI's textGrain system show that detection rates drop sharply for passages under 150 words and when moderate synonym substitutions are made. Changing just 25% of the words reduces the detection rate to 17%.

Why it matters

There is no clear legal benchmark defining what constitutes 'effective' watermarking. This leaves companies facing uncertain compliance costs and creates a regulatory gap that could undermine the EU's transparency goals. As models become more complex and users routinely edit AI outputs, the practical reliability of watermarking as a source-identification tool is seriously questioned. This could weaken the entire framework of Article 50.

Highlights

• Detection rates are approximately 95% for 300-word texts but drop to about 80% for 150-word texts. A mere 10% synonym substitution reduces the rate from 92% to 66%.
• textGrain treats entropy as a budgeted 'resource' to balance detectability with output quality, offering developers adjustable control.
• OpenAI positions watermarking not as a standalone solution, but as one layer working alongside metadata credentials and SynthID.

Critical take

The article relies heavily on OpenAI's own test data and the framework of a single co-author. OpenAI declined to comment on the compliance issue. This fails to provide independent verification of whether textGrain truly meets the EU's threshold for being 'effective, interactive, robust, and reliable,' a standard that has been deliberately left ambiguous.