The example above that paragraph shows how words might be chosen using fair dice. Watermarking works by using weighted (unfair) dice. It's saying that you can hide watermarking by weighted the dice, i.e. "lean on how the dice land."
If you then know how the particular dice weights, you can later use that to confirm watermarked text.
Is it actually biasing specific words over the course of the text? Or something less direct in terms of how any given next token is predicted? Seems like biasing words would be too big of a tell.
Comments
"Whoever gets to lean on how the dice land"
I'm a native English speaker and I have no idea what this means.
The example above that paragraph shows how words might be chosen using fair dice. Watermarking works by using weighted (unfair) dice. It's saying that you can hide watermarking by weighted the dice, i.e. "lean on how the dice land."
If you then know how the particular dice weights, you can later use that to confirm watermarked text.
I find these explanations that drag in things like "dice" incomprehensible. What dice?
What the article is trying to say:
"For AI-generated text, watermarking usually works by secretly biasing which words/tokens the model prefers while it writes."
Simple. Understandable. No "dice" or "leaning" or other convoluted explanations
Is it actually biasing specific words over the course of the text? Or something less direct in terms of how any given next token is predicted? Seems like biasing words would be too big of a tell.
“Provenance” seems to be the flavor of the day.