Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.
Until a formal analysis is presented and an experimental study is published, which covers the most important influencing factors, the reliability of this concept is limited.
Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.
No, it is actually statistically impossible for a human to replicate this on sufficiently long runs of text.
This is not about replicating writing like a model. This is basically about guessing which words to pick from the list of suitable words based on a rule that you don’t know (because the key is secret).
To reduce this to the simplest possible example, imagine you are writing a “text” from just two letters: “a” and “b”. Let’s say for convenience that the text is supposed to be random. So the text would look something like “ababaaabbababbbabababaabbbabaababbaaabbabaabbaaaaabaaabbbaabaabababbabbbbbbbbabbabaabbbbbbbaabbabaab”
(generated with '''.join(random.choice(['a', 'b']) foriinrange(0, 50)))
The watermarking works as follows: the model owner holds a key, and then uses that key to influence the random choices between “a” and “b” somehow, in a context-dependent way. The actual algorithm is quite complicated, but for simplicity let’s just say we have a secret pattern which biases the random choice towards it. In order to see the exaggerated results, let’s say the secret key is “aaaabbbb” (of course this is a bad secret key, once again just an example), and that the bias is strong (let’s say 80%). So this would mean that the first four letters in our text are more likely to be “a”, the next four letters are more likely to be “b”, then the next four letters are more likely to be “a”, and so on.
Then the text would look something like “aaaabaabaabaabbbabaaaabbaaaababbbaaaabbbabaabbbbaaabaabbaaaaabbbabababbaaaaabbbbaaabbbbbaaaaababaaba”.
(generated with ''.join(random.choice(['a', 'b'] + ([key[i % len(key)]] * 3)) for i in range(0, 100)))
You can see visually that the secret key has affected the text. Of course in this example even if you didn’t know the secret key you could probably figure it out, in reality the algorithm is way more complicated than that, relying on cryptography, so you wouldn’t be able to know the secret key or see that the string has been biased at all.
If the text is long enough, and you know the secret key, you can guarantee that the text was generated with it. In our examples, the letters in the text match our key 77% of the time. The probability of an actual random algorithm generating a text like that is already very low, despite the base entropy being only 100 bits. If my math is correct, for our example the p-value is 2.7 * 10⁻⁸, or about 0.00000027%. I would bet a hungy that the text was generated by our watermarking algorithm, with odds like these!
Of course we did exaggerate the bias and our base algorithm was random. In reality the bias is smaller, the algorithm for determining the likelihoods of possible next tokens is very complicated (it’s the LLM itself), and the algorithm for determining which token to bias is also way more complicated (involving cryptography and real secret keys). That said, hopefully it should help you understand why, for sufficiently long texts, this fingerprinting is just not possible to be replicated by humans.
I do not have the time to work through every part of the example, but imo the main claim is still overstated. Showing that a result would be extremely unlikely under a particular null model is not the same as showing that it is “statistically impossible” for a human to produce. It also does not give a guarantee how the text was written. A tiny p-value is still a probability under assumptions and not a proof of provenance.
Furthermore, a human does not even need to know the secret key. By pure chance a human written text can display an unusually high alignment with the detector’s secret partitioning.
The published watermark work, which is also cited by the article, appears to be much more careful about this (based on a quick skim). It reports false positive/negative rates, thresholds, length requirements, and more. Those can be very strong results, provided the assumed conditions apply. They do not turn a detector into an infallible test. Moreover longer text only helps if the assumptions and watermark signal actually remain intact, which can fail in general.
In such controlled settings, sure, I do not have much issues there. But the claims of “a human cannot replicate this” or “we can guarantee the text was generated with the watermark” are much stronger than the statistics, and especially the cited literature, actually appear to support.
Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.
Until a formal analysis is presented and an experimental study is published, which covers the most important influencing factors, the reliability of this concept is limited.
No, it is actually statistically impossible for a human to replicate this on sufficiently long runs of text.
This is not about replicating writing like a model. This is basically about guessing which words to pick from the list of suitable words based on a rule that you don’t know (because the key is secret).
To reduce this to the simplest possible example, imagine you are writing a “text” from just two letters: “a” and “b”. Let’s say for convenience that the text is supposed to be random. So the text would look something like “ababaaabbababbbabababaabbbabaababbaaabbabaabbaaaaabaaabbbaabaabababbabbbbbbbbabbabaabbbbbbbaabbabaab”
(generated with
'''.join(random.choice(['a', 'b']) for i in range(0, 50)))The watermarking works as follows: the model owner holds a key, and then uses that key to influence the random choices between “a” and “b” somehow, in a context-dependent way. The actual algorithm is quite complicated, but for simplicity let’s just say we have a secret pattern which biases the random choice towards it. In order to see the exaggerated results, let’s say the secret key is “aaaabbbb” (of course this is a bad secret key, once again just an example), and that the bias is strong (let’s say 80%). So this would mean that the first four letters in our text are more likely to be “a”, the next four letters are more likely to be “b”, then the next four letters are more likely to be “a”, and so on.
Then the text would look something like “aaaabaabaabaabbbabaaaabbaaaababbbaaaabbbabaabbbbaaabaabbaaaaabbbabababbaaaaabbbbaaabbbbbaaaaababaaba”.
(generated with
''.join(random.choice(['a', 'b'] + ([key[i % len(key)]] * 3)) for i in range(0, 100)))You can see visually that the secret key has affected the text. Of course in this example even if you didn’t know the secret key you could probably figure it out, in reality the algorithm is way more complicated than that, relying on cryptography, so you wouldn’t be able to know the secret key or see that the string has been biased at all.
If the text is long enough, and you know the secret key, you can guarantee that the text was generated with it. In our examples, the letters in the text match our key 77% of the time. The probability of an actual random algorithm generating a text like that is already very low, despite the base entropy being only 100 bits. If my math is correct, for our example the p-value is 2.7 * 10⁻⁸, or about 0.00000027%. I would bet a hungy that the text was generated by our watermarking algorithm, with odds like these!
Of course we did exaggerate the bias and our base algorithm was random. In reality the bias is smaller, the algorithm for determining the likelihoods of possible next tokens is very complicated (it’s the LLM itself), and the algorithm for determining which token to bias is also way more complicated (involving cryptography and real secret keys). That said, hopefully it should help you understand why, for sufficiently long texts, this fingerprinting is just not possible to be replicated by humans.
I do not have the time to work through every part of the example, but imo the main claim is still overstated. Showing that a result would be extremely unlikely under a particular null model is not the same as showing that it is “statistically impossible” for a human to produce. It also does not give a guarantee how the text was written. A tiny p-value is still a probability under assumptions and not a proof of provenance.
Furthermore, a human does not even need to know the secret key. By pure chance a human written text can display an unusually high alignment with the detector’s secret partitioning.
The published watermark work, which is also cited by the article, appears to be much more careful about this (based on a quick skim). It reports false positive/negative rates, thresholds, length requirements, and more. Those can be very strong results, provided the assumed conditions apply. They do not turn a detector into an infallible test. Moreover longer text only helps if the assumptions and watermark signal actually remain intact, which can fail in general.
In such controlled settings, sure, I do not have much issues there. But the claims of “a human cannot replicate this” or “we can guarantee the text was generated with the watermark” are much stronger than the statistics, and especially the cited literature, actually appear to support.