I said I *didn't* think Taylor Lorenz was an industry shill but... wow... now I'm starting to rethink that.
You know I might be wrong, but what is wrong with watermarks?
14
Replies
4
Boosts
@futurebird there were doubts about her integrity for a long time, shill is the least surprising pivot.
Let me outline my understanding of "LLM Watermarking"
It is possible to embed special characters and patterns in the output of LLMs that would make text generated by these systems easier to reliably detect. This is mainly being done so that when LLMs scrape the web for new information they can avoid ingesting machine generated content.**
When you train an LLM on machine generated content it may lead to "model collapse."
**see next post for correction
The companies that run LLMs can also use this for PR to calm concerns from the public about the proliferation of such content.
From a CS perspective I can't think of any way to have a watermark that couldn't be easily defeated through additional processing. LLM dependent people currently take pains to remove em dashes so this new wrinkle has some of them in a panic.
But, why not just be honest and say you used and LLM? Why so bashful?
But, why not just be honest and say you used and LLM? Why so bashful?
In my experience, it's because they know most people absolutely hate having slop sprayed at their face, so they'll go to great lengths to disguise it in an attempt to "get one over" on them.
It resembles the same type of shit with people who "test" someone else's allergies by sneaking stuff into their food: They're trying to "catch them in a lie" with the false assumption that they'll be fine with it if they don't know it's there.
And even just that type of behavior is extremely concerning for me, even if it doesn't involve allergens that can Actually Kill People. It shows a clear lack of respect for others, and deserves a swift football kick to the unmentionables and immediate expulsion.
Isn't that amazing? Even people who really love boosting AI feel hurt when someone feeds them AI generated content.
It's embarrassing to admit to liking or not noticing AI generated content. There was a music video I really enjoyed a few months back and I think it might have AI generated music or AI assisted animation. The creator hasn't been very transparent and everyone who liked it is kind of worried and unhappy.
When the same account posted something new I ignored it.
This is the song/creator
I even thought about taking this post down because I don't really know. But, I decided not to and to simply wait to see what was really going on.
I still don't know. But, due to generated content there is a cloud hanging over many creative ventures, fairly or not.
@futurebird @becomethewaifu It’s funny, there’s so much accusation of AI around that video and to me it has none of the hallmarks and in fact looks far more likely to be human created (there’s several animated sequences that are exact repeats, which is unlike AI) but I think “simply styled animation” reads as AI.
There’s a comic on here that keeps getting accusations of being AI generated when it’s likely the artist’s work has been used as primary training material, and he’s really unhappy about it.
@MichaelTBacon @becomethewaifu
I don't think the animation is generated, it's the music that I have questions about. But I've done animation, and I've never really done any music. So I feel more out of my depth and there is something about the vocals and the very good but predictable use of breaks that IDK...
No that's the main point. I Don't Know.
Up front, I really don't know either.
I have done a good bit more music than animation, and there's certainly nothing in that that couldn't have been done with a garden variety synth/electric piano setup. The only thing really tricky is that the drum and the bass are really tight, which really propels the song. (The lyrics are clever and hilarious and well timed as well but it's the rhythm section that makes it so re-listenable.)
I guess the thing that's so annoying in this instance is that all of it *could* have been done by a dude who was pretty good with basic animation skills, a synth setup, and a friend who had been a bass player in a local scene for a couple decades. But maybe it could have been done by AI?
At this point, I don't want AI watermarks so much as I want composable digital signatures for real work. Like, a signature from the mixer that this feed came in off an analog feed.
Just as you say. I Don't Know.
@MichaelTBacon @futurebird @becomethewaifu@tech.lgbt I’ve done a fair bit of animation and this reads as human-made imo. There are a few spots where the underlying geometry peeks from behind the curtain rather than being a mere sequence of stills.
From a CS perspective I can't think of any way to have a watermark that couldn't be easily defeated through additional processing
The way that the current ones work is that they tweak the probabilities to generate specific patterns in the output. Where there's an equal probability of two words following another in the raw weights, they'll tune it so that there's a higher probability of one than the other.
If you know the weights and know the biassing factor, you can look at each word pair and see what the probability would be of the model generating that.
This means that the watermark is smeared all over the output. And it's not a binary thing though, each pair of word contributes something to the probability of matching the watermark and looking at the whole thing will give you a probability at the end.
Changing every other word should give you close to a 0% of matching the watermark but, at that point, why bother using the LLM at all? If you're going to change half of the words, you may as well just write them yourself. And that's a problem for people who want to share low-effort slop and pretend to be creative.
![Chris [list of emoji]](https://nfts.freeradical.zone/accounts/avatars/109/127/168/397/951/576/original/e8a0b6da977923a8.jpeg "Chris [list of emoji]")
I wonder if there's an efficient way to procedurally reword stuff to defeat the watermark. (The goal here is to hasten model collapse.)
@david_chisnall @futurebird this also means that you have know the weights and know the biassing factor to be able read that watermark reliably.
And I am going to bet that Anthropic is not going to release those.
In other words, Anthropic will be the only company offering the slop generator, and the service to tell that something got generated by it.
That's real power right there. Would they not "tweak" the results if it was useful for them, say, to create a plagiarism scandal around someone?
@david_chisnall @futurebird @404mediaco has a pretty good article OpEd on it as well.
