Discussion
big push to try to push "responsible" AI coding as "not *slop*"
just hit another one
> "oh no they're very anti-slop here's their CONTRIBUTING.md"
> list of ways to use the slop bot in their project
come off it with this bullshit, it fools nobody and makes you look like a smarmy collaborator
@davidgerard, hijack it: “re*slop*sible”.
It's like those companies that force new hires to write glowing reviews on Glassdoor, as a condition of passing their probation.
@davidgerard this is where a lot of people in tech are right now and the cognitive dissonance is incredible
“oh I’d never do slop because I” type lgtm in all the PRs, type the code myself on the chatbot’s instructions, only use it for tests, only use it for docs, blah blah blah
and meanwhile the project is broken in ridiculous ways and they haven’t noticed because the slopbot says it’s fine and they’ve cognitively offloaded all validation to the LLM. I haven’t met anyone who uses the slopbot and doesn’t cognitively offload to it; all of them do the glass-eyed “here’s what claude said” more and more til that’s the only thing making decisions and driving conversations in the project.
@davidgerard like no, you aren’t carefully reading thousands and thousands of lines of LLM output to make sure they’re not slop.
yes, the parts of the codebase you’re offloading to the LLM are important, and allowing any part of your project to be slop makes the whole thing slop because of how LLMs work in practice.
no, I don’t particularly want “your buddy claude” (all of these guys use this exact non-joke for some reason) to be part of the discourse. I don’t think lazily consulting the slopbot adds anything positive. I keep watching negative outcomes happen unacknowledged when you consult it. the codebase is now a disjoint mass of a hundred broken abstractions and non-abstractions because that’s what “your buddy claude” recommends.
@davidgerard and one thing I almost never see called out is the surrender of prioritization
it’s very funny that you got access to Claude Fable and filled the queue with urgent tickets for all the “urgent crucial real vulnerabilities” it says it found
great! did you do even basic due diligence on any of them to check:
- is the bug the result of previous LLM-generated code? (almost always)
- can the bug actually be encountered in practice? did you do any impact analysis at all?
- did the bug result from a flawed design? can you fix it for the whole codebase by rethinking things a little bit instead of fixing a hundred different vulnerabilities the fucking LLM tagged individually?
no? you didn’t do any of this because Claude Fable told you the sky would fall unless you took action right now? amazing.
@davidgerard the main difference between “the good claude” and “the bad claude” is the “good” one takes longer to run and its output is more confident and uses more impactful words so the human operator thinks they’ll be fired if they ignore its output
that’s pretty much it. letting it run longer lets it pull more shit into its context window but at a base level the goal’s just automated manipulation.
giving the thing more resources, more scripts to run (which are godawful by the way), and a bigger model might mean its output contains more legitimate bugs
but it also contains a fuckload more illegitimate bugs, and it’s much much more confident-sounding that they’re all critical
cause that’s how fuzzing works. we’ve had it for decades.
@zzt @davidgerard We've known for a *long* time that writing code is easier than reading code (I've heard it described as "downhill invention" vs "uphill analysis"), but somehow a swath of people are convinced that's suddenly not true.
@bryce @zzt @davidgerard The rapidity with which I’ve seen a bunch of the sloppers take to prompted “testing” (the “test” output is a bunch of .md files in full of prose and shit) tells me they’ve always just considered that part an obstacle instead of a value-contributing part of the dev cycle
Feels like it’s part of the same thing as sloppers being unable to tell whether something is good or not (be it art or code or whatever)
@davidgerard ohno, this made me look into a slop project similar to my non-slop floppy recovery software and the last commit was "hey, claude you must guarantee you don't make the code worse when refactoring"
I am just..
How can you seriously use this as a professional.
"hey, claude you must guarantee you don't make the code worse when refactoring"
I find this absolutely mind boggling. If you know how an LLM works, it's obviously nonsense, sure.
But if you buy into the marketing that ascribes intelligence to these machines: what does that even mean? How are you defining 'worse'? And why would you need to explicitly state that? And these are people who, apparently, have experience using formal language to instruct machines built around systems of logic!.
@david_chisnall @davidgerard In my understanding it's not completely nonsense?
I think the reason is you try to get it to check its output over and over (compared to how the "AI" vendors try to catch prompt injections).
I mean as a seasoned IT professional this is absolutely nonsense. It's a bit like looping over the random number generator until you have the desired number while also making sure it takes as many resources as possible.
A lot of these magic incantations nudge particular models into particular places in their latent spaces that happen to provide better outputs. Perhaps a bunch of diffs plus code-review comments had 'don't make it worse' in their feedback for early revisions and so it will nudge away from diffs that look like that.
The problem is that they're not based on any theory building and so may make things worse on a new generation of an LLM.
@JennyFluff @davidgerard why don’t they add “pinky swear” too? Or magical incantations, at this stage of irrationality they might as well.
@Nicovel0 @davidgerard yeah this actually happens. Just have a look at slop projects AGENTS.md and you see a lot of pledges to pinky promise to be a good code generator and don't break things
So what is that markdown file supposed to do?
@davidgerard I recently saw a project with a policy that stated "authors using AI are responsible for verifying their code contributions do not violate copyright."
Literally how?
@bryce@mastodon.brycedixon.dev @davidgerard@circumstances.run could one perhaps build an LLM strictly from a hand-curated set of files from their own personal projects? (asking since that'd be ridiculous, not because that'd be a good idea by any means, fuck LLMs)
@h @davidgerard It's not a very "large" language model at that point, is it?
The scale of "general" AI models is incomprehensible to humans. We're talking "ingest literally every word ever written, and generalize them" and pretending you can ask that generalized output to somehow figure out what parts were *supposed* to be ignored.
@bryce@mastodon.brycedixon.dev @davidgerard@circumstances.run that's completely fair, and part of why I'd find the notion of trying to make one that doesn't violate people's rights absurd
honestly, I personally feel uncomfortable with even copying code someone else contributed to something I worked on as a personal project between projects as an ad-hoc "library" or claiming a project I did most of the work on but someone helped reformat/etc to be more manageable as mine, so I have no idea how people who use LLMs to generate entire projects and then claim said projects as theirs mentally justify it
@davidgerard @h I feel like there are a few ways that could be gone about that could *maybe* (strong maybe) work? Like a dataset of only public domain or CC (or other non-attributive, permissive licenses) material, one that's a superset including GPL2 material leading anything it outputs to being under GPL2, etc.
The problem then is trying to prove the disclosed dataset is complete (ie: nothing else was snuck in)
@bryce @davidgerard @h It's not optimized for code and it's "only" 1B parameters, but Mimir model from Denmark was trained only on "permissible" datasets" (mostly synthetic) with 8 GPUs and it beats comparable models from hyperscalers. So maybe there's a way… https://cybernews.com/ai-news/danish-model-mimir/
@davidgerard @jnv @h Yeah, I was going to say the same. Here's the actual paper: https://arxiv.org/pdf/2608.13517
> For training HRM-Mimir v1, we curated a mix of data covering objectives, ranging from English and Danish instruction & knowledge to mathematics and agentic-style post-training data. Our
mix draws from 161 datasets with almost all of them being freely available on the HuggingFace Hub.
The whole problem is "available" doesn't mean "permissible."
@davidgerard @jnv @h Someone can build a media catalogue from "freely available sources" but if those sources are The Pirate Bay and 9Anime that doesn't really mean "it's not violating copyright anymore."
I'm especially worried about "synthetic datasets" being used to mean "we laundered data by unethically training an LLM so that we can use it's output, which no one owns, to train another one on 'uncopyrighted' data."'
Install bonfire.cafe
Get the full app experience
![[object Object]](https://media.mas.to/accounts/avatars/109/287/758/133/964/366/original/0f3099c0adfb36a0.png "[object Object]")
