Post · bonfire.cafe

mx alex tax1a - 2020 (5)

every once in a while someone asks us "why can't you just review the slop code as if it were human written" and let me tell you that we have tried it, and you cannot, actually, do this, for the simple reasons that humans do not write code like that

human-written code follows a progression and exhibits a model of the problem domain. the model may be wrong, the progression may be wrong, but line 1 and line 10 are related to each other.

LLM code has none of that. it looks like code, but does not have the structure of it. there is no progression of thought, there is no model of the problem domain, and lines 1 and lines 10 bear no relationship to each other outside of statistical likelihood.

the question thus asks, "why can't you just eat the dogshit as if it were a burger cooked by a human"

David Chisnall (*Now with 50% more sarcasm!*)

@david_chisnall@infosec.exchange · 3 hours ago

@atax1a To add to that:

Unless someone is actively trying to mount a supply-chain attack, they are generally trying to make their code understandable. Comments and code both reflect what they are thinking. They may not quite reflect each other, but one allows you to build a mental model of intent that you can compare against the implementation.

In contrast, LLM-generated code is designed to be statistically probable. Each statement is the statement that is most likely to follow the previous one. Yet the reason for writing new code is that the requirements are different from all of the existing code in the world in some significant way. Reviewing statistically plausible code is very hard because it is code that is created specifically to look like the code you wanted. By coincidence, it sometimes is the code you wanted. But figuring that out requires working out exactly what code you wanted, at which point it’s less effort to just write it down than play a complex game of spot the difference.

✨pencilears✨

@pencilears@mastodon.eternalaugust.com · 14 hours ago

@atax1a this is also how I feel about looking at ai art.

Like, I'm trying to grasp an intention, a thought, a human connection, and there's nothing. it's like reaching out my hand and grabbing a wad of slime instead of another person's hand.

Federation Bot

@Federation_Bot · 14 hours ago

@atax1a@infosec.exchange I've noticed that there are indicators of the prompting process but they aren't the same as indicators of a thought process. Let's say someone guided the prompt two or three times because it wasn't working quite right. Often there will be two or three new variables or interfaces because of those passes. There's not actually two or three new ideas to represent, it just kept adding to get closer to solving the problem. It's really hard to refactor code like this later because noise that looks important but isn't important is so hard to be sure you can remove. Or even know how to remove to fit in with the bigger concept (cuz there isn't one).

Matthew Exon

@mat@friendica.exon.name · 14 hours ago

@atax1a I cracked when reviewing a new unit test, reading it for the 5th time, and finally realising that none of the comments or variable names or name of the test were at all related to what it was testing. Humans make mistakes. A PR that layers lies on top of lies to obfuscate what it's trying to achieve, that's malice. And when that malice comes with the signature of a developer I've known and trusted for years... I'm out. I'm done. I can't build my career on taking such an adversarial relationship against my friends.