Post · bonfire.cafe

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 6 hours ago

i looked up the rust LLM policy. it is in fact the first policy that doesn't give LLMs a free pass to shit all over the repo and in fact is the first example of using LLMs as an opportunity to develop more robust general contribution policies that i've ever seen. it aligns strongly with my policy proposal https://circumstances.run/@hipsterelectron/116582345110286678 and directly mitigates several ethical issues. i would never accept it nor propose it myself but for a programming language developed by google and microsoft this is worthy of applause

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 11 hours ago

curious points about LLM bans:

it's important for the policy to specify the basis of the ban (on copyright alone, or copyright+ethical grounds) so that it can be applied to other technologies. this in fact sets a precedent, and you are acting as judges in your project's courtroom

15

1

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 5 hours ago

it's nice that they mention the threshold of originality, even if they dismiss the copyright claim more generally. that's not nothing—it asserts that copyright law exists in policy.
bug discovery: honestly this is the only point i find annoying. a bug report requires a repro and a coherent description and if you've provided those i don't care if you decide to credit an LLM with it. in general requiring credit for an LLM is just marketing the product and can't be verified anyway.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 5 hours ago

the use of LLMs as review bots is brought up again. this is one of two genuinely thoughtful and meaningful processes to address how LLMs make life hell for maintainers. highlights:

LLMs must have their own account, and must be blockable: this means people who want to avoid the machine incorrectly negging their highly specialized compiler functionality can avoid that noise. opt-in would obviously be preferable, but github doesn't support that, and since the LLM accounts must be approved by the rust project, this won't be a whack-a-mole
LLMs cannot block a merge: this is a necessary corollary of making them blockable, but it cements the extremely important requirement that LLMs cannot stand in for human judgement, and mirrors the rejection of LLM acceptance criteria described earlier.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 5 hours ago

LLM code review is only ever a complete fucking waste of time, and often generates false statements about what the code does. why on earth any engineer would ever want to have a machine generate accusations impugning their own code is beyond me.

since rust decided they want that for some reason, this policy ensures people who do real work and aren't useless shills can avoid being penalized because someone else likes to talk to the sycophancy machine. that directly addresses another harm from LLMs (not an ethical one, but still a harm)

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 5 hours ago

so then there's LLM code changes, which are correctly identified as VERY FUCKING DIFFERENT than other uses (openslopware refuses to make this distinction at all, conflating "some guy went on a podcast" with "has accepted LLM commits"). i obviously hate that rustc will allow people to use my code in the compiler without crediting me, but i still think these conditions are worth lauding.

note again that LLM code changes are described as an experiment as above. the requirements are concise:

~20 more replies

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

"Solicited" means that a reviewer has communicated ahead of time that they are willing to review an LLM-created PR.
ℹ️ New contributors cannot use an LLM unless they first talk with a reviewer. This must be the same reviewer who will be assigned to the PR.

this is by far the most significant item to me. code review takes a lot of effort. a policy of blanket LLM acceptance means maintainers generally have no option but to nod and smile and accept unreviewed code, which as i noted in my policy proposal https://circumstances.run/@hipsterelectron/116582482892207200 is a great way to introduce vulnerabilities e.g. into the linux kernel.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

instead, this puts individual maintainers on the hook, in an opt-in fashion!!!! for the duration of the experiment, the maintainers who want to allow rust to incorporate code no one understands or cares enough to write are specifically tasked with not fucking things up for everyone else who built the language.

this is a model that can be employed outside of LLMs—in fact, it's analogous to how feature branches work. until a complex long-term feature has been proven to work, a contributor is on the hook for rebasing their changes against main/develop and figuring out how to resolve conflicts, instead of forcing everyone else to fix something they half-assed

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

there's a subtlety here, though: if an LLM code change touches code i own, and i haven't opted in to slop reviews, then either it goes in without my consent, or i have to review the slop anyway. i can't see a way to solve this, and i personally consider it to be further evidence that LLMs shouldn't be allowed.

i would mention this in the issue (maybe i'll try), but rust governance discussions scare me. this is a notable miss, but i don't see it as intentionally sneaky, particularly since it results from the self-selection approach that lets LLM shills deal with their own shit instead of subsidizing their religion through state policy

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

this second condition identifies areas of the code as high-risk and off-limits entirely:

"Non-critical" means that it is extremely unlikely for the PR to cause a soundness regression.
ℹ️ Examples:
Changes to internal tooling like tidy, x setup, and linkchecker are probably ok.
Changes that have a strong soundness impact, like the trait system, MIR building, or the query system are probably not ok.

identifying sections of a codebase as high-risk and subject to stronger review requirements is imho incredibly significant. this is the kind of precedent i would like to see in actual legislation, and indeed the EU "AI Act" was specifically lauded for establishing an analogous hierarchy of risk that subjected certain applications to greater scrutiny.

(unfortunately, the same act also completely discarded scrutiny and safety requirements if the "AI" was used for border control or counter"terrorism". so it creates a regulatory structure as a sham and refuses to regulate the most fascist shit, which is classic EU behavior. but they created the sham because it would have been actually good to have)

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

a conflicting take on this though is that it also indicates which parts the rust compiler developers personally consider important, and which features they don't really respect. in particular, nobody ever considers repo tools or the build system to be a high-assurance system, even though the build system is what "reflections on trusting trust" was about, and where vulnerabilities can be inserted.

this is kind of my personal axe to grind, and in fact the user who proposed this policy is actually the only person who works on improving the rustc bootstrap build system (and if no one cares about the build, they won't contribute broken changes to it). so i'm not implying sneaky behavior or wrongdoing here. it's just also worth noting that your designation of components which are critical to system correctness tend to refer to the parts you personally like

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

the "high-quality" and "well-tested" requirements are the first subjective ones we encounter. normally, subjective statements in policies can be a point of concern, particularly if there's no escalation or adjudication process defined. in fact, "real law" has answers for subjectivity: in US copyright, the subjective determination of "fair use" must satisfy a multi-factor balancing test, which is argued by reference to precedent.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

however, in the context of the experiment, we can interpret these subjective requirements as the measured outcomes of the experiment, resulting in failure (this cannot be achieved) or success (and the knowledge gained can be used to specify concrete policy).

there are several real issues with this, especially revolving around how to keep the LLM shills continuing to ensure quality and test coverage after the experiment. if this experiment determines a successful result, it's worth considering whether to establish periodic (quarterly?) retros to reconvene and review these requirements and results.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 4 hours ago

unfortunately, my beautiful vision of scientific experimentation is shattered when the policy defines what it's actually looking for:

Instead, it's to collect information about whether this experiment is working: Are people doing interesting and useful things with LLMs? Are they learning? Are they making repeat contributions?

i don't care about any of these things, i care about whether it introduces broken code. these questions are the very definition of what corporations want from open source contributions: the appearance of a vibrant community, so they can tell shareholders how much free labor they're getting. and it indicates the LLM contributions will probably become a permanent fixture, because those subjective questions are not really disprovable (except repeat contributions, but LLM contributions are easy to repeat)

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 3 hours ago

but it wasn't a surprise that the rust compiler was largely composed of microsoft and google engineers who support the use of their employer's products and seek to encode it in policy. it was a surprise that the policy that covers their work would choose to limit the use of their employer's products, and to thoughtfully ensure maintainers who don't want it aren't forced to pretend the machine has a brain.

i think these are worth studying, and particularly i think these mitigate some of the harms of LLMs, including some ethical issues. as a result, i think this is worth a tactical show of support.

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 3 hours ago

the policy goes on to disallow "playing detective", or accusing anyone of LLM usage. as in my anti-LLM policy proposal https://circumstances.run/@hipsterelectron/116582531546661338, i think this is something every LLM policy should incorporate, because that shit is extremely toxic. accusing someone of "secretly" using an LLM is something that LLM shills do. this is why openslopware may or may not be an industry op. who knows!

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 3 hours ago

it concludes with the admonition to "be honest" about usage. again, in my repos i don't want to hear a damn thing about LLMs at all, and if you use one to generate a contribution you're not going to be able to satisfy my review requirements without rewriting it by hand. but if you're going to institute procedures like this which ensure only LLM shills review LLM shit, you necessarily need users to state that so you can apply those policies