Post · bonfire.cafe

@mttaggart@infosec.exchange · 13 hours ago

So I guess for several months now KeePassXC has been using generative AI in their development pipeline. This includes a policy regarding disclosure of generative AI usage for PRs, which is prudent. It also, however, appears to involve their use of Copilot by one of the primary maintainers.

Interestingly, the policy for contributions was added at the same time as the necessary files for Copilot to work on the repo autonomously. And while independent contributors are required to disclose their AI use, it would appear KPXC does not feel the need to disclose their own usage in the README. That's unclear at best, misleading at worst.

I do not ascribe ill intent to the KPXC developers here, but this does appear rather ham-fisted, and in a community deeply sensitive to this topic—including myself.

All tagged AI-assisted PRs. https://github.com/keepassxreboot/keepassxc/issues?q=label%3A%22pr%3A%20ai-assisted%22

Commit with the policy and Copilot files: https://github.com/keepassxreboot/keepassxc/commit/c4b4be48a5e3d0b3d5dc61e1e5be21a55d9969c0

pH_

@pH_0x05@infosec.exchange replied · 11 hours ago

@mttaggart while things seems like it “remains to be proven wrong” for many, I am less forgiving when it’s my own passwords and creds are at stake.

If they wish to use AI and feels it’s the way to go, good on them, I wish them well and I won’t be along for the journey.

Taggart

@mttaggart@infosec.exchange replied · 12 hours ago

This is upsetting, and I think a common response is going to be to ditch KeePassXC. I'm not certain that's the best move here. Yes, even as someone who loathes generative models. Here's why.

You cannot meaningfully ban generative code from contributions. So long as this technology exists, people will use it. That's upsetting, but also not immediately changeable. A simple deny policy is both unenforceable and encourages lying about the use of models. A disclosure policy, while also unenforceable, does not encourage lying. Therefore the generative code which will inevitably be submitted has a higher probability of being called out as such with a disclosure policy, rather than a ban. The code submitted under the disclosure can then be put under greater scrutiny, as appropriate.

That said, KeePassXC needs to eat their own dog food. What I see in the Copilot-submitted PRs is all the evidence you want why this is not the best tool for a given job, and exactly why I avoid using these tools in my codebases.

I would hope, at the very least, that the README.md for the repo is updated to make clear that not only must generative code be disclosed, but that such code is contributed already by core maintainers.

Everyone should do what they feel is necessary of course, but as I've warned before, if you constantly demand moral purity of all you work with—including your tools and their creators—you will spend much less time fighting true evil.

Stephen Brooks 🦆

@sjb@mstdn.io replied · 8 hours ago

@mttaggart There probably are good ways to use GenAI in coding, but it seems we're always being sold the bad ones.

I've done things like write it myself, then ask Grok to write the same thing, then compare the two and merge the good features. Or even feed my version into the AI to ask it to look for errors.

Mark Wyner Won’t Comply :vm:

@markwyner@mas.to replied · 11 hours ago

@mttaggart

“…if you constantly demand moral purity of all you work with—including your tools and their creators—you will spend much less time fighting true evil.”

This is true for so much at the moment.

mcc

@mcc@mastodon.social replied · 11 hours ago

@mttaggart "You cannot meaningfully ban generative code from contributions." Utterly ridiculous. You can't stop murder, theft or any number or other bad things, but that doesn't mean it's not worth trying. If you ban generated code you get *some* generated code, if you don't ban it you get *more* generated code. The goal should be zero but if we can't get zero the goal should be as little as possible

Taggart

@mttaggart@infosec.exchange replied · 11 hours ago

@mcc This is a classic ban-vs-regulate debate, and there are plenty of non-ridiculous ways to arrive at either conclusion, thanks.

Usually the heuristic is whether the ban of a harmful act or good creates more harm than an attempted regulation of it. Violent crime is, imo, a really poor example because there is literally no version of legal pathway to harm or kill your neighbor. Well, unless you're a cop. But I digress.

Addictive substances may be better as an analog, since generative models seem to be one. Do you support drug criminalization? I suspect not. Ban vs. regulate.

So here we have clear potential outcomes. Let's imagine the project bans all generative code. Will everyone contribute in good faith? In a project this generally popular, assuredly not. The onus is therefore on the maintainers to deduce which contributions are human and which are not. Some will be missed. Maybe many. Outcome: generative code, no additional scrutiny.

In the scenario we have, contributors are asked to disclose their usage and have no disincentive to do so. This flags generative code for additional scrutiny.

From there we can get more granular. Should gigantic changes be introduced by machine? Of course not, but then that's also true of human commits. No proposed change should be so massive as to defy human review and understanding.

Look, I really hate generative models. My day job is demonstrating their risks. I also know hidden generative output is more dangerous than identified generative output.

If I were starting a new project today, I probably would ask for no generative code in PRs, but that's a privilege of a small project. One of KPXC's size must account for a much broader set of sensibilities and expectations from their community. As my project grew, my confidence that my ban was being honored would decrease, and the risk of hidden generative code would increase.

So all in all, I think regulate makes more sense than ban at the project level.

But I still want the technology to burn to the ground.

mcc

@mcc@mastodon.social replied · 11 hours ago

@mttaggart If you want to burn it to the ground, then burn it to the ground. *Saying* you dislike it while continuing to use software you know to have been written using LLMs is just another way of supporting it. This is how every single failure of capitalism occurs

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart It is incredibly important to understand the technology and its limitations. That is exactly what we did and will continue to do. There is never any implicit trust of any code. Even maintainer hand-developed code is reviewed by a co-maintainer prior to merging. No other FOSS password manager has that level of intense scrutiny.

David Zaslavsky

@diazona@techhub.social replied · 11 hours ago

@keepassxc @mttaggart That's good to know. But this whole discussion has piqued my curiosity about how well AI code contributions are being reviewed, and whether they really consistently get brought up to the same standard that contributions by a human expert would be before getting merged, and what that standard is. I'll have to take a closer look at the repo and development process to try to understand that. So it would be fair to say that this has diminished my trust in KeePassXC, though not to the point that I'm going to stop using it.

JustaJason

@JustaJason007@beige.party replied · 12 hours ago

@mttaggart ☝️

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart link to README: https://github.com/keepassxreboot/keepassxc/#generative-ai

Taggart

@mttaggart@infosec.exchange replied · 12 hours ago

@keepassxc Thank you! As I noted, that section does not clearly disclose your own use of generative models, and I humbly suggest it should, especially given how it's already been used.

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart Every pull request that has AI in it is marked with a label. You can look them up, it’s all fully transparent.

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart direct link to AI Assisted PR's (this is all, even draft unmerged ones): https://github.com/keepassxreboot/keepassxc/pulls?q=sort%3Aupdated-desc+is%3Apr+label%3A%22pr%3A+ai-assisted

Taggart

@mttaggart@infosec.exchange replied · 12 hours ago

@keepassxc I already included that exact link above. Please respond to my question: why wouldn't you include disclosure of your own usage in the README? It would make immediately clear that you are behaving as you expect others to do.

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart Frankly because we don't use generative ai that often. This project is easily 99% hand coded. Even generative ai submissions are co-developed with human input and fixes. Also, there is absolutely no requirement to disclose use of AI, as discussed we already make it clear which submissions are AI assisted.

1+ more replies (not shown)

Taggart

@mttaggart@infosec.exchange replied · 12 hours ago

@keepassxc I understand that the label is present. I am contending that the PR label is insufficient disclosure and that your own use of generative AI should be front-and-center on the README, alongside the requirement for disclosure of others. Is there a reason that doesn't sound right to you?

Team KeePassXC

@keepassxc@fosstodon.org replied · 12 hours ago

@mttaggart We use the labels to disclose our own use. What other disclosure do you need? We don’t commit to the main branch directly. Pull requests are not only for external contributors.

Taggart

@mttaggart@infosec.exchange replied · 12 hours ago

@keepassxc Respectfully, I feel like you may be missing my point here. As you've discovered, this issue has some strong feelings in the FOSS community. Acting in maximal good faith regarding the use of generative code goes a long way. Your README indicates contributions must disclose AI use in the PR. External contributors aren't adding the tag, they are indicating their use in the PR. You, near as I can tell, are adding the label. But that's beside the point.

I don't think it's appropriate to require users to read through PR labels to understand a project's own AI usage. Putting it up front is an act of transparency and good will. And I still haven't heard a reason not to.

1 more replies (not shown)

Knightmare

@LanceJZ@mastodon.social replied · 12 hours ago

@mttaggart this makes me sigh, and sigh hard.

Alexandro Lacadena 📷

@petaqui@masto.es replied · 12 hours ago

@mttaggart what the...

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances

Bonfire social · 1.0.0 no JS en

Automatic federation enabled