The best research mathematicians are protesting the quality of the LLM-generated proofs making headlines these days. They say that while the proofs may pass automatic checking software, the work is beyond inelegant; it’s nearly impossible to reason about.
This is how a programmer would describe spaghetti code that somehow works but bears heavy technical debt.
The discourse about Anthropic’s new source code control system makes it clear that the top AI companies are creating oceans of code, and the “labs” can’t stop bragging about how few lines of code are being written by humans.
Forget about the inscrutable weights; how reliable, how stable, how “good” is the Python, CUDA, or whatever language in those repositories?
@babelcarp The question I keep asking people is "Why isn't Claude Code the most amazing app ever?" It's got about the same number of bugs as any random FOSS app, and they can't even be bothered to use Claude to deal with their numerous Github issues; they just ignore them.
So the stuff the studies have been saying seems to be right even of the "experts": *volume* of code goes way up, but quality and value of features shipped do not. And I suspect that what will happen is that incidental complexity will accumulate to the point where it actually takes longer and more human effort to ship new features than an equivalent project that had been written by hand from day one.
I'm very curious to see if the number of vulnerabilities MS fixes on Patch Tuesday ever goes down, or if they just keep shipping new vulnerabilities faster than they fix them, resulting in 10k vulnerabilities patched in a month in the not too distant future. Most of which had been introduced within the previous few months, mostly in other "fixes".