Friends, you may have seen this but if not, sharing:
"The LLM-generated proof hinges on a particular mathematical argument that it presented as its own but that actually first appeared in a 2016 paper by Miller and a collaborator."
13
Replies
20
Boosts
@timnitGebru yet again, the operators of these things lack the means to evaluate their outputs.
Love the graphic.
Can't read the article as I am burnt out on any hype on AI.
@timnitGebru that site blocks my vpn wtf
@timnitGebru Who could possibly have predicted this? 🤦
@timnitGebru OpenAI is a bunch of no-talent ass clowns.
@timnitGebru The one thing you can't claim about an LLM is that it hadn't seen the prior art.
"Two of the most exciting results, the experts say, incorporate preexisting ideas from the recent mathematical literature without properly citing them. This contradicts OpenAI’s initial press release, which said that the problems Astra addressed “have been open and seen no progress on the main result for at least a decade.” (OpenAI has since updated the language to be more accurate).
“They are running roughshod over the work of others who came before them in a deliberate way,” says Stephen Miller, a mathematician at Yeshiva University, who argues that OpenAI has effectively plagiarized his own research. “It seems completely systematic to me, and it points to research misconduct.”
"The LLM-generated proof hinges on a particular mathematical argument that it presented as its own but that actually first appeared in a 2016 paper by Miller and a collaborator..."
"Like a number of recent AI breakthroughs, it pasted together ideas from the mathematical literature to build a new theorem. Once again, the LLM’s trick is its superhuman patience for assembling puzzle pieces, not the ability to make some profound intellectual leap."
"“there is the big PR machine that wants to sound as impressive as possible and does not care about being 100 percent accurate,” he says."
@timnitGebru It's particularly frustrating that the academic misconduct is most likely completely unnecessary to OpenAI's goals. First, the LLM is most likely capable of providing the correct references if you prompt it to do so (at least that's what Ive heard, from people who have received neat proofs from an LLM and have asked it to cite instances in the literature where that argument was used before). Second, adequately giving credit to people doesn't make novel proofs of conjectures any less exciting to mathematicians; doing it right would serve their PR purposes equally well.
@oantolin I don't think it would serve their PR purposes because it would look more like the machines that win at chess or other approaches that look more like brute force.
"The discovery stunned Francesco Fournier-Facio, a mathematician at the University of Cambridge, who studies group theory—at least until he “engaged with this breakthrough as I would if a human had written it,” he says. The result, he and some of his colleagues found, wasn’t as novel as it first appeared."
@timnitGebru define “properly”, is it mentioning the 10 zwar old paper or not?
