Post by @mcc@mastodon.social

RE: https://mathstodon.xyz/@highergeometer/117415296386994698

The thing is, to win, OpenAI doesn't have to get good at math research. They just have to make it nonviable for humans to work in math research

theHigherGeometer

@highergeometer@mathstodon.xyz 21 hr. ago

"A senior colleague of mine found one of his favourite problems among those solved and attempted to read the accompanying paper. He told me it was so unintelligible that, had he received it as an editor at a mathematics journal, “it would have gone straight into the bin”.
—Melissa Lee, Monash University

https://theconversation.com/is-this-the-mathocalypse-why-openais-latest-results-dump-has-left-mathematicians-in-shock-293837

1 Boost

0 Likes

Woozle Hypertwin

@woozle@toot.cat 10 hr. ago

@mcc ...has anyone actually checked those "proofs" for, you know, validity?

I guess in this era I shouldn't be surprised if the answer is "no" and nobody in the media has even bothered asking.

Silly me, I assumed they wouldn't be calling them "proofs" unless mathematicians were agreeing that they did in fact prove the things they claimed to be proving.

mcc

@mcc@mastodon.social 9 hr. ago

@woozle People have been looking… there seem to be a lot of problems.

Personally I think the policy should be "if a large language model was involved, ignore it". Don't acknowledge it as a solution, don't even check to see if it's valid. If OpenAI dumps 500 proofs, and 50 are wrong, and 400 are right but duplicate existing work, and 45 are wrong but turn out right once a human does significant work to extend their logic, and 5 are right, then OpenAI gets press about their 5 or 50 "right" proofs

Inga stands with 🇺🇦 🇵🇸

@IngaLovinde@embracing.space 7 hr. ago

@mcc @woozle not to mention that going through 200-page long "paper" of complicated slop required quite a lot of effort. Unpaid of course, not even in reputation, because if you find an error, nobody will really care, and if you don't find an error, nobody will even mention your name as openai brags that their solution is right.

Woozle Hypertwin

@woozle@toot.cat 7 hr. ago

@IngaLovinde @mcc

Possible jiu-jitsu move: ask other LLMs to check it. If even one of them says "nope", then you can say "well, this LLM says that LLM is wrong" -- which means that one or the other of them got it wrong, which should help break the whole "LLMs are machines, and machines are perfect... are perfect... are perfect..." spell.

1 more reply

Hugo Mills

@darkling@mstdn.social 10 hr. ago

@woozle @mcc Yes, there are people working on it. Talia Ringer @TaliaRinger , who does a lot of work with (and on) formal proof assistants has been posting about it, for example.

There was also an article I saw a link to which was noting that the mathematical and proof-assistant versions of at least one of the recent batch didn't match up at all, so people have got far enough in on at least some of them to spot the discrepancies.