Post by @mcc@mastodon.social

RE: https://mathstodon.xyz/@highergeometer/117415296386994698

The thing is, to win, OpenAI doesn't have to get good at math research. They just have to make it nonviable for humans to work in math research

theHigherGeometer

@highergeometer@mathstodon.xyz 2 days ago

"A senior colleague of mine found one of his favourite problems among those solved and attempted to read the accompanying paper. He told me it was so unintelligible that, had he received it as an editor at a mathematics journal, “it would have gone straight into the bin”.
—Melissa Lee, Monash University

https://theconversation.com/is-this-the-mathocalypse-why-openais-latest-results-dump-has-left-mathematicians-in-shock-293837

2 Boosts

0 Likes

Robert Hutchinson

@ertchin@mstdn.social yesterday

@mcc I remember reading a month ago something along the lines of "they spent $22,000,000 in compute to generate this proof", and thinking that that was obviously a much better outcome than, say, giving 44 mathematicians $500,000 each to work on the same thing.

Woozle Hypertwin

@woozle@toot.cat 2 days ago

@mcc ...has anyone actually checked those "proofs" for, you know, validity?

I guess in this era I shouldn't be surprised if the answer is "no" and nobody in the media has even bothered asking.

Silly me, I assumed they wouldn't be calling them "proofs" unless mathematicians were agreeing that they did in fact prove the things they claimed to be proving.

mcc

@mcc@mastodon.social 2 days ago

@woozle People have been looking… there seem to be a lot of problems.

Personally I think the policy should be "if a large language model was involved, ignore it". Don't acknowledge it as a solution, don't even check to see if it's valid. If OpenAI dumps 500 proofs, and 50 are wrong, and 400 are right but duplicate existing work, and 45 are wrong but turn out right once a human does significant work to extend their logic, and 5 are right, then OpenAI gets press about their 5 or 50 "right" proofs

Inga stands with 🇺🇦 🇵🇸

@IngaLovinde@embracing.space yesterday

@mcc @woozle not to mention that going through 200-page long "paper" of complicated slop required quite a lot of effort. Unpaid of course, not even in reputation, because if you find an error, nobody will really care, and if you don't find an error, nobody will even mention your name as openai brags that their solution is right.

Woozle Hypertwin

@woozle@toot.cat yesterday

@IngaLovinde @mcc

Possible jiu-jitsu move: ask other LLMs to check it. If even one of them says "nope", then you can say "well, this LLM says that LLM is wrong" -- which means that one or the other of them got it wrong, which should help break the whole "LLMs are machines, and machines are perfect... are perfect... are perfect..." spell.

2 more replies

Hugo Mills

@darkling@mstdn.social 2 days ago

@woozle @mcc Yes, there are people working on it. Talia Ringer @TaliaRinger , who does a lot of work with (and on) formal proof assistants has been posting about it, for example.

There was also an article I saw a link to which was noting that the mathematical and proof-assistant versions of at least one of the recent batch didn't match up at all, so people have got far enough in on at least some of them to spot the discrepancies.