I don't know how much louder anyone can scream "the endgame of AI is a dependency trap for whole life surveillance"
https://openai.com/index/gpt-6-astra/
22
Replies
12
Boosts
@jonny for a moment, I thought it was something from OpenAI about GTA 6
@jonny Nah, they will move on, head out into space, colonize the galaxy, etc. Timescales won't matter to them
@jonny Which they will use to rip everyone off on any purchases through the service. Already studied, and already happening.
https://www.pcmag.com/news/shop-around-study-finds-google-ai-mode-shows-higher-prices-than-search
@jonny If this AI is so good, it should've generated a website where the charts aren't broken.
@jonny Also: What is it with AI companies and incredibly heavyweight webpages which slow my laptop* to a crawl? They all seem to love them.
*: Laptop (Core Ultra 5 225U, 16 GiB RAM) is not a potato so the issue is on them.
@jonny I love how they put in a presentation about their Gaia model that doesn't exist, knowing full well that CEOs and consultants are going to be citing it
Its neat how openai has caught up with Google's brand strategy from like 2016 though
@jonny
Don't be evil?
This is such a great x axis. This must look awesome if you don't know anything
@jonny the y axis as well, percentage points but truncated between 20 and 70 hmmm yes seems good graphmaking
@jonny Q: What did the x-axis say to the y-axis? A: "They're all plotting against me!"
@jonny "70% success rate is fine. Humans make mistakes too. Just the other day I saw a toddler put a round peg into a square hole."
*cries*
the adjoining graph measuring "output tokens" when everyone knows goddamn well that means nothing when there is at least one hidden slush fund category "reasoning tokens" that is the one that's actually inflating
I was trained in psychology, I am familiar with the lineage of developing measurement instruments named "Gary's Perfect Intelligence Metric" and the test is just rotating a bunch of plus signs. I live in the united States and am aware of bills named "The Great Infinite Jello Glitch" that legalizes stripmining in orphanages. So naturally I love AI company press releases.
> "GPT-6 score 99% on the AGI test> looks inside AGI test> the arcade pack CD-ROM you'd shake out of a cereal box in 2003
2 media files
![[bar plot of GPT-6 at 99%, opus 5 at 30% score. score is undefined. output tokens are undefined.]
ARC-AGI-3 tests how well agents learn as they solve unfamiliar interactive tasks. GPT‑6 Astra saturates the eval, scoring 99.9%. The average human tester scored 48%. GPT‑6 Astra was measured with our responses API harness, which better reflects real-world performance than the original benchmark harness, which discards past reasoning and past messages. With this harness, we estimate Sol would score in the ballpark of ~30%.](https://media.neuromatch.social/media_attachments/files/117/217/685/815/413/911/original/a83abbb802315396.jpg)
We know we have reached true AGI when a brute force machine can reach 100% on a game that has a small number of discrete states with like 4 inputs without disclosing anything about the testing conditions
So hang on. The screenshot test says I can pay 10 cents for a robot to move my mouse to a button?
That can't be right, it must be 10 cents to run the whole benchmark, there are only 1500 samples per run.
But the output tokens are all less than 1000... So either it can move my mouse with fewer than 1 token on average... Or...... 10 cents to move my mouse is really what is being shown here.............
3 media files
![[gpt-6 achieving 90% accuracy between 7 and 11 cents, gpt-5.5 60-80% accuracy between 3 and 4 cents]
ScreenSpot-Pro tests whether models can locate the correct interface element in high-resolution screenshots of professional software. GPT‑6 Astra sets a new high with 92.7% accuracy.](https://media.neuromatch.social/media_attachments/files/117/217/770/627/753/648/original/9ac661ab980533e2.jpg)
![[gpt-6 at 90% accuracy between 0 and 750 output tokens. gpt-5.6 at 60-80% between 100 and 1000 output tokens]](https://media.neuromatch.social/media_attachments/files/117/217/770/663/369/696/original/7c2a69a50c869032.jpg)
@jonny now that is interesting, I've seen a lot of posts on reddit (i know big mistake visiting it) about how good astra is with blender, like you supply some schematics and getting a nice looking model out (with thousand of separate elements lol) and that will mean that with blender being free and astra being "soo good" that it will be used in movie studios and all that. If those costs are really for single mouse movement, i dont see it being cheaper than paying some human to do the model.
But what do I know? Big studios will probably prefer a bot that a humans who might dare to unionize.
@astromateusz
Two other possibilities are that they just suck at data viz and mix two transformations of the data (recursively on the table because any expectations of competence are undermined by the fact that LLMs certainly produced the document in some critical way), and also that the numbers are just made up (tautologically true because the api costs are made up, but you know what I mean)
I have not tried this myself, but if it actually was 10 cents per mouse move it would be too funny to be real anyway, the universe cannot tolerate a vacuum of something so funny
