![[object Object]](https://media.mas.to/accounts/avatars/109/287/758/133/964/366/original/0f3099c0adfb36a0.png "[object Object]")
so it’s day 2? 3?? of the AGI era, how’s that foom going
did the slopbot solve global climate change or kill us all with fantasy nanomachines yet
8
Replies
3
Boosts
you might think I’m misrepresenting the idea of superintelligent AI here but no, the intelligence explosion is supposed to be a faster-than-overnight revolution that changes the entire world
I uhh didn’t make this shit up
I did learn a ton about it because anti-cult work is always good work
The singularity was an interesting thought experiment. If humans can create a machine that is more intelligent than the people who created the machine, what would happen? And one possible outcome is that it would create a more intelligent machine. And, because each iteration would be more intelligent than the previous one, the time taken between generations would shrink and intelligence would increase at an exponential or, at least, geometric rate. Within a very short period, you’d have intelligences so advanced that humans couldn’t even comprehend them.
This is a fun hypothesis, but it ignores a bunch of reality:
First, it assumes intelligence is some linear gradient. This is the kind of nonsense popularised by the (thoroughly debunked) notion of IQ: there is a number that measures intelligence and you can rank two minds on a one-dimensional scale. This is a delightful belief for people working on ‘AI’ because it assumes ‘intelligence’ and ‘ability to build the kind of thing that they’re working on’ are the same thing. There is no reason to expect that a machine that, across some common set of metrics of intelligence, is better than a human would be better at building an intelligent mind than its designers. And, if it were, it’s quite feasible that it would have a very specialised intelligence that is very good at creating machines that are better at creating machines with that specific kind of intelligence. Alastair Reynolds has explored this possibility in at least one series.
Second, it assumes that intelligence is entirely a software problem. Create a machine intelligence and it can build something better in the same substrate. That is obvious nonsense. The Church-Turing thesis tells us that the substrate one intelligence runs in could simulate the substrate that the smarter (whatever that means) one needs, but nothing about how fast it would run. Even assuming that the first intelligent machine could design the new process technology required to fab the chips that its successor would run on and design those chips, that’s a multi-year lead time.
There’s a variation of this that implies that the intelligence would spread out and consume all available compute resources but that ignores communication latency. There’s a reason we don’t build massive CPUs: signal propagation delay from one side of a big chip to the other is a bottleneck (okay, there are also yield and cost reasons). A distributed system can improve throughput at the expense of latency. Again, Alastair Reynolds used this in one book very effectively.
It’s a nice thought experiment. It’s a fun premise for science fiction with a side of fantasy (I enjoy Ken MacLeod’s work in this space, it’s a fun setting, it doesn’t need to be a plausible one). But expecting it to be real is like trying to do the Schrödinger’s Cat experiment in real life and observe the cat.
Though cults have been created around less plausible beliefs in the past.
@david_chisnall @zzt Personally, I think the singularity is absolutely real, and that we're living in it. But so did our grandparents.
Zoom out to the grand sweep of the entire time this species has existed, and if you consider technological sophistication, what you see is one enormous long near-horizontal line which then steepens to near-vertical at the very end. Someone living 100.000 years ago would be able to recognize, understand and use all the tools made by someone living 200000, 50000 or 15000 years ago. Sure, there were *changes*, but many of those were about changing aesthetics more than about radical breakthroughs. Then about 10000 years ago there's a small bump upwards in the curve when agriculture is invented, some 5500 years ago both early metallurgy and the first written languages bumped it further up, and since then it's gotten steeper and steeper, with the intervals between the breakthrough events shortening. Sure looks like "foom" to me, even if it doesn't feel like it from the much shorter perspectives of individuals living in it.
But even then. My grandmother was born into a world where going to the nearby town required planning for extra breaks or treats for the family horse, who hated walking on paved roads. The one family in her village who owned a radio became a community gathering point. When she died, she had flown intercontinentally and owned an iPhone.
One of the metrics I like is the amount of time working required to be able to afford an hour of light. For most of history, artificial light was out of reach of most of the population because the effort to refine the oil for an oil lamp or make candles was more than the value of an hour of labour. A bit before the Industrial Revolution, this reached an inflection point where an hour of labour could buy you a candle that would let you work for more than an hour after sunset. This didn’t make a huge impact because most of the time spent working then was just paying for the light. Mass production of gas and, later, electricity caused massive drops in the cost of an hour of lighting. By the early 20th century, the cost of lighting was low enough that most people would use the extra time for leisure not labour.
When I was young, we used incandescent bulbs everywhere. A 60W bulb could just about light a room well enough to read but you typically wanted more. I had a 40W desk lamp (which doubled and a bedside light) and a 100W bulb in the middle of my bedroom and later two 60W spots as bedside lights and a 60W room light after we moved). The bulbs blew fairly frequently (they typically lasted 6-12 months: we had a big cupboard full of replacements that was constantly restocked). The bulbs were still not a major cost of the lighting though, the electricity was. Lighting was the largest single contributors to our electricity bill and one of the largest costs for our power bills (heating was the other and wasn’t electric).
When I went to university, there was some subsidy for compact fluorescents. We got a load of 15 W ones, which were about as bright as a 100W bulb (part of the reason people hated these was that they outright lied about the equivalence, so buying a ‘60W equivalent’ bulb made the room gloomy). I had an 8W bedside light and a 15W room light. The bulbs cost about £1.50 each, so were actually a significant part of the total cost: for a 15W bulb, being on 10% of the time for a year cost as much as the bulb.
Now, most of my house uses LED bulbs. I think the brightest are 7W? We have a small draw that has a few spare ones because the rectifiers burn out periodically but they last for years.
So, a few things about why I don’t think this meets the definition of a singularity:
First, there was rapid acceleration but it wasn’t at all smooth. Gas lamps, filament bulbs, compact fluorescents, and LEDs were all step changes.
Second, the knock-on productivity effects haven’t really changed for over 100 years. Anyone who wanted / needed to work while it was dark outside has been able to for over 100 years (at least, in affluent countries). Beyond that, the cost going down hasn’t changed the economics. And we’ve already hit diminishing returns. Going from needing to buy and candle to get dim light to a gas or electric light where you hit a switch and got light bright enough to read with was big. Going from 150W to brightly light a room to 15W was a huge improvement. But going from 15W to 7W to light the same room? It’s nice. But it’s basically a rounding error. The prior improvement moved lighting from being the biggest contributor to the electricity bill to being way down the list. The most recent one made basically no difference. If some future technology lets me light the room for 2-4W, I’ll buy it if it’s no more expensive than LEDs, but I don’t really care.
Third, it isn’t a compounding effect. The existence of CFLs did not directly cause the creation of LED bulbs, both were created in response to the same underlying economic conditions.
There is some compounding effect across ent entire economy due to increased time for people to spend in education.
@zzt it made OpenAI profitable
@zzt yeaaaah I think people do not realize how much the few of us that do anti-cult work have been following that stuff for decades....
