A recurring theme in my arguments with AI proponents is that they universally have a flawed mental model of technological progress. They think that because AI can, for example, help discover better ways to make chips, that means that now things will continually accelerate.
But that's not how progress works. AI represents, at most, a step change, *not* an inflection point. It helps us make better use of the data available to us, but it doesn't make us any better as at actually integrating those discoveries into our society, institutions, and mental models.
To give an example, AlphaFold 2 represented a leap forward of about a decade in protein folding. But that's it. We're not now advancing a decade every year, we're just a decade ahead of where we were.
Likewise, we now have models that might help with discovering better approaches to quantum computing, and new room temperature superconductors. Those might get us to those things 10 years earlier than they otherwise would have, but that's it.
The reason for this is simple: they're just accelerating searches. They're not creating fundamentally new theories, just letting us explore the possibilities opened by existing theories faster.
Technological progress is not just about discovery and invention. In fact, those are the least important parts. Real, durable progress comes from developing better mental models of the world. And if anything, AI will slow that down by enabling more discoveries using worse mental models.