Theory-free inference is a very pragmatic way to approach the world: "I don't need it good, I need it *Thursday*." Scientists burn to know why a molecule stopped you from dying, but you are likely satisfied to not be dead. What's more, our ability to observe correlations will always race ahead of our understanding of causality, so the power of theory-free inferences pushes out the frontier of things we can act on, beyond the realm of the understood.

23/

Lots of stuff in the world works this way. Our understanding of the causal relationships that make up reality has massive holes in it that we fill with mere correlation. Correlations are easier to discover than causes, and while correlation is (famously) not causation, causes and effects *are* correlated, and if you can evince the effect you're seeking without understanding precisely what happened to make that effect appear, well, at least you got the effect you were seeking.

22/

With theory-free inference, you don't have to figure out how the world works in order to act upon it: you can just gather up all the data about how things happen in the world, use statistical methods to find the correlations, and then intervene to change the outcomes. You don't have to know why a molecule improves a medical condition - it's enough to discover that fact, produce that molecule, and administer it to people with that condition.

21/

Researchers and investors came to expect an AI that was "untouched by human hands," that taught itself how the world worked. This was the self-licking ice-cream cone of machine learning, the world of "theory-free inference" that had fueled the Big Data industry.

20/

The early gains from this were very exciting: they dangled the promise of software that could essentially "teach itself" how to do complicated, valuable things in a series of accelerating returns. The fact that the early improvements in AI systems that used this technique were *so* much greater than anyone would have expected based on AI research up to that point dangled an even more exciting promise: that the improvements would continue to scale faster than the inputs.

19/

Deep learning swapped the painstaking work of describing reality in software for a brute-force approach: throw *lots* more training data at the system and then throw *lots* more (parallel) computing power at that data and let the computer figure it out without your having to explain how the world worked.

18/

The minor variation that produced the major improvement in AI performance was *scale*. Prior to the "deep learning" era, AI research relied on a mix of hand-built models of reality and training data that computers fitted into those models.

17/

These forces and accidents have acted on AI in ways that profoundly shaped its development. The latest AI boom started when a group of machine learning researchers tried a minor variation on existing techniques and saw a *major* improvement in the outcomes. This is one of the most exciting kinds of breakthrough: if tweaking a single variable in a small way produces a large improvement, then it may be that further tweaking will produce even more improvements.

16/

The point being that world-historic forces, combined with accidents of history, shape the outcomes of individuals, companies and disciplines. It's much easier to be an accomplished graphics wizard in an era in which GPUs are doubling in power every year than it is in an era when linear computing is getting the lion's share of investment and improvement.

15/

This feedback loop raised the profile of parallel computing applications, attracting more bright and ambitious people to those applications, whose even more impressive accomplishments brought more people into the field, more capital into hardware development, and more resources to parallelization research.

14/

In the same way, the computing industry's focus on parallel computing made life easier for people who burned to do something parallelizable. Then the achievements of the parallel computing partisans drove more investment in improvements to parallel computing hardware *and* theoretical work on how to parallelize other problems.

13/

In the centuries that followed, many other people had the insight that twirling blades on a screw of some type could provide lift for some kind of heavier-than-air craft. But it wasn't until strong alloys, internal combustion engines and light, energy-dense refined hydrocarbon fuels came on the scene that the helicopter became possible, whereupon it was all but inevitable, with several people independently inventing the helicopter all at once:

https://memex.craphound.com/2010/10/13/kevin-kellys-what-technology-wants-how-technology-changes-us-and-vice-versa/

12/

This is the era of performance gaming, VR and AR, cryptocurrency, and, of course, AI.

In *What Technology Wants*, Kevin Kelly introduces the idea of the "adjacent possible" through the example of the helicopter. Da Vinci sketched a "helicopter" - blades in the shape of maple keys attached to a kind of wine-press screw - in the 15th century.

11/

As parallel computing took off, so did parallel applications. This is the beginning of the graphics revolution, as GPUs - components made up of many, many low-powered computers - became more central to academic research and commercial product roadmaps. But it wasn't just graphics that saw a huge lift here: any task that could be parallelized got easier and cheaper to perform every year, in a steady trend that has run to this day.

10/

When Moore's Law tapped out - when the pace at which transistors got smaller and computers got faster slowed and plateaued, and the expense of even modest performance gains climbed infinitywards - computing changed with it. Parallel computing - putting more cores on a chip, more chips on a board, more boards in a system - took off, as chipmakers and system builders switched from a focus on building their computers tall to building them *wide*.

9/

Even if your application or field of study was expensive and difficult to realize on today's computer, you could just stand still for a year or two and a much faster computer would park itself on your doorstep, ready to solve your problems.

8/

Take computing: for decades, computing was ruled by Moore's Law, an unbroken run in which computers got faster and cheaper every year. If you were interested in the kinds of computing applications that were well-suited to serial computation - programs that worked best when run on a single computer - you were in luck.

7/