Post by @jsbarretto@social.coop

I have been thinking recently about how language is a high entropy latent space and what it means.

Latent spaces are sort of magic: they don't just reduce the dimensionality of a higher dimensional space, they also constrain it such that arbitrary movement in the latent space maps to some *meaningful* position in the higher dimensional space. Think about how almost any textual prompt you give to an image generator results in a picture with recognisable shape and form, yet the vast majority of the possible images are basically just random noise.

5 Replies

0 Boosts

Joshua Barretto

@jsbarretto@social.coop 3 hours ago

Language is, obviously, like this. Once you've learned to conquer the basic structural rules like syntax, grammar, and basic pattern matching of words with similar semantic clustering, movement within it almost always produces 'meaningful' (read: intelligible, not philosophically meaningful) results. Consider how an articulate human being can fairly easily produce passable bullshit in a topic with which they have no experience simply be reading a little of the topic's literature.

One way to think about this is that language is a tool that makes conceptual reasoning easy: once you have the basic rules laid down, it's almost more difficult to produce an output that *doesn't* carry some meaning than it is to produce an output that does.

Joshua Barretto

@jsbarretto@social.coop 3 hours ago

Consider a strongly-typed programming language like Rust, Haskell, Lean, whatever: like human language, almost every well-typed transformation of the code corresponds to some meaningful change. In the same way that programmers can 'bounce off' the barriers of a type system to avoid bugs, a person using human language can 'bounce off' conceptual barriers that would otherwise lead to them producing unintelligible nonsense, simply by following the grammatical rules of the language.

However: the map is not the territory. For every well-typed transformation, there are innumerable invalid transformations that would correspond to meaningless rubbish in less expressive languages. Effectively, the language is turning the problem of *domain reasoning* into one of *pattern-matching* and *symbol manipulation* by constraining the set of possible programs to a small latent space within that covers a particularly high concentration of meaningful programs.

Joshua Barretto

@jsbarretto@social.coop 2 hours ago

Problem is, for all of human history, symbol manipulation has been seen as *very difficult* because it's something that is less immediately intuitive to our brains. So whenever we see someone - or something - perform manipulation of a latent space like language our tendency is to presume that they first did their reasoning in the low-entropy, noisy, complicated space of *reality* and then transformed it into the latent space as a final step. But there is no reason for a machine to take this course of action: why should it?

The point I'm very slowly getting to is that human language is a latent space that we have fashioned over millennia of careful analysis, categorisation, taxonomic research, and study of the real world. That is a specific kind of work that we usually forget about in our text-and-language-filled lives where we live in the map and forget that the territory even exists. It's also work that an LLM cannot do: they are symbol manipulators, not creators of symbols.

Joshua Barretto

@jsbarretto@social.coop 2 hours ago

Their reasoning is bounded by the linguistic latent space that we have laid down before them (as are we all, to some degree). But our unique ability is that we can operate outside that latent space too.

It's often seen as curious and a little bemusing that machines can generate mathematical proofs or find counterexamples to long-held conjectures while simultaneously being incapable of performing basic tasks like frying an egg or folding clothes, but perhaps it should not be: the former requires only rapid manipulation of the semantically rich latent space that has already been mapped by human minds, while the latter requires more robust reasoning about reality, the sort of thing that only a world model can excel at.

1 more replies