Post by @thomasfuchs@hachyderm.io

RE: https://hachyderm.io/@thomasfuchs/117157175901405554

The other important thing to state is: LLMs are deterministic. The same input will yield the same output.

Note: randomness and things like "current time" are inputs. There may also be implementation details that have nothing to do with the LLM itself that cause variations, for example race conditions between GPU cores. But as designed, they're deterministic; just like every other software algorithm.

Thomas Fuchs

@thomasfuchs@hachyderm.io 7 hr. ago

I think it's important to occasionally state this, but LLMs are purely static and immutable sets of numbers and do not "learn" or change with use.

4 Replies

0 Boosts

Aaron Brick — אהרן בריק

@aarbrk@mstdn.mx 6 hr. ago

@thomasfuchs Is this accurate? To date I've understood that transformer inference involves cooling down excited content, like simulated annealing. Empirically, I once gave ChatGPT the same question three times and got three categorically different answers. Was the randomness I saw really just a design choice?

Thomas Fuchs

@thomasfuchs@hachyderm.io 6 hr. ago

@aarbrk Computers compute things deterministically. That’s it, therefore everything that runs on a computer is deterministic.

Aaron Brick — אהרן בריק

@aarbrk@mstdn.mx 3 hr. ago

@thomasfuchs LLMs don't follow Wikipedia's definition of determinism, which is producing the same answer every time.

Thomas Fuchs

@thomasfuchs@hachyderm.io 50 min. ago

@aarbrk Yes, as I explained it's because randomness is added as an input. Computers cannot produce randomness; therefore they use e.g. a timestamp or other external information to seed a random number generators (that will then produce a deterministic sequence of numbers).

Discussion

Discussion