One thing I discovered early on about LLMs is them being non-deterministic. This is actually a big deal. What that means, essentially, is that unlike most science and indeed most programming, you put one set of inputs, you get the same outputs every fucking time.
In fact, normally in programming if you get randomly different outputs, despite the same inputs, that shit is fucking broken and you have a problem.
But we learned that sometimes....the AI would just fuck up. Sometimes it won't work, and you'd have my boss doing all this stuff to explain it like, "Well, the first time I did this, and the second time I did this, so that's probably what messed it up."
And I'm like no, you could have done the exact same thing both times, and indeed I'd take his 'failing' prompt, re-enter it, and it would work!
"As a programmer," I told him at the time, "This is so fucking weird. Like... I literally haven't encountered this scenario before in programming, and normally I'd say we should never rely on something like this because we have no clue what it's going to do, deterministically."
This was probably....almost 2 years ago now. But that problem certainly hasn't gone away.