Discussion
RE: https://graphics.social/@metin/116929283984909484
"Large language models do not scale logarithmically. When they’re given more words to process, they get slower and use more memory—the time and resources increase faster as the input grows. In technical terms, LLMs scale quadratically. Any computer-science student knows that this is very bad."
@steltenpower Quadratic isn't that bad for an algorithm, but the plain problem is: We can only process so many variables at one time. And its hard to partition a LLM. We need to become smarter in using LLMs, not make the LLMs bigger.
(Apologies for being direct, but...)
Quadratic is terrible!
I used to send a note to students of the upper year numerical methods course that quadratic run time would be an automatic zero on the assignment. I always got one or two...
Earlier this year I did a demo using a few hundred megabytes of data. A 🇺🇸 lab had used an Neural Net/AI setup (not even an LLM) to analyze it.
I didn't.
My results on the same data, same problem, took 9 seconds on one old CPU core. The 🇺🇸 lab took 64 hours on a computer at least a hundred times as powerful. Net, that is a resource difference of the order of millions.
My way used millions of times less resources, and got better results, too.
This was on data equivalent to a hundred photos.
My way doesn't sell GPU's, though.
Install bonfire.cafe
Get the full app experience
