Huh, I was at Apple.com and glanced at the Mac Studio page - https://www.apple.com/mac-studio/ - I notice they're explicitly pitching it for LLM on the desktop. I wonder how realistic that is… Hmm, let’s ask @simon, he’s closer to this stuff than most of us.
I’m pretty convinced that irrespective of all the issues, the economics of #genAI just don’t work with the current compute infrastructure requirements. Open models and what’s left of Moore’s law could be a really big deal.
@timbray I've already seen a few people who supplement their use of something like Claude (tokens get spendy) for coding with open model LLMs run off their own computer (not sure if Mac or PC). @simon