@timbray @simon https://fedi.simonwillison.net/@simon/117107511994840560

@timbray Remember when we used the term “home computers”? This feels a lot like it… expensive, quickly outdated, quirky, limited but also fun and amazing, enriching, and sometimes narrowly useful.

@timbray @simon You can run some serious models on mac studio’s. If you look up models under the “MLX LM" tag on hugging face it should give you an idea. MLX refers to the metal based models, if I am not mistaken.

@timbray I'm still frustrated that Apple had something with the 2019 Mac Pro and just abandoned it when they switched to Apple Silicon.

I kept hoping that when it moved to Apple Silicon they had something clever up their sleeves to let it scale up massively.

But instead they just decided that their "integrate everything" approach was good enough. It was a great step up for the low end to entry level pro machines but it was always a step down for the biggest pro workloads.

@timbray The AI YouTuber crowd, at least, has been pushing the idea of clustering 4 of them because they can do RDMA over TB5.
A little more about that here: https://arstechnica.com/apple/2026/08/with-new-mac-studio-and-mac-mini-apple-leans-hard-into-local-ai-inference/

@timbray @simon can confirm that the main constraint for LLMs are memory bandwidth and memory capacity, which Apple excels at in the desktop space -- 1.2TB/s of bandwidth across up to 512GB of memory allows you to run pretty massive and capable models locally. GLM 5.3 Flash (320B parameters) would fit on a maximum-spec M5 Mac and trade blows with Opus 4.7-4.8 on benchmarks.

@timbray @simon it seems Apple took media-consumption ideas from iOS and optimised macOS for large media files

it performs so abysmally on millions of tiny files, which today is exactly what we need with git and work trees and agent delegation

if Apple cared about performance here, they'd officially embrace Asahi or at least provide an official path for Linux on their hardware

a rare case of Apple skating to where the puck _was_

@timbray

While I only have minimal experience running these on my personal M2 and work M3, you can learn about folks' experiences over here:

https://www.reddit.com/r/LocalLLaMA/

Also, hi @simon I adore your blog, thx for writing it!

@timbray @simon Ornith and qwen are great local models and there are quite a few task-specific other ones. None are gonna match foundation models/infra but we rly shld not be investing in those anyway.

You can also fan out very large open weights models across the studios. Apple’s new kit has great memory bandwidth.

@timbray FWIW and IMHO, Qwen 3.8 on a 64GB M2 Ultra Mac Studio has good quality, at least on par with Claude Haiku.

It is slower by a fair bit, but that is something that should improve incrementally with time. Something newer than my Studio helps too, I'm sure.

@timbray @simon I was playing with pi and some open models on my M1 Max. Have yet to try anything seriously but there are probably some applications to consider. I feel like token costs will explode which will also add to the interest of local.

@timbray I've already seen a few people who supplement their use of something like Claude (tokens get spendy) for coding with open model LLMs run off their own computer (not sure if Mac or PC). @simon