Discussion
"no no the API prices going up don't affect AI at all, you can totally do what you need with an AI local model on your PC"
-- even the quantised model uses at least 256Gb VRAM, nobody has done the particular thing on a local PC
"well you can in the near future"
@davidgerard It's been the "near future" since the far past.
@davidgerard The primary reason is if average users saw how much power consumption / cost was actually involved in the big models, they wouldn’t use it. Or rather, their power company wouldn’t allow them the upgrades necessary, to a residential address.
@jripley notice how Nvidia is declaring your PC now needs a 900W PSU
@davidgerard fortunately the ram prices to run comparable local models are super reasonable
@davidgerard And don't you still need to boil the ocean to produce "local" models anyway?
@davidgerard that's how the goose is valued, i think
@davidgerard yeah my response to anyone who makes the "AI is the future because local models" argument is: then why aren't you just using local models?
Because every time I say I would never even consider using a non-local model, they're all "local models aren't powerful enough, you'll be left behind" and "that's a waste of time" (despite me not even using local models). You can't have it both ways.
@davidgerard Shades of crypto becoming environmentally friendly, fast, secure and easy to use within the next few years. It's almost as if it's the same lies from the same liars.
… right after RAM prices drop and 256 GiB of RAM becomes affordable?
Maybe after the bubble bursts and suppliers discover they sitting on unsellable piles of HBM. The only other option is that they reduce production and release the stockpiles slowly and a cartel would never do that.
@david_chisnall
Oh, it's even worse. That's 256 GiB of VRAM. I don't think consumer grade GPUs go that high.
@davidgerard
Install bonfire.cafe
Get the full app experience
