The fact that Google decided to dump a 4 GiB language model file on every Chrome installation is yet another sign of how the generative AI craze is unsustainable. Don't look at it from the side of users, look at it from Google side. Having every user download a 4 GiB monster which will need to be routinely updated is a significant cost. It takes a ton of bandwidth to do that, far more than Chrome updates consume. And yet they're doing it because they're desperate to externalize the cost of "AI".
Post
Replies:
8
@gabrielesvelto it's also curious that a 4GB model that can run on consumer hardware is useful enough to force onto every user of Chrome, and somehow the AI companies expect everyone to pay them for these services.
@gundersen indeed. But there is nothing rational about this market. It's all C-suite FOMO, vibes, smoke and mirrors.
@gabrielesvelto I think you are absolutely correct!
@gabrielesvelto This download is going to happen every time they update the blob isn't it?
@pwloftus yes. It's basically a bunch of sparse matrixes, that get recomputed every time you re-train the model. It's very unlikely that it will be amenable to binary patching or incremental updates.
@gabrielesvelto I bet it's actually cheap for Google. They have network caches (or used to at least for YT) very close to the ISPs, and this kind of payload is very cache friendly.
