RE: https://mastodon.nz/@ClareBear/116608881300602104
I would go much, much farther than this excellent start.
"AI" is such a tangled mess of unknowns. Unless a booster can honestly answer some tough questions, it is manifestly unsuitable for any mission-critical applications. Start with these:
What is the provenance of the information that is being used for inference? Sources and references not only help establish the data's reliability, but also provide vital context. LLMs are just matching word fragments and predicting what would most likely come next.
What is the cost to run the model? The big vendors' token pricing is changing rapidly (hint: it's not going down) and a lot of big software shops are suddenly finding out that their monthly spend will put them out of business really quickly. See Shopify and Stripe for real-world examples.
What is the effective yield per token? How much "work" do you get out of an LLM for a given spend? No one knows, because you can give an identical set of prompts and get completely different outputs each time. There is nothing deterministic or predictable about this and it sure as hell is no way to run either a business or a public sector agency (they're different, BTW, but some requirements are universal).
How stable (and by extension trustworthy) are the vendors? Do you expect them to still be around after the bubble pops? Check out Ed Zitron's excellent rants for a withering analysis of how shaky this whole house of cards really is...https://www.wheresyoured.at/ai-is-too-expensive/
Just like the companies who have reduced their dev headcount, orgs that lay off a bunch of backroom analysts will find out the hard way that they were doing Real Work and that many other functions relied on what they did. When the LLM goes dark, even if you do manage to hire some of those domain experts back, they will be appalled and hampered by the scale of the mess you're asking them to clean up. #AI
Before those 8,700 jobs go, the public deserves four things:
A complete cost projection from Treasury, including what the AI itself will cost over time.
A public register listing every place AI is being used in government.
An Office for AI with real power to assess and stop risky projects.
A clear answer on which overseas companies our information is being handed to, and on what terms.
AI is not a magic wand. It is a contract. New Zealanders are entitled to see the terms before our names are signed to it.
