@david_chisnall

@david_chisnall
We are currently in a three-way battle with our Marketting department and Web provider.

The website has a useless chatbot, We keep disabling it for technical enquiries, Marketting keeps re-enabling it because it provides answers quickly.

So far, it has not contributed a useful answer out of hundreds of enquiries.

@david_chisnall I worry that one of the unacknowledged social-economic transformations underway is that companies are trying to normalize profound risk, including but not limited to making fraudulent technology part of their critical operations, such that even if some big disasters happen directly because of LLM use the legal precedent set will always focus on some individual employee's bad judgment. They had a lot of incentive to do that even before LLMs came along.

@david_chisnall

'I was always told that "Capitalism is About Freedom of Choice" ... so ... offer an AI and an AI-free version of your products and do that "Let the Market Decide" thing that I've been told so much about.'

@david_chisnall HIPAA compliance (health patient privacy law) is compromised by offsite LLMs. My spouse can't turn it off in Windows and it is constantly pestering her to let it summarize MS Teams meetings. Her IT department hasn't gotten around to figuring that out.

@david_chisnall

This.

The AI policy was basically: "look, we give you all of these shiny toys! Please use them! By the way, you accept all legal liability for their output! Have fun!"

> Insurance companies explicitly exclude coverage of any system using AI and any outputs of AI systems.
What's shocking to me is that this isn't already the case.
@david_chisnall

@david_chisnall wow, how dare you call it "passing-the-blame-to-employees-for-correctly-using-company-provided-tools", when they already named it a nice way, saying "human-in-the-loop" :)

@david_chisnall Agreed. Avoiding AI in production code until risks clear up.

@david_chisnall from what i know you are already in the matrix, you don`t realize it yet

@david_chisnall The caveat emptor stuff is interesting, but the time to really start worrying is when the AI profiteers get legislators to insulate them from third party liability.

@david_chisnall I've seen the second one where I work as "We can't trust that the providers won't use our inputs as training data. Since our clients are secretive, we must not use AI tools."

@david_chisnall

1. https://www.tomshardware.com/tech-industry/artificial-intelligence/insurers-move-to-limit-ai-liability-as-multi-billion-dollar-risks-emerge

@david_chisnall the definition of an "AI system" is already difficult and excluding any AI component in a larger system doesn't make sense, AI components can be perfectly fine when embedded and surrounded by quality controls/safeguards? 🤔

@david_chisnall I would like to have the same optimism like you. When I see the newest studies about users of LLMs I doubt that we can get rid of this pest (plus the immense damage on environment, drinking water, and knowledge).
After the burst of the US real property bubble, it was the poor and citizens (via taxes) who payed the bills for the banks. Poor countries fell in debt with consequences lasting today. We had no system change: the rich got richer, even faster and faster.

@david_chisnall Is there a reference for that "exclusion of coverage" thing? In the business press maybe? Quick search didn't turn up anything recent, just some two-year-old blathering full of hypotheticals. 🤷

@david_chisnall this is important because the only people who are paying anywhere near what it costs to run the service are big companies who are usually buying it as software as a service add-on they just talk to their bill because it sounds trendy and futuristic. Individual level consumers will lose them money even the paid tier so losing businesses means game over

@david_chisnall

They can also add a clause in the contract enforcing that the customer is solely responsible for using the AI enhanced product.

@david_chisnall So they'll expect an insurance bailout as well, I suppose. https://prospect.org/2025/11/07/openai-maneuvering-for-government-bailout/

@david_chisnall There are huge disclaimers that we need to sign to use AI at work. Then we are also told we won't get promotions without using AI, so we are effectively forced to click through the disclaimers.

The managers who are forcing this are also now telling software engineers that they are always responsible for their actions and code.

It is evident that they know there are huge risks to using AI for software coding, but they don't want to back down while the bubble is still inflating.

@david_chisnall

We assume an independent judiciary in the same breath we admit that the party with the most money usually wins in the US legal system.

In petrofascism, the monetary system is based on the consumption of fossil fuels (like natural gas for those AI chips). Meaning the money won't be printed for an unending string of civil suits to shutdown the toaster attached to the LNG generators.

They have all the liquidity in the world for insurance markets and lawyers.

@david_chisnall 1. is probably enough to make it all burst. Most "leaders" in modern companies don't have the guts to take any risks, so they won't if it isn't insured. And if one or two of the ones that do have to pay heavy damages due to AI doing weird shit, and regulators won't grant blanket exceptions for AI either it's all over.

@david_chisnall ironically enough https://mastodon.social/@urlyman/113820824544996398