Post

@mttaggart just another scam for the tech gods to destroy society

@mttaggart "the whole thing falls apart" is of course a built-in endpoint for any bezzle or confidence scam

@mttaggart Skimmed this... But on top of your screenshot, 60% also recorded the wrong prescribed drug. Why are none of the recommendations not to use AI systems that cannot accurately and precisely perform the required job? One of the recommendations is actually to *increase* AI use, despite the aforementioned evidence that 25-60% of the information the scribes provide being useless.

@mttaggart

was the machine EVER right?

@mttaggart Congratulations, medical industry. You have have made your doctors work harder and made more people sick. Shining example

@mttaggart Wait a moment: they used Microsoft Defender to assess that Microsoft Copilot is the only safe LLM to use? And all of that because they payed for Microsoft services?
It's in the first pages.
Maybe the document dates back to April 1st. Maybe it's just one of the usual jokes spreading on the net, and I never get them.

@mttaggart

The Ford government's approach to slop looks, well... sloppy.

At a time when OPS staff morale is at an all-time low and the relationship of government members and their staff to what was once a model public service remains one of ideological hostility.

I get to see this in action. Doctors want transcription and summarization services because of the challenges they face getting quickly familiar with a patient in a crazy short amount of time. They also want to automate notetaking for rounds, which can be chaotic. Problem is, these tools suck in chaotic situations, and even in relatively normal ones, hallucination abounds.

There will always be a claim of human review, but I know all too well that it's working against the current to have a human reviewer not assume the model got it right. What's more, those safeguards will eventually be seen as cost centers and redundancies—well, at least until the lawsuits.

One other thing. As noted above, these model-generated fields in charts are a) being used as training material for other models, and b) being used as input for other generative tools without human review. The potential for compound errors and model collapse is immense.

@mttaggart we all want the autodocs we were promised in sci-fi, but genAI is not that.

PS: medicine is already a field suffering heavily from biases. Adding automated bias at scale is gonna literally kill millions more of us.

@mttaggart Seriously, any medical professional would get banned for malpractice if they had such a huge error rate.

@mttaggart Medicine should probably stick to machine learning pattern recognition in diagnostics, that seems useful? from what I've read?

@mttaggart It's already too late. This garbage is not only already in our medical records, but it's also being used to train the next generation of models, which will coupound (and hide) the issue!

@mttaggart The minister responsible for that AI Scribe project's explanation of why it was all OK was incredibly bad https://mastodon.social/@reedmideke/116570464172955876