@mttaggart
They found 45% of the systems hallucinated (added information), 60% were incorrect, and 85% were incomplete. But these error rates were across two tests, it looks like. I think, with enough tests, they would discover that all the vendors' systems have these problems; I believe they would find that these rates apply across the board.

@mttaggart

This is what testing in production gets you!

@mttaggart

You know who must love this result? Attorneys for malpractice plaintiffs.

With the chart no longer an authoritative, or even plausible, record of what happened, a malpractice lawsuit will just come down to who can tell the jury a better story.

@mttaggart
And they’re using weasel words in their results document.
These are not ‚inaccuracies‘. These should all be called failed tests / software errors that need to be fixed.

@mttaggart I think this technology is good for a casual, routine meeting at a company that's making products where no lives depend on them.

Errors correct themselves.

To use this in diagnostic and medical care is beyond reckless.

Errors in these fields cause suffering and death.

@mttaggart I liked* the bit where they only gave accuracy of transcription a 4% weight but made-in-Ontario was 30% of the score when determining which tool was best for purpose.

* hated, but appreciated the accidental honesty

@mttaggart what things like this miss (purely with respect to accuracy not privacy etx) is not the absolute data but the relative data. The question is not "how often does ai get it wrong" but "is AI better worse than humans already pretty bad note taking" (the kind of thing that comes up in eg EMR poetry).

@mttaggart just another scam for the tech gods to destroy society

@mttaggart "the whole thing falls apart" is of course a built-in endpoint for any bezzle or confidence scam

@mttaggart

was the machine EVER right?

@mttaggart Congratulations, medical industry. You have have made your doctors work harder and made more people sick. Shining example

@mttaggart Wait a moment: they used Microsoft Defender to assess that Microsoft Copilot is the only safe LLM to use? And all of that because they payed for Microsoft services?
It's in the first pages.
Maybe the document dates back to April 1st. Maybe it's just one of the usual jokes spreading on the net, and I never get them.

@mttaggart

The Ford government's approach to slop looks, well... sloppy.

At a time when OPS staff morale is at an all-time low and the relationship of government members and their staff to what was once a model public service remains one of ideological hostility.

I get to see this in action. Doctors want transcription and summarization services because of the challenges they face getting quickly familiar with a patient in a crazy short amount of time. They also want to automate notetaking for rounds, which can be chaotic. Problem is, these tools suck in chaotic situations, and even in relatively normal ones, hallucination abounds.

There will always be a claim of human review, but I know all too well that it's working against the current to have a human reviewer not assume the model got it right. What's more, those safeguards will eventually be seen as cost centers and redundancies—well, at least until the lawsuits.

One other thing. As noted above, these model-generated fields in charts are a) being used as training material for other models, and b) being used as input for other generative tools without human review. The potential for compound errors and model collapse is immense.

@mttaggart their audit reads like a Wirken requirements document written by someone who did not know Wirken existed. I do five or six calls on this a week now, which is why I open-sourced and started giving Wirken away for free. I've updated the marketing copy here, but I'll soon release a line-by-line response to the Ontario audit: https://gebruder.ottenheimer.app/wirken

@mttaggart we all want the autodocs we were promised in sci-fi, but genAI is not that.

PS: medicine is already a field suffering heavily from biases. Adding automated bias at scale is gonna literally kill millions more of us.

@mttaggart Seriously, any medical professional would get banned for malpractice if they had such a huge error rate.

@mttaggart Medicine should probably stick to machine learning pattern recognition in diagnostics, that seems useful? from what I've read?

@mttaggart It's already too late. This garbage is not only already in our medical records, but it's also being used to train the next generation of models, which will coupound (and hide) the issue!

@mttaggart The minister responsible for that AI Scribe project's explanation of why it was all OK was incredibly bad https://mastodon.social/@reedmideke/116570464172955876