@mttaggart
They found 45% of the systems hallucinated (added information), 60% were incorrect, and 85% were incomplete. But these error rates were across two tests, it looks like. I think, with enough tests, they would discover that all the vendors' systems have these problems; I believe they would find that these rates apply across the board.

@mttaggart

This is what testing in production gets you!

@mttaggart

You know who must love this result? Attorneys for malpractice plaintiffs.

With the chart no longer an authoritative, or even plausible, record of what happened, a malpractice lawsuit will just come down to who can tell the jury a better story.

@mttaggart
And they’re using weasel words in their results document.
These are not ‚inaccuracies‘. These should all be called failed tests / software errors that need to be fixed.

@mttaggart I think this technology is good for a casual, routine meeting at a company that's making products where no lives depend on them.

Errors correct themselves.

To use this in diagnostic and medical care is beyond reckless.

Errors in these fields cause suffering and death.

@mttaggart I liked* the bit where they only gave accuracy of transcription a 4% weight but made-in-Ontario was 30% of the score when determining which tool was best for purpose.

* hated, but appreciated the accidental honesty

@mttaggart what things like this miss (purely with respect to accuracy not privacy etx) is not the absolute data but the relative data. The question is not "how often does ai get it wrong" but "is AI better worse than humans already pretty bad note taking" (the kind of thing that comes up in eg EMR poetry).

@mttaggart just another scam for the tech gods to destroy society

@mttaggart "the whole thing falls apart" is of course a built-in endpoint for any bezzle or confidence scam

@mttaggart Skimmed this... But on top of your screenshot, 60% also recorded the wrong prescribed drug. Why are none of the recommendations not to use AI systems that cannot accurately and precisely perform the required job? One of the recommendations is actually to *increase* AI use, despite the aforementioned evidence that 25-60% of the information the scribes provide being useless.

@mttaggart

was the machine EVER right?

@mttaggart Congratulations, medical industry. You have have made your doctors work harder and made more people sick. Shining example

@mttaggart Wait a moment: they used Microsoft Defender to assess that Microsoft Copilot is the only safe LLM to use? And all of that because they payed for Microsoft services?
It's in the first pages.
Maybe the document dates back to April 1st. Maybe it's just one of the usual jokes spreading on the net, and I never get them.

@mttaggart

The Ford government's approach to slop looks, well... sloppy.

At a time when OPS staff morale is at an all-time low and the relationship of government members and their staff to what was once a model public service remains one of ideological hostility.

I get to see this in action. Doctors want transcription and summarization services because of the challenges they face getting quickly familiar with a patient in a crazy short amount of time. They also want to automate notetaking for rounds, which can be chaotic. Problem is, these tools suck in chaotic situations, and even in relatively normal ones, hallucination abounds.

There will always be a claim of human review, but I know all too well that it's working against the current to have a human reviewer not assume the model got it right. What's more, those safeguards will eventually be seen as cost centers and redundancies—well, at least until the lawsuits.

One other thing. As noted above, these model-generated fields in charts are a) being used as training material for other models, and b) being used as input for other generative tools without human review. The potential for compound errors and model collapse is immense.