@endrift I find they can generally be much better than previous approaches at translating "human" errors and nonstandard language, but yeah they have the standard LLM issue where if you feed in garbage they don't know how to say "no" and they just hallucinate whatever is statistically most likely.
Replies:
6
@endrift Not that previous approaches did any better though. Google Translate has been doing stupid stuff since its inception and it has never had a "this doesn't make any sense" flag.
The main difference is LLMs are more likely to come up with grammatically correct, plausible sounding text instead of something clearly broken, when given clearly broken input.
@endrift I honestly don't know why they didn't have, like, a confidence flag in old models that could actually refuse to translate or warn.
"Standard" LLMs (the text completion kind), if that's what they're using now, probably can't implement that reliably, but you can probably architect a translation model that can (I forget what it's called but I think there's a model architecture better suited to translation).
@lina @endrift
I'd like it if machine translations added (?) markers and footnotes to explain areas of low confidence in the translation. That'd help the person using them to know they should look for more context, and deepen their understanding.
Answers without showing the working are just *much less useful*, in any field of study. Other machine translation methods could show their working. LLMs can't, there isn't really any.
@petealexharris @endrift LLMs do actually have the ability to show alternate terms/synonyms, that's like the one bit of data you can get out of them relatively easily.
But that doesn't really help with overall confidence in the translation, just identifying word alternatives.
@lina @endrift
*syntactically correct
Yeah, never failing to give an answer destroys confidence that *any* apparently suitable translation came from what was in the source text. If an output can come from literally nowhere, how do we know when that's happening as part of translating real input? The algorithm is the same in both cases.
@petealexharris @endrift Edited (whoops, that could certainly be read wrong).
But yeah, machine translation has always been like this, so this is nothing new.
This does not make MT (LLM or not) useless, it's just not a crystal ball that works in every case. You need to either be okay with the error rate, or have enough familiarity with the source language to at least be able to catch glaring errors or sense when something is off (and obviously you should not use it to produce professional output without a professional translator in the loop).
