alcoholic trying to get sober after a relapse tells Google's "Health Coach" about it.
it tells them to drink every hour and messages them to make sure they're drinking as soon as they wake up.
75
Replies
·
0
Boosts
@molly0xfff Is this thing confused about "drinking" and "drinking"? but even if that, drinking water every hour would be excessive, seems like it really went off-track. If this thing is really pushing people to drink more alcool, even if they are not alcoholic, there is just no excuse to that, there is no "healthy" dose of alcohol. More is always worse.
I use google health, but didn't use the coach, it's mostly an on phone data-storage with metrics and general advice, adding a chatbot in here, while tempting, is indeed pretty risky.
@tshirtman It can be extremely dangerous, even deadly, for serious alcoholics to quit cold-turkey. Even real doctors will sometimes recommend a patient keep drinking (for example, if they're waiting to get into a detox facility, or on a taper plan). My guess is that's what the AI was picking up on.
@molly0xfff @tshirtman That kind of nuance is what makes LLMs entirely unsuitable for providing medical advice. There's no understanding behind an LLM's words. Medical advice requires understanding an individual patient's current condition and history.
@molly0xfff there has to be liability here
@molly0xfff With all due respect, I think this guy needs a guardian.
Google is slipping, just think of the AdWords opportunity here of embedding Ads in Google AI products. Had Google been on the ball Alcoholic beverage companies and retailers could be bidding on ads for their products against a pre qualified pool of alcoholics.
@molly0xfff Jesus Fucking Guillotine
