alcoholic trying to get sober after a relapse tells Google's "Health Coach" about it.
it tells them to drink every hour and messages them to make sure they're drinking as soon as they wake up.
75
Replies
·
0
Boosts
@molly0xfff AI takes the risks associated with delirium tremens too seriously
@molly0xfff ”It’s just an early release. The next model will have fixed this, we promise!!”
@molly0xfff People who are addicted to benzodiazepines, alcohol, and barbiturates cannot abruptly stop - something I know from my own experience. If this person is drinking 2 fifths a day like Gerald Ford, then that might explain the advice from the AI.
@molly0xfff Realism drives to darkness
@molly0xfff ai is made for general help like in exams or paperwork (but don't abuse it and check the stuff the ai made and see it's right or not)
NOW, there's a big Problem here.
@molly0xfff the second-best thing about this post, other than adding to my citable cases of how unsuited LLM-stuff is for anyone to access (let alone pin world economies on), is that it gave me a whole bunch of people to block and never have to worry about interacting with ever again
...yay?
@molly0xfff reads like a psa and help me rolled into 1. ai unites humans. who’d knew
@molly0xfff the thing with personalized services is that there its much harder to gauge what the service is doing to a certain cohort of users.
We will never know if the Fitbit health coach targets alcholics differently.
And we know from previous examples of Uber etc. that the algorithms exploit people's vulnerabilities.
@molly0xfff Junkie just making excuses.
@molly0xfff I highly advise seeing a doctor right away. This is worth visiting an emergency room for professional advice and keep track of the AI interface.
Sober alcoholic here. That is the worst “health advice” ever.
@molly0xfff I sincerely wish all people would remember AI is a tool and nothing more. They should not relinquish control of their lives or identity. I also think it is blatantly irresponsible for companies to promote it in a way that suggests doing this is a good thing.
@molly0xfff I've actually enjoyed my Google health coach and generally it's worth it and just re-up'd at $100 a year. The only weird thing it's done so far it asked me about "hyperarousal". I might have had a normal night boner but hyperaroused seemed like a bit much. It's also weird that it tracks or can detect arousal/hyperarousal but only in the background, show me my horny stats.
fwiw, i did some digging before posting the screenshot.
based on their comment history they're a regular AI user (commenting about using Claude to code, etc.) they've also been posting about trying to quit drinking for 13 years
and frankly i think if someone is trying to karma farm by bashing AI, they're probably not going to do it on r/stopdrinking
@molly0xfff my first thought was did it mistake context and recommend drinking water ? that seems like reasonable fitness advice. The fitbit part is confusing.
@molly0xfff
Does seem like the kind of person that would latch onto any excuse to keep repeating bad habits while disowning their agency.
@tuneintodetuned why are we blaming alcoholics for google's irresponsible software?
@molly0xfff @tuneintodetuned I only chimed in because it sounded so far fetched. Google did buy fitbit and I have access to the AI feature as well. I just tested it and it advises to seek help from a medical professional. There is also a disclaimer about medical advice on the bottom of the chat box. Not that I am the first to jump up to defend google, but I think the person's post misrepresents reality. As someone who drank a lot for a few years, it was helpful when people called me out on my bs
@molly0xfff
I rather blame Google, this specific alcoholic as well as every person that could've helped them fight their addiction and didn't.
This hits too close to home. It's a long and tough struggle but not a futile one.
These companies' constant abuse is unforgivable and this example affecting a specially vulnerable demographic deserves attention, but I'd still never watch someone harm themself and reinforce their issues by telling them they had no choice.
@molly0xfff I feel obligated to point to Michael Reeves' short https://www.youtube.com/shorts/WP5_XJY_P0Q
@molly0xfff thank you for posting this and digging into it. “Is this legit” was my first question
@molly0xfff I am uncertain about this post. Fitbit doesn't have any sort of background messenger that will reach out and say "don't forget to drink." They might be misinterpreting or intentionally misreading the standard coaching notifications. If he is drinking 6-12 beers daily, he won't be thinking clearly.
I only say all of this because as much as I think we need to question AI, we need to avoid making stuff up or drawing conclusions without evidence.
@molly0xfff clearly a troll, the zero votes show that
@molly0xfff This is really scary, but a good friend of mine did die because he tried to quit cold turkey, which was the prevailing advice he got from everyone at the time, especially AA, but this is a massive overcorrection of that advice.
@molly0xfff This has to be satire.
Google should be shutdown and dismantled into cubicles.
But drinking is the least of this guy's problems. Just a symptom. This idiotic dependency is generations in the making by Republicans, and it'll take twice as many to claw back.
He needed humans a long, long time ago.
I hope he gets help. Being a drunk fucking sucks. But he's clearly had the capacity to help himself educated right out of him.
@molly0xfff Jesus fucking christ, that poor person. I hope they got the support and help they need (from a person). Fucking hell.
@molly0xfff what in the actual
@molly0xfff A friend told me about how he was able to use an LLM to do a project that he would have otherwise not been able to do. I told him that I see it like cyber-cocaine. Some people just take a little bump and it helps them do things that they otherwise could never have done.
Some people only take a little bump once in a while and never go over that.
Most people don't have that experience.
@molly0xfff It is crazy that any fucking product on the market would ever behave like that. What the fuck?!
@molly0xfff Any word on the litigation that was trying to say Section 230 didn’t protect platforms from what their own AI product says? Seems logical to me that they shouldn’t hide behind the “we’re just hosting our user’s content” defense when this AI nonsense is entirely in-house and something they claim as a differentiator compared to other platforms.
@molly0xfff Is this thing confused about "drinking" and "drinking"? but even if that, drinking water every hour would be excessive, seems like it really went off-track. If this thing is really pushing people to drink more alcool, even if they are not alcoholic, there is just no excuse to that, there is no "healthy" dose of alcohol. More is always worse.
I use google health, but didn't use the coach, it's mostly an on phone data-storage with metrics and general advice, adding a chatbot in here, while tempting, is indeed pretty risky.
@tshirtman It can be extremely dangerous, even deadly, for serious alcoholics to quit cold-turkey. Even real doctors will sometimes recommend a patient keep drinking (for example, if they're waiting to get into a detox facility, or on a taper plan). My guess is that's what the AI was picking up on.
@molly0xfff @tshirtman That kind of nuance is what makes LLMs entirely unsuitable for providing medical advice. There's no understanding behind an LLM's words. Medical advice requires understanding an individual patient's current condition and history.
@molly0xfff there has to be liability here
@molly0xfff With all due respect, I think this guy needs a guardian.
Google is slipping, just think of the AdWords opportunity here of embedding Ads in Google AI products. Had Google been on the ball Alcoholic beverage companies and retailers could be bidding on ads for their products against a pre qualified pool of alcoholics.
@molly0xfff Jesus Fucking Guillotine
