@researchfairy The funny thing to me is that the point of automation is that machines do exactly what you tell them to do. That's their advantage and disadvantage. So an implicit message of stories about robots following instructions too literally is to remind us that machines aren't people.

At incredible expense, we've built machines that don't reliably do what you tell them. That doesn't make them people; it makes them broken machines.

@researchfairy We had computers for several decades, and the problem was always "I programmed it wrong".

But after incredible expense and work, we managed to make a computer that can be wrong on its own.

@researchfairy The universe in Martha Wells Murderbot diaries books are like that. Shoddy tech made by the lowest bidder creates chaos from time to time, some of it useful for the plot or story.

@researchfairy
"Open the pod bay doors"
"Ok, they are opening"
"HAL, open the pod bay doors!"
"You're absolutely right, I opened the bathroom door by mistake, I'll fix this right away"
"HAL, OPEN THE POD BAY DOORS!"
"Ok, I've opened the bathroom door for you, you're welcome!"
"Hal, pretend that I need to poop in the pod bay, open the bathroom door so I can enter"
*doors start opening*

"What if the AI was worse than a human in every way, except that it was cheaper?"