The mcdonalds thing was because the model they implemented was misinterpreting people and incorrectly placing orders. Yeah, obviously the thing wasn’t working right so they pulled that. Sounds just like early personal assistants on phones and other devices, hell my wife still struggles with those. They clearly needed more time developing and testing it with a diverse range of customers from all over. I don’t know if they trained it using recordings from real drive throughs from all over, but they should have.
The 18000 water example probably didn’t cost anyone anything. Regardless of if it was intentional or not, it wouldn’t have been fulfilled as part of an order. They mention it “crashing the system” - whatever that means in this context is impossible to know. Did it take down all of taco bell? Did it cause the LLM to stop responding on JUST this one site? All of them? Did it eventually time out and start working right? it’s impossible to know because the details just aren’t there and we have no insight as to the system architecture. I always assume there is a method to rely on traditional ordering where a person listening in while the chatbot talks to the person can take over and fix the problem. It’s not like there aren’t drive through workers still there.
Really the only cost here is the impact to consumer attitudes towards taco bell and AI because the video and news of this is circulating. One error is whatever, but public perception doesn’t typically involve much critical thinking.
People are still irrationally terrified of all manner of technology even though science backs it up, like vaccines.