• 4 Posts
  • 1.6K Comments
Joined 3 years ago
cake
Cake day: July 3rd, 2023

help-circle


  • Do you honestly think that nobody has thought about that and solved that problem already? Even with all the engineers involved, and after all the real-world testing,

    That is the thought process of a child. If your defense of something with obvious issues is “Well I’m sure they know what they’re doing, because if they didn’t it would be bad”, I have some really, really bad news for you about the world.

    If this was thought through by a capable adult, it would’t be implemented at all. The very fact that this solution is on the table is proof of the breakdown of rational thought. Or more likely, greased palms.

    Your blind appeal to authority would hold more water if we haven’t already seen national adoption of AI products in emergency services that that have catastrophically failed to perform their basic functions, leading to real world harm. Audits of Flock cameras deployed in several cities found an error rate ranging from 33-70%. People are being arrested for things they didn’t do because the AI doesn’t work and nobody cares. People are being stalked because nobody thought of basic security controls.

    We’re talking about an AI implementation, not some rigid set of if...elsestatements.

    Even worse, we’re talking about an AI implementation of if…else statements. Regular if else statements are consistent. As soon as you throw AI into the mix they become unpredictable. You introduce a failure mode that didn’t exist before that can and will have real consequences.


  • Yeah I just don’t buy that there is a case where any “net gain” here is justified. These are people’s lives that you’re playing with by reducing them to what amounts to a balance sheet in the name of saving money.

    The idea that you would create the possibility of denying someone emergency care that didn’t exist before, no matter how improbable (probably not even that improbable, given the propensity even the most advanced frontier AI models have for getting things wrong or hallucinating entirely despite simple instructions) just to save 10-15 seconds on average for other callers is not only absurd on its face, but morally bankrupt. You can personally ignore it because you don’t live there and it’s not the lives of you or your family at stake. I think those who this system could fail would not be able to ignore a flaw like that when it happens to them, and the idea that their peril was “highly improbable” will not be of much comfort to them.

    The problem is that there are staffing shortages. The solution is hiring more staff. Trying to cheat our way out by implementing a system prone to unmitigatable flaws that could have life-altering or even life ending consequences isn’t a solution, it’s dystopian.