We are only in the first generation of generative artificial intelligence, but administrations, companies and law enforcement agencies already use it for critical tasks, as if AI were a proven tool. It’s not, because he hallucinates every now and then. An AI fan of Disney created a report for a Utah police station, claiming that a police officer had turned into a frog. The worrying thing is when it fails in things that are difficult to detect.
Law enforcement officers around the world have already integrated AI into their daily tasks, mainly to write reportswhich is quite tedious and takes a lot of time.
Police in Heber, a small Utah town of about 20,000, use software called Draft One con an AI to write reports using the body cam (body camera) that all police officers in the United States wear.
The AI that saw a police officer turn into a frog
Draft One recognizes the images that the agent records on the camera worn on his chest. When it finishes its round, the AI writes a report examining the recording. Describe what you see in the video and add the hour and minutes. It is capable of detecting emotions and tones of voice. The agent then reviews the report to check that it corresponds to what he has seen.
In early December, Sergeant Keel was reading one of these reports, and his jaw dropped: according to the AI, when he entered a house after a complaint… the police officer turned into a frog.
How could the AI have come to this conclusion? Was it a hallucination? He decided to check the camera recording, and discovered what had happened. In the living room of the house, They were watching the Disney movie Tiana and the toadwhere a prince transforms into this amphibian.
Somehow, AI confused the TV image with realityand concluded that the policeman had transformed into a frog.
Despite the serious error, Utah police have said they will continue using Draft Onebecause it saves them many hours of work. Sergeant Keel claims that he spends 1 or 2 hours a day writing reports, and with AI you save between 6 and 7 hours a weekwhich can be spent on patrol. This software costs $30 per month per agent.
He also emphasizes that the system has not failed, since all reports are reviewed by an agent before filing them, and that is how this was discovered. AI failure.
An AI with biases and hallucinations is not trustworthy
The problem for many people is not these absurd errors, which are quickly detected, but the subtle ones, which are difficult to see.
It is known that all AIs have biasescriminalizing certain races, ages and genders more than others. And even if an agent reviews the report, there is a risk that they will end up trusting the AI, just take a quick look, and miss changes that turn an innocent person into a guilty party, or the other way around.
The AI that turned a police officer into a frog It’s just an anecdote, like Google’s AI glue pizza. The worrying thing is that critical tasks are entrusted to what is a first-generation AI that still makes many mistakes.