Artificial intelligences can process huge amounts of data in seconds, generate texts with surprising fluidity and even create realistic images. However, a new study has revealed a surprising limitation, that most of these systems They have serious difficulties in reading analog watches.
According to researchers, analyzed models misunderstand the time in 75% of cases, which highlights their problems to handle basic visual tasks.
The study published in arXiv was carried out by the Edinburgh Universitywhere experts tested the ability of several multimodals to interpret watches and calendars. Among the analyzed models were GEMINI 2.0 from Google, GPT-4o of OpenAI and Claude 3.5 of Anthropic, among others.
The results made it clear that, although these systems have advanced in complex reasoning, They still stumble upon such common tasks as reading the time.
IA have difficulty reading the spheres of the clock
The researchers used images of watches with different designs, with Roman numbers, stylized spheres, with and without a second, among others. In most cases, AI failed to identify the time correctly.
The figures showed that even the best models succeeded less than 25% of the time, with more frequent errors in stylized clocks or unconventional numbering.
Generated with Ia
One of the hypothesis of the Edinburgh team is that IA have trouble recognizing the position of the hands in relation to the sphere. Unlike humans, who learn to read watches from an early age, these systems depend on previous patterns and data, which seems not enough for this specific task.
The experiment was not limited to watches. The ability of AI models to interpret calendars was also evaluated. They were asked questions such as «What day of the week is New Year?», Or «What is the 153rd of the year?» Although the results were somewhat better, errors remained notable, even the most precise models failed in 20% of cases.
One of the most precise systems in this area was Openai GPT-01which succeeded in 80 % of the responses related to dates and calendars. However, the lack of reliability in basic issues demonstrates that there is still work to do to improve the visual understanding of these models.
Why does this limitation matter?
It may seem a minor failure, but the inability of AI to interpret watches and calendars has significant implications. In environments where automation requires a precise interpretation of visual data, such as tasks programming or assistance to visual disabilities, these types of errors can generate serious problems.
According to Rohit Saxena, leader of the study: «Our findings show an important deficiency in AI’s ability to perform basic skills for people. These deficiencies must be addressed so that AI systems are successfully integrated in practical and urgent applications, such as programming, automation and assistance technologies. «
His colleague Aryo Gema adds: «Today, research in AI is usually focused on complex reasoning tasks, but, ironically, many systems still have difficulties in performing simpler daily tasks.»
This study, whose results will be presented at the International Conference on Learning Representations (ICLR) in April, opens the door to new research. Solving this type of deficiencies will be key to developing more reliable and versatile systems.
In addition, this finding adds to other recent investigations that have demonstrated the limitations of current models. For example, an analysis of the Tow center revealed that the search engines driven by AI generate incorrect information in 60% of cases.
These failures highlight the importance of continuing to improve the precision and understanding of artificial intelligence before depending completely on it in critical tasks.
Know How we work in NoticiasVE.
Tags: Artificial intelligence