AI models fail 99% of the time in complex financial calculations

The technology company Saturn analyzed how the most common models of artificial intelligence (ChatGPT, Claude, Copilot, Grok, Gemini and others) cope with solving financial problems. In the answers to averaged questions, they were wrong in 57% of cases, on more complex ones in 99%. The results of the study are reported by the Financial Times.

Researchers tested AI models on a list of more than 100 finance-related questions. Questions could be repeated up to five times. Experts note that neural networks were mistaken in calculations, ignored, for example, upcoming changes in tax legislation or invented norms. Chatbots were best at budgeting.

“Millions of people trust artificial intelligence models in financial matters, but they get the wrong answers that can cause them to lose their money,” said Amal Jolie, CEO of Saturn. Experts note that the best according to the results of the study was the model Claude Opus 5 in the mode of “reasoning”.