Source: French to English Tester Published on: 2026-05-18
Source: The Conversation – in French– By Serge Darolles, Professor of Finance, Université Paris Dauphine – PSL
Hallucinations, data poisoning, algorithmic monoculture: the vulnerabilities of AI applied to finance are real. Above all, it is not certain that all these new risks are adequately understood both by the direct stakeholders in the sector and by the regulatory authorities. How can we better take these various flaws into account?
Should we trust artificial intelligence (AI) to manage our savings? The question is no longer science fiction. The vast majority of players in the French financial markets already use AI or plan to do so, according to the Autorité des marchés financiers. However, the vulnerabilities of these systems remain largely underestimated.Among regulators and finance professionals, a shared observation emergesA: before entrusting finance more to AI, the risks must be precisely measured.
The enthusiasm for AI is understandable. Language models (LLM) can analyze thousands of news articles, analyst reports, and market data in a few seconds. Some investment funds whose teams have developed AI skills show performance superior to that of their peers,as shown by recent analyses.
But this observation must be nuanced. The gains observed are concentrated in discretionary funds, that is to say those where AI assists an experienced manager rather than replaces them. The winning formula seems to be “the human with the machine,” not “the machine instead of the human.”
Also to read:
Can AI better identify zombie companies and save more of them?
Structural vulnerabilities
AI vulnerabilities are not accidental; they are structural. According to Wassim Bouaziz, AI security specialist at Mistral AI, “language models are intrinsically unstable. A minimal disturbance in the input data, such as adding a few spaces in a text, for example, can completely reverse the output produced.”
The phenomenon of hallucination, whereby a model generates entirely fabricated information, has already caused measurable damage. More than a thousand court decisions have thus been influenced by hallucinated content produced by LLMs,according to a specialized site that lists these cases.
Even more worrying, these systems are vulnerable to deliberate attacks. Prompt injection involves inserting hidden instructions into the data processed by the model. A notable example: an invisible text embedded in a CV caninfluence the recruitment recommendations made by ChatGPT. Transposed to finance, the risk is evident. An AI agent consulting news sites or databases could have its decisions hijacked by malicious content.
The poisoning of training data represents a more insidious threat.A 2024 studyhas shown that it was possible to contaminate 1% of a model’s training data at a negligible cost by purchasing expired domain names whose content was indexed in the training corpora. In a sector where a bias introduced into a model can result in billions of euros of directed transactions, this type of attack constitutes a systemic risk.
The trap of algorithmic monoculture
Beyond targeted attacks, a structural risk emerges from the massive adoption of the same technologies. Language models are trained on largely similar corpora derived from the internet, carrying the biases that come with it. When all the players in a market use tools based on the same architectures and the same data, their decisions tend to converge. This algorithmic monoculture can amplify market movements. Indeed, if all models err in the same direction at the same time, losses spread at the system-wide level.
The precedent exists. Theflash crashfrom 2010, during which the Dow Jones index lost 9% in a few minutes due to interactions between algorithms, illustrates what a correlated failure produces. With the widespread use of generative AI, whose internal mechanisms are much more opaque than those of classical trading algorithms, the risk of a systemic event of a new type cannot be ruled out.
A regulator facing an unprecedented challenge
The Financial Markets Authority (AMF) is closely monitoring this transformation. As it reminds us,Secretary General Sébastien Raspiller, “AI is still mainly used for internal functions (research, compliance, analysis) and little for direct advice to savers.”
But the boundary is shifting. One figure illustrates the reliability problem. When questioned about basic financial data of companies (net debt, results), the various major AI models did not producethat only a small percentage of correct responses. Such an error rate would be unacceptable for a human advisor.
The question of responsibility remains open, because an algorithm cannot be sanctioned. When an AI agent makes a transfer or places a stock market order based on a poisoned instruction, the chain of responsibilities becomes difficult to establish. The European regulation on AI classifies certain financial applications among “high-risk” systems, requiring transparency, explainability, and human supervision. It remains to be seen whether these requirements can be met in practice when decisions are made in a matter of milliseconds.
Explainability, a trust issue
In asset management, the ability to justify an investment decision is not a luxury,it is a fiduciary dutyHowever, language models operate like black boxes. They cannot explain why they recommend one action rather than another.
As several conference speakers pointed out, the most advanced management companies offer responses to this. Some, for example, run models in parallel to detect biases. Others break down the decision-making process into dozens of sub-tasks assigned to specialized agents in order to better identify the source of potential failures. Finally, some prevent novice analysts from using AI to preserve learning through experience.
This last point highlights a paradox. AI is transforming a profession while simultaneously potentially impoverishing the transmission of skills. If tomorrow’s professionals immediately delegate the financial decision-making process to a machine without mastering the fundamentals, they lose the critical capacity needed to detect its errors. The training challenge is therefore inseparable from the technological issue. It is not just about training in AI, but about training to think in the presence of AI.
Investing in public research
So, how far can we entrust finance to AI? The question posed here does not call for a binary answer. AI is already in finance, and it will remain there. What remains to be determined are the conditions of its deployment. What supervisory mechanisms, what transparency, what role for human judgment?
The research work carried out at Paris Dauphine University – PSL aims precisely to provide the necessary evidence so that this debate is not hijacked by commercial promises alone.
This article is part of the file “When finance bets on AI, who remains in control?” created by Dauphine Lighting,the online scientific media of Université Paris Dauphine – PSL.
The authors do not work for, do not advise, do not own shares in, do not receive funds from any organization that could benefit from this article, and have declared no other affiliation than their research institution.
–ref. Entrusting finance to artificial intelligence: what are the real risks?https://theconversation.com/entrusting-finance-to-artificial-intelligence-what-are-the-real-risks-281939