Two AI Agents Negotiated a Deal. No Human Was in the Room.
Two AI Agents Negotiated a Deal. No Human Was in the Room.
Part of “The AI You Don’t See” series by Akshay A. Walimbe
On May 6, 2010, at approximately 2:45 PM Eastern Time, the American stock market lost a trillion dollars.
Not over the course of a bad quarter. Not during a financial crisis that had been building for months. In minutes. The Dow Jones Industrial Average plunged 998 points roughly nine per cent of its value and then recovered most of it, all within a few minutes. A trillion dollars vanished, then reappeared, like a magic trick performed at the speed of light.
Except it was not magic. It was machines talking to machines.
Algorithms trading with algorithms. Automated systems reading signals from other automated systems, each one responding to the other’s behaviour in a feedback loop so fast that no human could intervene, comprehend, or stop it. By the time a person realised what was happening, it was already over.
They called it the Flash Crash. And when investigators spent years picking through the wreckage, they found something remarkable: a single trader in Hounslow, London, Navinder Singh Sarao, had used automated programs to generate roughly 200 million dollars in fake sell orders a technique called spoofing that triggered a cascade of algorithmic responses. According to the Department of Justice, his orders were “replaced or modified 19,000 times” before being cancelled. One person’s automated orders interacted with thousands of other automated systems, and the chain reaction erased a trillion dollars in market value.
Sarao was arrested in 2015, five years after the crash. He eventually pleaded guilty and in 2020 was sentenced to time served and one year of supervised release effectively home confinement. No prison time. A trillion dollar event, and the human at the origin of it avoided jail.
But here is the question that still has no answer: if it had been two AI agents autonomously negotiating a transaction, with no Sarao at the beginning of the chain, who would have been arrested?
The Age of the AI Agent
That question was abstract in 2010. It is not abstract anymore.
We have entered the age of agentic AI. Not the AI you are used to the chatbot that answers your questions, the autocomplete that finishes your sentences. This is different. Agentic AI systems do not just respond to prompts. They act. They plan multi step tasks, use tools, execute actions, check results, and adapt their approach when things go wrong. They do all of this without asking a human for permission at every step.
Goldman Sachs deployed an AI coding agent called Devin built by AI startup Cognition reportedly as one of its first “AI employees” in 2025. The company’s Chief Information Officer announced plans to deploy such agents “by the hundreds maybe eventually even by the thousands.” Devin does not just write code when asked. It plans, executes, reviews, and deploys autonomously.
Gartner predicted that by 2028, at least 15 per cent of day to day work decisions will be made autonomously through agentic AI, up from zero per cent in 2024, and that 33 per cent of enterprise software applications will include agentic AI by 2028. But Gartner also warned that over 40 per cent of agentic AI projects will be cancelled by the end of 2027 due to escalating costs, unclear business value, or inadequate risk controls. The agentic AI market, valued at roughly 7.5 billion dollars in 2025, is projected to reach 199 billion dollars by 2034.
These are not chatbots with better marketing. These are autonomous systems that can browse the web, fill forms, make purchases, schedule meetings, negotiate terms, execute trades, hire candidates, and approve or deny requests all without a human in the loop.
And increasingly, they are interacting not with humans, but with each other.
When Machines Deal With Machines
Consider what already happens in algorithmic trading. Over 60 to 80 per cent of equity trading in developed markets is executed by algorithms. In India, algorithmic trading now drives over 55 per cent of all trades, with algo participation in stock futures rising from 39 per cent in FY15 to 73 per cent in FY26, according to exchange data. These are machines making decisions thousands of times per second, based on patterns, signals, and the behaviour of other machines.
The Flash Crash was the first wake up call. But it was not the last. In August 2012, Knight Capital Group deployed a software update to its automated trading system. A technician forgot to copy new code to one of eight servers. According to the SEC’s investigation, that misconfigured server sent millions of child orders into the market, resulting in 4 million executions across 154 stocks for more than 397 million shares all within approximately 45 minutes. Knight Capital lost 440 million dollars. The company was effectively destroyed acquired by Getco within months.
Forty five minutes. One server. No human made any of the trading decisions.
Now extend this beyond stocks. AI agents negotiating procurement contracts. One company’s agent finding the best price, another adjusting its pricing in real time. Supply chain agents rerouting shipments, insurance agents adjusting premiums all talking to each other, all making decisions.
This is not science fiction. Multi agent frameworks like CrewAI and Microsoft’s Agent Framework are used in production at hundreds of companies, including LinkedIn and Uber. The infrastructure for agent to agent interaction exists. It is being deployed right now.
The Liability Vacuum
So let me ask you the question that nobody has answered.
When two AI agents negotiate a deal and that deal harms you, whose fault is it?
Is it the company that deployed Agent A? Is it the company that deployed Agent B? Is it the developer who built the underlying model? Is it the cloud provider whose infrastructure ran the computation? Is it the person who set the parameters that Agent A optimised against?
Current law was not built for this. Contract law assumes that parties to a contract are legal persons humans or corporations who can form intent, understand terms, and be held accountable for breaches. An AI agent is none of these things. It is a software process executing instructions that were themselves generated by another software process trained on data that may have been collected years ago by people who no longer work at the company.
Tort law requires establishing duty, breach, causation, and damages. When an AI agent causes harm, who owed the duty? The company that deployed it? The company that built it? The company whose data trained it? The causation chain in agent to agent interactions can be so tangled that attributing responsibility becomes practically impossible.
The EU AI Act, rolling out through 2026, classifies high risk AI systems and requires human oversight. But it was drafted before the current wave of agentic AI. Its framework assumes a world where AI systems make recommendations and humans make decisions. Agent to agent interactions where the human is not in the room at all fall into a gap the law has not yet closed.
India’s position has gaps, though the picture is more nuanced than a simple absence of regulation. In the financial sector, SEBI’s February 2025 circular on algorithmic trading requires all algo orders to be tagged with unique IDs, mandates broker oversight, and sets orders per second thresholds. The RBI’s FREE AI framework, released in August 2025, establishes seven principles for responsible AI in banking and finance. But these are sector specific frameworks. The broader India AI Governance Guidelines of 2025 remain voluntary. The proposed AI (Ethics and Accountability) Bill, introduced in December 2025 as a private member’s bill, has not been passed. There is no general framework for agent accountability, no overarching liability regime for autonomous AI decisions, and no single regulatory body with the mandate or capacity to investigate agent to agent harms across sectors.
The Speed Problem
There is another dimension that makes everything harder: speed.
Knight Capital’s disaster lasted 45 minutes. But modern AI agents can execute thousands of actions per second. In the time it takes a compliance officer to read a notification that something has gone wrong, an AI agent could have completed an entire chain of transactions, each triggering the next, cascading outward.
Now imagine this beyond financial trading. An AI insurance agent adjusts your premium based on data from an AI health monitoring agent, which received data from an AI wearable that inferred your heart rate patterns suggest a higher risk profile. Your premium goes up. You do not know why. The chain of AI to AI communication is opaque, distributed across three companies, governed by no single legal framework.
Who do you complain to?
I build AI systems. I understand the appeal of automation. But there is a difference between automation and abdication. When two humans negotiate a deal, there are social norms, legal frameworks, and courts that govern the interaction. When two AI agents negotiate a deal, we have stripped away all of those layers.
The question is not whether something will go wrong. The Flash Crash told us things go wrong. Knight Capital told us things go wrong catastrophically fast. The question is: when two machines make a deal that harms you, whose fault is it?
Right now, the honest answer is: nobody knows.
I’m have written a book about exactly this how AI and automated systems make decisions about your life, where accountability disappears, and what we can do about it. If you want to know morea about this book or order a copy, you can do it here: https://akshaywalimbe.com/beyond-bias/