Rules Without Referees: Who Is Qualified to Mediate an AI Dispute?

Dispute resolution institutions are not ordinarily accused of moving too quickly. In the case of artificial intelligence they have moved first, publishing rules for a category of disagreement that had barely begun to appear on their dockets. The frameworks arrived before the caseload. That is an unusual sequence, and it has produced an unusual gap.

JAMS published its Rules Governing Disputes Involving Artificial Intelligence Systems in April 2024, effective the following June, the first ADR provider to do so. The World Intellectual Property Organization operates a dedicated ADR program for artificial intelligence disputes, drawing on its existing expertise in intellectual property and technology matters. AAA-ICDR has gone further in one respect and made an AI arbitrator available for a narrow class of low-value, documents-only construction matters. The institutional architecture, in other words, exists.

What the architecture does not supply is the person who sits in the room.

What an AI dispute actually looks like

The phrase suggests something exotic. In practice these disputes arrive inside ordinary commercial agreements, and four patterns account for most of them.

The first is the performance claim that does not survive contact with production. A capability demonstrated on curated data behaves differently against a customer’s own, and the disagreement becomes a question of what was warranted, what was aspirational, and which document controls. The second is provenance. Training data was licensed, scraped, purchased, or inherited through an acquisition, and the representations given about it are now being tested. The third is automated decision-making alleged to have produced a biased or unlawful outcome, in hiring, in lending, in pricing, or in claims handling. The fourth is allocation, meaning the indemnity, liability cap, and insurance argument that follows once anyone concedes harm occurred.

Model drift threads through all four. A system that performed as promised at signature and performs differently eighteen months later raises a question most contracts were not drafted to answer, since the obligations were written as though software were a fixed object.

What the frameworks solve

The institutional rules address a genuine problem, and it is worth being precise about which one.

The central difficulty in an AI dispute is evidentiary. The thing in contention is frequently a trade secret, and examining it means examining the model weights, the training corpus, the evaluation methodology, and the deployment configuration. The JAMS rules meet this directly, providing for the hardware, software, models, and training data at issue to be made available to experts in a secured environment, with a protective order applying by default rather than on application. They also provide for the appointment of neutrals with knowledge of artificial intelligence, which is a quiet acknowledgment that the ordinary panel was not assembled with these matters in mind.

Confidentiality does the rest of the work. A public proceeding about model behavior is a disclosure event, and for most parties the prospect of it is a stronger argument for private resolution than cost or speed will ever be.

What the frameworks cannot solve

Rules allocate authority and set procedure. They do not confer understanding, and this is where the gap sits.

An AI dispute asks a neutral to hold several unrelated bodies of knowledge at once. The technical substance, including how a model is trained and evaluated and why its performance moves. The data governance layer, including processing terms, provenance documentation, and the regulatory obligations attaching to automated decisions. The commercial layer, meaning how enterprise software is actually bought, and the well-known distance between what is shown in a demonstration and what is incorporated into an order form. And the human layer, because the parties are usually a customer who feels misled and a vendor who believes it delivered exactly what was specified, and both are correct within their own frame.

A neutral fluent in only one of those four spends the session being educated at the parties’ expense. The scarcity is not of rules. It is of people who can read a data processing addendum, an evaluation report, and an order form in the same afternoon and see how the three fit together.

Where the machine stops

The AAA-ICDR program is instructive precisely because of how carefully it is bounded.

An AI arbitrator, trained on more than 1,500 prior awards, is available on an opt-in basis for low-value construction matters decided on the documents alone. Those are conditions under which the question is well defined, the record is closed, and the outcome is largely a function of applying settled principles to documents. It is a sensible use of the technology and it maps the boundary rather than removing it.

Almost nothing about a substantive AI dispute meets those conditions. The parties are misaligned about what was promised, the harm is contested, the relationship has to survive, and the resolution that works is usually a renegotiated commercial arrangement rather than an award. Those matters turn on reading a room, on understanding what each side can accept internally, and on cultural and organizational dynamics that shape how a concession can be made without anyone losing standing. Ellen Kim has written on that theme with Mo-Yun Lei Fong for both SHRM and CPR Speaks, arguing that conflict resolution tooling misreads conflict wherever it is blind to the culture the conflict occurs in. The argument applies with particular force here, since the disputes most likely to be handed to an automated process are the ones where a human judgment about what people can live with is doing the most work.

The qualification that is actually being asked for

The useful question is not whether artificial intelligence will change dispute resolution. It plainly has, and the institutions have already answered it.

The useful question is what a party should require of a neutral when the subject matter is a model, the evidence is a trade secret, the harm is statistical, and the commercial relationship must continue regardless. The answer is a combination that remains uncommon: formal grounding in AI governance and privacy, genuine experience of how enterprise software is negotiated and sold, and the process skill to hold a room in which both parties are technically fluent and entirely unable to speak to one another. Institutions write the rules. Someone still has to be qualified to sit in the room, and that qualification is being defined now, by the people who take these matters early.