Beyond the Pilot: What legal AI needs to earn trust

Vasu Aggarwal

Some of the best conversations about legal AI start after the demo ends.

That was the thinking behind the dinner we co-hosted with our friends at Stubbs Alderton & Markiles at Eataly in Los Angeles last Tuesday. There were no slides or pitches,  just lawyers and guests around a long table, good Italian food, and an honest conversation about what it really takes to bring AI into a legal practice.

People came with examples from their own work, and with doubts to match.
Like who checks the work? And what happens to the client's data? If a task takes less time, how should a firm charge for it? And how much should a lawyer hand over to a system that can act on their behalf?

None of these can be settled by a product demo. As the evening went on, the talk drifted from how models generate answers to something harder: what it takes to use those answers responsibly.
If I had to sum up the night in one line, it would be this. Producing an answer and earning the right to act on it are two very different problems.

Here is what stayed with me.

Reading is one thing. Acting is another.

A lot of the conversation was about what AI already does well: helping people read. Cases, contracts, research, large sets of records. In almost every example someone shared, the value came from making material easier to examine, compare and question.

Agents change the stakes. A system that can make a reservation, send an email or complete a transaction saves you the work of carrying a task to the finish line. It can also make a mistake that reaches someone else before you notice.

For lawyers, that line matters a great deal like a rough draft can be fixed in review but an unchecked message sent to a client can damage trust in a way that is much harder to undo. Just because a system can complete an action doesn't mean it should be allowed to do so unsupervised.

So the real question for firms isn't "are we ready for agents?" It's more specific than that. Like which actions can be delegated? Or Which need sign-off? Or How will an error be caught, and who owns the fix?


"Closed system" isn't an answer

Data privacy came up again and again, and for good reason. Phrases like "closed system" sound reassuring, but they tend to leave the important questions hanging.

Where is the data processed? Who can access it? Is it retained? Is it used to train a model? What has the provider actually committed to, and how can a firm check?

We talked through different approaches, from cloud infrastructure to on-premise setups. What became clear is that a familiar vendor name or a broad security label isn't a complete answer on its own.

The better conversation starts with the information itself and the obligations that come with it. A tool used for public research carries very different risk from one handling confidential client documents. That difference should shape the questions a firm asks, the tools it picks and the rules it sets for its lawyers.

Better work, not just faster work

The economics of AI usually get boiled down to a simple equation: fewer hours, lower fees.

Around our table, it wasn't that tidy. People talked about better work product as a big part of the value. And interestingly, nobody described a uniform push from clients to cut fees just because AI was involved.

That distinction is worth holding onto. A faster first pass can free up time for a lawyer to stress-test an argument, chase down an inconsistency or weigh an alternative. The payoff might show up as a sharper recommendation rather than a shorter bill.

Efficiency still raises hard pricing questions. Fixed fees came up as one way to capture the gains on repeatable work. But nothing that night suggested the billable hour is dead, or that one pricing model will fit every matter.

The harder job is explaining what the client actually gets, and why it's worth paying for.


Staffing is really a supervision question

When the talk turned to junior lawyers and non-lawyer staff, nobody reached for an easy headcount prediction. I appreciated that.

Some routine work will likely be organised differently as AI gets better. People discussed setups where non-lawyer staff use AI for standardised tasks, with lawyers responsible for review and sign-off. Complex advisory work is a different story, one that turns on experience, judgment and accountability.

That puts supervision at the centre of how the workflow is designed. Who can tell when an output is wrong? What needs a lawyer's eyes? Where does the task stop being about assembling information and start being about legal judgment?

Shifting work between people and software doesn't make those questions go away. If anything, it makes the answers matter more.

The best examples were the specific ones

My favourite part of the evening was hearing how people are actually using these tools day to day. It was a welcome counterweight to the big predictions.

The examples were refreshingly concrete: structured reading of legal decisions, finding familiar contract language, combing payroll records for anomalies, drafting time entries from the day's work. One person described using AI to figure out which questions to ask a technical counterpart.

Each of these is a different task with a different bar for success. A useful summary, a plausible clause and a flagged anomaly all need a different kind of checking. The right starting point is the work itself: what information is available, what the lawyer needs to decide and how the result will be reviewed.

For firms evaluating AI, that's a far more useful place to begin than a promise to transform the whole practice at once.

Trust is still the test

No matter where the conversation wandered, it kept coming back to clients and trust.

AI can help a firm prepare, investigate and respond. It can't make an unsupported promise safe, and it can't turn an impersonal interaction into a trusted relationship just by making it faster.

That's a useful constraint for firms, and for those of us building legal technology too. Claims about revenue, returns and transformation need care. The work has to back up the promise.

What we talked about over dinner was practical: giving lawyers more room to understand a matter, ask better questions and serve their clients well. Getting there means making clear decisions about data, review, supervision and responsibility.

Access to AI is just the beginning. Earning trust in how it's used is the real work that follows.

A big thank you to the team at Stubbs Alderton & Markiles for co-hosting with us, and to everyone who pulled up a chair and joined the conversation. We're already looking forward to the next one.