Ride the Lightning

Allen & Overy Rolls Out GPT-based Legal App Harvey Firmwide

February 21, 2023

LawSites reported on February 17 some extraordinary news.

Allen & Overy, one of the world’s largest law firms, announced that it has integrated the legal artificial intelligence product Harvey into its global practice, where it will be used by more than 3,500 lawyers across 43 offices operating in multiple languages.

Harvey uses the GPT-3 technology (not ChatGPT) to enable lawyers to create legal documents or perform legal research by providing simple instructions using natural language.

Allen & Overy (A&O) describes itself as “the first law firm to use generative AI that’s based on OpenAI’s GPT models.” Without question, other law firms will adopt similar technology – Harvey’s founders report that they are working with other firms which are preparing to deploy Harvey, some firmwide and some for specific practice areas.

A&O began a trial of Harvey in November 2022. By the end of the trial, some 3,500 A&O lawyers had asked Harvey approximately 40,000 queries for their day-to-day client work.

A&O has used Harvey in 50 different languages and 250 practice areas across all its offices. “It’s almost a fourth of the firm daily and about 80% of the firm monthly,” according to one of Harvey’s founders.

David Wakeling, A&O partner and head of its Markets Innovation Group, which led the Harvey trial, said the technology is a game-changer.

“I have been at the forefront of legal tech for 15 years but I have never seen anything like Harvey,” Wakeling said. “It is a game-changer that can unleash the power of generative AI to transform the legal industry. Harvey can work in multiple languages and across diverse practice areas, delivering unprecedented efficiency and intelligence. In our trial, we saw some amazing results.”

Harvey will be fine-tuned for each firm that deploys it, according to founders Winston Weinberg and Gabriel Pereyra.

“We develop these different systems for specific use cases and for specific firms, so you’ll have the specific model for A&O which becomes fine-tuned for them,” Pereyra said. “You can even specialize it more than that, where you can get specific models for cases — you can have a case where you can have a specific client matter or specific litigation and the model is fine tuned for that litigation or transaction.”

Harvey’s founders say that Harvey is trained over at least three types of data, starting with the general internet data that underlies the GPT model. Harvey is then further trained against general legal data, such as case law and reference materials. Finally, it is fine-tuned against the law firm’s own data, such as its historical work product, templates, etc.

Additionally, it can be fine-tuned using data which involves a specific matter or client.

To ensure confidentiality, when Harvey is trained for a specific firm, that model stays specific to that firm and is not used as a base model when deploying Harvey elsewhere.

“We firewall it based on the firm,” Pereyra said. “Any training that A&O does on their model just makes their model better. And that’s kind of the point. Right? It’s what makes A&O a unique law firm — the feedback from their attorneys is going to make their model look significantly better, or at least different, than another firm.”

According to the founders, within each law firm, there will be different models protected by different sets of permissions and firewalls.

“You can’t just take all of A&O’s client matters and use the same model because you don’t want leakage, even within the same firm, across client matters, so we’re being extremely careful of firewalling off all of these different things,” Weinberg said.

Harvey’s firm-specific training is like teaching an associate the firm’s unique practice.

Some users of ChatGPT for legal purposes have noticed its tendency to “hallucinate” — to make up answers from whole cloth. This seems troubling since we are placing so much trust in AI.

Pereyra and Weinberg maintain that Harvey’s method of fine tuning the AI dramatically reduces occurrences of hallucinations and, in highly context-specific applications, eliminates them almost entirely.

For contract review, for example, Harvey can reduce hallucinations “basically to zero.” In fact, Pereyra said, the error rate is lower than for review by a contract attorney.

Entirely reassuring? No.

Pereyra and Weinberg say they are still not demonstrating the product publicly. But they expect to have several new announcements of partnerships and capabilities coming within the next few months.

We should all watch carefully. Hallucinating AI is more than a little scary!

Sharon D. Nelson, Esq., President, Sensei Enterprises, Inc.
3975 University Drive, Suite 225, Fairfax, VA 22030
Email:   Phone: 703-359-0700
Digital Forensics/Cybersecurity/Information Technology
https://senseient.com
https://twitter.com/sharonnelsonesq
https://www.linkedin.com/in/sharondnelson
https://amazon.com/author/sharonnelson