ChatGPT, the chatbot that started the public’s obsession with generative AI, turned two years old over the weekend. 

Prior to November 30, 2022, the day ChatGPT was publicly launched, most people—including most legal professionals—had never heard of GenAI. However, that minor fact didn’t stop the legal industry from proclaiming the technology the future of legal work. 

The ensuing experiments and their outcomes ran the gamut. Some were highly successful; others, not so much. Has the industry learned the right lessons from those successes and failures? 

Here’s a brief summary of how far (or not) we’ve come as an industry with GenAI in the past two years. 

Legal GenAI: How it Started vs. How it’s Going 

The legal industry almost immediately started questioning if ChatGPT could be used for legal work. Some rightly pointed out that the correct question was not whether it could be used in legal, but whether it should be. Over time, the discussion shifted away from ChatGPT to GenAI in general, as most people started to understand that it was the underlying AI models that made the tool so impressive and that ChatGPT itself was not a tool meant for legal work—a lesson that too many have learned the hard way. 

Perhaps the most significant overall shift from 2023 to 2024 is that very few discussions now revolve around “if we implement GenAI,” as they did in the early days. Now, the discourse is built around “when and how we implement GenAI.” In tandem with that shift, many—though not all—of legal’s notions about the use of GenAI have also evolved. 

What Hasn’t Changed 

While many of the early (mis)understandings around the use of GenAI have shifted, a couple things haven’t changed. 

2023: Hallucinated case briefs 

2024: Hallucinated case briefs 

When the first case involving an attorney submitting fake case citations generated by ChatGPT to a court hit the news in May 2023, it rocked the world—and not just the legal world, as the story was first widely reported by The New York Times. Many were quick to blame the tech, but eventually sentiment swayed to realizing this was a case of human error, not a tech failure. The incident was shocking since it flew in the face of all attorney professional and ethical obligations. 

So, did legal professionals learn their lesson from this epic cautionary tale? Apparently not. 

Unfortunately, many similar instances have been reported since Mata v. Avianca made headlines a year-and-a-half ago. As recently as November 2024, an expert in—of all things—the constitutionality of criminalizing deepfakes is alleged to have submitted a sworn declaration to the Minnesota District Court that referenced two nonexistent research articles. Meanwhile, a Texas lawyer was just fined $2,000 and ordered to complete GenAI-focused CLE after filing a brief that cited two nonexistent cases. 

2. GenAI vs. The Billable Hour 

2023: Will AI kill the billable hour? 

2024: Will AI kill the billable hour? 

Nearly every time a new technology has hit the market that promises to improve efficiency and save lawyers time, people have predicted the death of the billable hour. After all, how can attorneys continue to bill high hourly rates if they now have technology to do things for them more accurately and faster? 

Not surprisingly, the “death of the billable hour” talk was reignited in earnest with the advent of GenAI. Also not surprisingly, the billable hour is still alive and well, as it has been following each previous technological innovation. Still, the question continues to be raised all the time. 

Now, GenAI may be in a better position to wound the billable hour than past advancements, and there’s certainly more talk of hybrid billing arrangements—where some work is billed hourly, while other work is not—than there has been before. However, the billable hour in legal is far from dead. 

What Has Changed 

Thankfully, many of the early-days notions surrounding GenAI in legal have changed. This is a good thing—not because the ideas were wrong necessarily, but because the technology itself is rapidly advancing and the discussions around it need to mature accordingly. 

Here are just some of the shifts we’ve seen in the last two years. 

3. Regulation, Not Bans 

2023: ChatGPT bans 

2024: AI regulation 

Between inherent mistrust of new, unproven technologies and widely reported instances of hallucinated case citations, many organizations were quick to entirely ban the use of ChatGPT, if not all GenAI tools. Eventually, however, it became clear that some people, including lawyers, were going to use these tools whether or not they had explicit permission to do so. The GenAI genie was out of the bottle, leading to the widespread understanding that we had to learn to live with the technology rather than insist on pursuing futile attempts to eradicate it. 

Permitting responsible AI use started small, with individual organizations putting use policies in place. Shortly thereafter came efforts to regulate GenAI, with varying degrees of success. Notably, the EU AI Act entered into force on August 1, 2024, with various enforcement dates going into effect throughout 2025 and beyond. While the U.S. has yet to enact AI regulation at a national level, Colorado was the first state to pass an AI Act in June 2024, and the White House signed an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence in October 2023, though its future is uncertain. Other global jurisdictions have also enacted AI regulations, with many more currently considering it or in the process. 

4. The Proliferation of LLMs 

2023: GPT-4 is all the rage 

2024: Legal embraces a multi-model approach 

ChatGPT, which originally ran on OpenAI’s GPT-3.5 large language model (LLM), may have left people breathless, but they were truly blown away when they were introduced to its successor model GPT-4. Legal’s first public encounter with the advanced model came when Casetext (now part of Thomson Reuters) released CoCounsel, one of two original AI legal assistants.  

Initially, OpenAI’s LLMs were the most popular among legal tech providers incorporating GenAI. Over time, however, both developers and users came to see the benefit of a multi-model approach—one major reason being that different LLMs proved to excel at different tasks. While OpenAI’s models are still undoubtedly popular, many tools now incorporate multiple different LLMs, with some even allowing users to choose between them depending on the nature of the work being performed. 

5. The Big and the Small of LLMs 

2023: Increasingly larger LLMs 

2024: A focus on small language models 

In 2023, as developers released increasingly powerful LLMs, “bigger is better” was the mantra of the day. When GPT-4 was released, it was rumored to have over 1.7 trillion parameters—a staggering size at the time. The idea was that models would increase in usefulness as they became bigger and more powerful. 

Flash forward to 2024, and it seems that bigger isn’t always better. In fact, even the largest AI developers started releasing small language models (sometimes referred to as SLMs, though the term hasn’t fully caught on). Small may be a bit of a misnomer, as these models still tend to be quite large even though they’re dwarfed by their larger counterparts. Smaller models have proven to be more effective for some specific, niche purposes. In line with the multi-model approach, legal tech is increasingly relying on a mix of LLMs and SLMs. 

6. Dealing With Hallucinations 

2023: Focusing on hallucinations 

2024: Benchmarking AI tools 

Spurred in large part by the ChatGPT case cite fiasco, the legal industry became almost fixated on hallucinations in 2023. Even if they didn’t fully understand what “hallucinations” in AI were, GenAI detractors quickly latched onto “but hallucinations” as an excuse to avoid the technology. The idea was that legal work was too important to tolerate less than 100% accuracy, never mind the fact that no human lawyer has ever been 100% accurate all the time. 

In reality, AI models sometimes hallucinate—it's in the very nature of how they work. The existence of hallucinations doesn’t make an AI tool inherently unfit for use. As 2024 went on, the legal industry started to understand that. Therefore, instead of a zero-tolerance threshold for hallucinations, the focus shifted to benchmarking tools, with a push for AI providers to validate their models to determine rates of accuracy. Ignited by a now-somewhat-controversial study out of Stanford HAI that found high hallucination rates in some of the most popular legal research products, the momentum for legal AI benchmarking is only gaining steam as we head into 2025. 

7. Domain-Specific AI  

2023: Fine-tuning models 

2024: RAG 

If it’s impossible to eliminate hallucinations, what can be done to limit them? Attempts to answer this question have been undertaken in earnest practically since ChatGPT came out. In 2023, the concept of fine-tuning LLMs—training a general model in domain-specific knowledge, such as legal—took center stage. While theoretically effective, fine-tuning turned out to be cost-prohibitive and therefore offered less widespread value than was hoped. 

Attention then turned to Retrieval-Augmented Generation (RAG)—improving the output of an LLM by providing it with context-specific outside information to ground it in a particular knowledge base, like legal. For a short period, some hoped that RAG was the magic answer to eliminating hallucinations, but that proved to be untrue. Still, RAG continues to be valuable and is now ubiquitous across most legal tech solutions as we close out 2024. Fine-tuning on top of RAG is also still worthwhile in the right circumstances. 

8. Courts’ Love / Hate Relationship With AI 

2023: Court-issued AI standing orders 

2024: Courts embracing AI 

Faced with the possibility of litigants filing court papers including fake case citations generated by AI and not checked by humans, certain judges immediately responded by issuing standing orders or similar directives requiring parties to disclose any use of generative AI—or, in some cases, any AI. Much like early organizational bans on ChatGPT, the moves were intended to prevent lawyers from using improper outputs by monitoring, and perhaps chilling, the use of AI entirely. 

To many, these standing orders were overkill, repetitive of attorneys’ obligations under Rule 11 and the Model Rules of Professional Conduct to ensure and certify that they have read and confirmed all cited sources before filing. Even as this sentiment grew, judges continued to issue similar edicts, with an estimated 200+ issued in state and federal courts to date, as well as in courts of countries like Canada and Australia. 

At the same time, though, other courts have been increasingly embracing GenAI. In December 2023, the UK, recognizing that the judiciary itself might use GenAI, issued official Artificial Intelligence (AI) Guidance for Judicial Office Holders, while just last month it was reported that court officials in Buenos Aires have been using ChatGPT to draft rulings and sentences (not without criticism, to be clear). In the U.S., judicial use of GenAI is still in the experimental stage, though certain court systems are actively exploring using GenAI to alleviate overburdened dockets and assist with urgent areas like evictions. 

9. To Prompt or Not to Prompt? 

2023: Prompt engineering is the new critical skill for lawyers 

2024: Is prompt engineering dead? 

The early peak of ChatGPT’s popularity came with a term that was new to legal—prompt engineering. The notion that legal would need to learn how to properly prompt AI systems in order to get the most out of them quickly took hold. The first bold move in this area came as early as February 2023, when UK firm Mishcon de Reya posted a job opening for a “GPT legal prompt engineer.” Viewed as part science and part art, prompt engineering quickly caught on as the next must-have skill in legal. 

Yet, never one to miss an opportunity to kill something off (see the billable hour), legal was just as quick to declare prompt engineering dead as it was to embrace it originally. In 2024, momentum grew around the notion that AI systems and tools would soon be so advanced that detailed prompts wouldn’t even be necessary. Instead, the focus shifted to semantic search, or the notion that advanced AI systems can understand the intent of search queries well enough to return good results, even without complex prompts. 

Today, the prevailing view is a hybrid of these approaches—legal understands that there’s still value in a well-crafted prompt, but they might not hire for a role strictly devoted to crafting it. Case in point: a recent job posting from Thomson Reuters for a “Legal AI Prompter / Evaluator,” a role which goes beyond just prompting and focuses on “crafting and refining prompts, developing methodologies for evaluating and grading LLM outputs, and integrating these technologies into our suite of legal products.” 

10. The AI Tool du Jour 

2023: Hot chatbot summer 

2024: The agent invasion 

ChatGPT, the tool that started it all, is a chatbot, so it should come as no surprise that chatbots saw a surge in popularity in 2023. Recognizing the limitations of public tools like ChatGPT for legal—namely, security concerns and the inadvisability of inputting confidential or privileged information—law firms started creating their own proprietary versions of ChatGPT to eliminate these concerns. News of firm chatbots came out on a nearly weekly basis and by fall of 2023, many firms had their own internal chatbots. 

In 2024, chatbots are now old news. Legal’s attention has instead been captured by agentic AI—AI systems capable of achieving goals by interacting with their environment, making decisions, and taking actions, all without direct human oversight or control. While agents first caused a stir in April 2023 and legal tech has been working on them ever since, the buzz surrounding legal agents reached a fever pitch in fall 2024 and shows no signs of abating any time soon.  

Bonus: AI + A2J 

2023: Touting robot lawyers 

2024: Concerted efforts to improve A2J with AI 

Given the efficiency gains and easy accessibility of GenAI tools, the potential for this technology to make meaningful strides in closing the justice gap were evident early on. The industry is still divided, however, on how much meaningful change GenAI will make in the access to justice (A2J) space. 

GenAI-powered legal self-help got off to a rocky start after the release of ChatGPT. People started theorizing about robolawyers that could change the system. Consumer legal services champion DoNotPay tried to make the fantasy a reality in January 2023 with plans to have a chatbot represent a defendant in a New York traffic case before an unknowing judge. Founder Joshua Browder took things a step further with what he called a “serious offer” of $1 million to any lawyer who would agree to rely solely on the company’s chatbot in arguing a case before the Supreme Court. Practical impossibilities aside, the move did not go well for DoNotPay, who subsequently pulled several of its legal-focused consumer products and overhauled its entire website to rebrand itself as “Your AI Consumer Champion” rather than its long-running “World’s First Robot Lawyer” motto following an FTC fine and settlement. 

While such early stunts may have initially been a setback for the A2J cause, other legal tech providers started making real efforts to allocate resources to bring AI to A2J. Take, for example, the recently launched Legal Innovators Incubator from Thomson Reuters, which provides legal aid organizations with a year of free access to CoCounsel tools, ongoing support and training, and more. 

There’s no doubt that there’s still a long way to go in adequately addressing the justice gap, and GenAI by itself is unlikely to fully solve the problem. It will, however, help address some of the major roadblocks, particularly as tech providers are showing increased commitment to creating real change. 

Looking Ahead to 2025 and Beyond 

As we enter the new year, the conversations surrounding legal GenAI continue to evolve and mature. A year feels like a lifetime given how quickly GenAI is advancing, so we can expect to have a whole new set of considerations to discuss when ChatGPT turns three. 

Legal got one thing right in 2023, though—GenAI is the future of legal work, just maybe not in the ways it expected at the time.