Assessing AI's Role in Legal Tasks for Cryptocurrency Cases

Large Language Models (LLMs) like ChatGPT could improve access to legal help. However, there is little research on how well they can perform Legal Tasks. This study focuses on securities cases involving cryptocurrencies to see how LLMs can support the legal process. We look at two main questions: Can an LLM understand which laws might be broken based on a given fact pattern? Is there a difference in juror decisions based on complaints written by a lawyer versus those written by an LLM?

To answer these questions, we provided scenarios from real cases to the GPT-3.5 model and assessed its ability to identify potential legal violations. We also had mock jurors evaluate complaints drafted by both the LLM and human lawyers. Our findings showed that, although GPT-3.5 had weak Legal Reasoning skills, it performed better at drafting. Jurors' decisions were not significantly influenced by who wrote the complaint. Although LLMs cannot yet replace lawyers, their drafting skills could help reduce the costs of legal services, making access to justice easier for more people.

#The Problem of Cryptocurrency Scams

Cryptocurrency scams have been reported widely, with victims losing significant amounts of money. Despite the growing number of scams, government enforcement actions have been limited. The U.S. Securities and Exchange Commission (SEC) has only filed a small number of cases related to cryptocurrency fraud. Many victims have sought justice through class action lawsuits, but the number of cases filed is still small compared to the number of scams. This situation suggests that many victims have limited access to legal help when dealing with these scams.

Many people who invest in cryptocurrencies are often from lower-income backgrounds and have been targeted during price surges. Data indicates that a high percentage of these individuals cannot get the legal help they need due to costs. Given this context, using AI tools might be an effective way to assist victims of cryptocurrency scams.

#Research Questions

We want to find out if LLMs can accurately identify laws potentially being violated based on a set of facts from a securities case involving cryptocurrencies. Additionally, we want to see if juror decisions differ based on complaints authored by a lawyer versus an LLM.

#Methods

To explore these questions, we used the GPT-3.5 language model and provided it with fact patterns from real cases. We also conducted a study where mock jurors assessed complaints written by either the LLM or human lawyers. The goal was to evaluate the LLM's capabilities in legal reasoning and drafting.

#Introduction to LLMs

LLMs are AI systems designed to understand and generate human language. They can answer questions and create text that appears written by a human. The latest advancements in this field have focused on models like GPT, which has been trained on vast amounts of data. The training allows LLMs to perform well in various tasks, even if they have not been trained specifically for a particular one.

#Using OpenAI's LLMs

OpenAI's LLMs, like ChatGPT, allow users to experiment with AI-generated text. Users input a "prompt," and the model generates responses based on that prompt. However, there's no set formula for creating effective prompts, which can make using the models challenging. Users can also fine-tune models by training them on additional relevant data, while others use them without any extra training.

#LLMs and Legal Tasks

There is limited research on how LLMs can perform in legal settings since most studies focus on simpler AI models. However, some researchers have explored how models like ChatGPT perform in legal contexts by testing their skills on exams and legal reasoning tasks. Previous studies showed that while LLMs can generate legal text, they struggle with legal reasoning.

#Legal Context of Cryptocurrency Securities Violations

Cryptocurrencies are digital assets that have gained popularity since Bitcoin was first introduced. These assets can sometimes be classified as securities, which comes with regulatory requirements. The SEC has determined that many cryptocurrencies fall under its regulatory jurisdiction. Violations can lead to enforcement actions, and victims of scams may seek recourse through lawsuits.

#Navigating Securities Law

U.S. securities law governs the offer, sale, and purchase of securities, and the SEC is responsible for enforcing these laws. When violations are suspected, the SEC investigates and can bring civil enforcement actions against the parties involved. Private individuals can also file class action lawsuits if they believe they have been harmed by illegal securities practices.

#Findings on LLM Performance

#Legal Reasoning Capabilities

In our study, we assessed GPT-3.5's ability to identify violations of U.S. law based on provided fact patterns. The overall performance was poor, with the model often missing key violations. The analysis revealed a tendency toward providing correct answers when it did identify violations, but it frequently overlooked other relevant laws.

#Legal Drafting Skills

On the other hand, when it came to drafting legal documents, the LLM showed more promise. Mock jurors indicated that complaints generated by the LLM were convincing. The decisions made by jurors did not significantly differ between the complaints authored by the LLM and those written by human lawyers, showing that the LLM could produce adequate legal text.

#Evaluating Juror Decisions

We also looked at how jurors assessed the complaints. Their agreement on whether the charges were proven did not significantly differ based on whether the complaint was drafted by a lawyer or an LLM. Both sets of complaints managed to convince jurors effectively, indicating that the quality of drafting from the LLM is satisfactory for initial legal proceedings.

#Implications for Access to Justice

Given the limitations in GPT-3.5's legal reasoning skills, it is unlikely that LLMs will completely replace lawyers in the near future. However, their drafting capabilities could greatly enhance access to legal services for those who may not be able to afford a lawyer. The ability to create legal documents quickly and at a lower cost can help fill gaps in the legal system, particularly for individuals affected by financial fraud.

#Conclusion

This study systematically evaluated LLMs' potential to assist in legal tasks using the context of cryptocurrency securities cases. While the LLM struggled with legal reasoning, its drafting capabilities were strong enough to produce convincing legal complaints. Even though LLMs cannot replace lawyers at this stage, they could help provide more affordable legal services, improving access to justice for many people.

#Future Research Directions

Future studies could look into expanding the dataset, testing the performance of other AI models, and applying these approaches to broader areas of law. Further investigation into how LLMs might support legal professionals in various tasks could reveal more about their potential applications in the legal field.