Open AI’s GPT-4 aces bar exam

Artificial Intelligence (AI) has reached a new milestone, as it outperformed the majority of law school graduates in the bar exam. Two legal tech employees from Casetext and two law lecturers experimented using the newly improved AI model, GPT-4, which scored an impressive 297 on the bar exams.

The bar exam is a crucial two-day exam that anyone who wishes to practice law in the United States must take. The exam assesses reasoning and understanding, comprising articles, tests, and multiple-choice questions that simulate legal work.

The authors noted that large language models, such as GPT-4, can tackle complicated tasks requiring in-depth legal knowledge. These models now meet the standards of almost all lawyers in the United States.

Only four months ago, the same researchers concluded that OpenAI‘s previous large model, ChatGPT, did not meet the passing score for the bar exam. However, the improved GPT-4 now performed exceedingly well, surpassing the average score of law school graduates by more than 7%.

GPT-4 achieved a remarkable score of approximately 76% on the bar exam’s multiple-choice questions, 26% more than ChatGPT. The results demonstrate that technology is continuously advancing and can outperform traditional methods in specialized fields such as law.

In a statement made by the National Conference of Bar Examiners, the body responsible for designing the multiple-choice questions, AI cannot meet up to the outstanding skills, education, and experience acquired by attorneys.
A Chicago-Kent College of Law professor, Daniel Martin Katz, mentioned that he found GPT-4’s ability to write legally related essays and answer multiple-choice questions surprising. He said many people said, “Well, it might get the multiple choice, but it will never get the essays.”

Artificial Intelligence in Law

Although AI has performed excellently in other tests and examinations like the GRE and the SAT, there is a high interest in the bar exams. Sean Silverman, a bar exam tutor, has claimed that the reason for the heightened interest is the difficulty of the exam. After spending three years in law school, only 78% of first-time test takers passed the exam.

Silverman also mentioned that it would not be much of a wonder if people discovered that AI performed well in a test for high schoolers rather than for law school graduates.

The use of AI in law has been on the rise in recent months, with the potential to reduce costs and time spent on legal cases significantly. However, there is a growing debate among legal professionals about the validity of AI in the field. In a recent development, the first robot lawyer was sued this month, fueling the ongoing discussion.

Photo Credits: Open AI