GPT-4 Scores at the Top of the Class on the Uniform Bar Exam

Earlier this year, two legal scholars and scientists had GPT 3.5 take the multiple-choice portion of the bar exam. It failed to pass any portion of the exam, scoring in the bottom 10%. Now GPT has passed both the multiple-choice and essay portions — and scored around the top 10% of test takers. It wasn’t GPT 3.5, however, that took the test. It was GPT-4, the latest AI model from OpenAI. GPT stands for “generative pre-trained transformer,” and GPT-4 is part of a series of foundational language models. GPT’s rapid advancement in this field is certain to have wide implications for the legal profession. While analyzing GPT-4’s capabilities, OpenAI made the model not only take the Uniform Bar Exam but the Law School Admission Test (LSAT), the Graduate Record Examination (GRE) Quantitative and various AP subject tests. On many of the tasks, it scored at a human level. That means if GPT-4 were a person being judged solely on test-taking ability, it could get into law school. For this latest test, the two legal scholars and scientists, Daniel Martin Katz and Michael Bommarito, collaborated with Casetext, the company that recently launched the AI legal assistant CoCounsel. Casetext has now confirmed that CoCouncil is powered by GPT-4.