In a move that has ignited a fresh wave of controversy, the State Bar of California has acknowledged the use of artificial intelligence in crafting some of its bar exam questions. This revelation comes on the heels of widespread criticism over technical issues faced by examinees during the February bar exams. The legal licensing body, already under fire, now finds itself at the center of a debate over the role of AI in professional testing.
Prospective lawyers have expressed outrage, with many pointing to potential conflicts of interest and questioning the reliability of AI-generated questions. Mary Basick, assistant dean of academic skills at UC Irvine Law School, remarked, “The debacle that was the February 2025 bar exam is worse than we imagined. I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”
The original article from the Los Angeles Times provides a detailed account of the controversy, highlighting the broader implications of AI adoption in legal assessments.
The State Bar’s decision to incorporate AI was partly driven by financial constraints, as it sought to cut costs by moving away from the traditional Multistate Bar Examination. Instead, it entered into a deal with Kaplan Exam Services to develop exam questions, a move that has not been without its critics.
According to a recent presentation by the State Bar, a subset of the questions was developed with AI assistance by ACS Ventures, the State Bar’s independent psychometrician. This has raised questions about the integrity and validity of the exam, which the State Bar has defended, citing that all questions were reviewed by content validation panels and subject matter experts.
Katie Moran, an associate professor at the University of San Francisco School of Law, criticized the State Bar’s approach, stating, “The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam. They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”
The controversy has prompted calls for transparency and procedural changes, with some advocating for a return to traditional in-person exams. The California Supreme Court has directed the State Bar to plan for in-person exams in July, a move that underscores the challenges of remote testing.
In light of these developments, the State Bar has been urged to release all 200 questions from the February exam for public scrutiny. Critics argue this is necessary to restore confidence in the testing process and ensure future examinees are adequately prepared.
As the debate continues, the State Bar’s use of AI in exam development remains a contentious issue, highlighting the complexities of integrating new technologies into established professional practices.