California Bar criticised for using AI to develop exam questions
The use of AI by the State Bar of California to develop multiple-choice questions for its February 2025 bar exam resulted in low quality questions and was seen to threaten the integrity of professional standards and public trust in the legal system. What happened The State Bar revealed that 23 out of 171 scored multiple-choice questions were developed with the assistance of AI by ACS Ventures, a psychometrician not trained in law, and these questions appeared on the live exam. The disclosure came amidst widespread complaints from test-takers about technical failures during the hybrid in-person and remote exam, including platform crashes, errors, and poorly written or nonsensical questions. The controversy led to calls for an independent audit, a federal lawsuit, and a demand from the California Supreme Court for the State Bar to explain its AI use and ensure the reliability of exams. The State Bar asked the Supreme Court to adjust scores for affected examinees. Why it happened The State Bar sought to cut costs after a USD 22-million deficit by switching from the national bar exam to a new test model to develop questions and hiring outside vendors, including Kaplan and ACS Ventures. The use of AI was intended to innovate and improve reliability and cost-effectiveness, following guidance from the Supreme Court to consider new technologies. However, critics argue that allowing non-lawyers to use AI to draft questions, and then having the same company validate them, created a conflict of interest and undermined question quality. What it means For examinees, the incident has cast doubt on the fairness and validity of their licensure process, especially for those who struggled with technical or content-related issues. Law professors and legal experts warn that the use of AI by non-lawyers in high-stakes testing threatens the integrity of professional standards and public trust in the legal system. For society, the uproar highlights broader concerns about transparency, accountability and the appropriate use of AI in critical decision-making contexts.
- Date it happened
- 2025-01-01
- Organisation involved
- The State Bar of California
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.