Los Angeles Times, State Bar of California Admits It Used AI to Develop Exam Questions, Triggering New Furor:

Nearly two months after hundreds of prospective California lawyers complained that their bar exams were plagued with technical problems and irregularities, the state’s legal licensing body has caused fresh outrage by admitting that some multiple-choice questions were developed with the aid of artificial intelligence.

The State Bar of California said in a news release Monday that it will ask the California Supreme Court to adjust test scores for those who took its February bar exam.

But it declined to acknowledge significant problems with its multiple-choice questions — even as it revealed that a subset of questions were recycled from a first-year law student exam, while others were developed with the assistance of AI by ACS Ventures, the State Bar’s independent psychometrician.

“The debacle that was the February 2025 bar exam is worse than we imagined,” said Mary Basick, assistant dean of academic skills at UC Irvine Law School. “I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”

After completing the exam, Basick said, some test takers complained that some of the questions felt as if they were written by AI.

“I defended the bar,” Basick said. “‘No way! They wouldn’t do that!’”

Using AI-developed questions written by non-legally-trained psychometricians represented “an obvious conflict of interest,” Basick argued, because “these are the same psychometricians tasked with establishing that the questions are valid and reliable.”

“It’s a staggering admission,” agreed Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation. …

“Until yesterday’s State Bar press release, the court was unaware that AI had been used to draft any of the multiple-choice questions,” a spokesperson said in a statement. …

Reusing some of the questions from the first-year law exam raised red flags, Basick said. An exam to figure out if a person had learned enough in their first year of law school is different from one that determines whether a test taker is minimally competent to practice law, she argued.

“It’s a much different standard,” she said. “It’s not just, ‘Hey, do you know this rule?’ It is ‘Do you know how to apply it in a situation where there’s ambiguity, and determine the correct course of action?’”

Also, using AI and recycling questions from a first-year law exam represented a major change to bar exam preparation, Basick said. She argued such a change required a two-year notice under California’s Business and Professions Code.

But the State Bar told The Times that the sources of the questions had not triggered that two-year notice.

Reuters, California Bar Seeks to Reduce Pass Score After Disastrous Exam Rollout:

The state bar on Monday said it wants to set the raw passing score for the attorney licensing exam at 534 — lower than the 560 score recommended by its standardized testing expert who looked at February's results. Raw pass scores can fluctuate each year and are converted according to a standardized scale.

The Recorder, Committee Backs Scoring Aid for February Bar Exam Applicants:

State bar leaders will ask the California Supreme Court to approve a lower-than-typical raw passing score for the February bar exam, acknowledging that the widespread technical problems that plagued the test hindered applicants' performances.

Prior TaxProf Blog coverage:

• California Bar Exam Suffers Catastrophic Meltdown: ‘I’ve Never Had This Much Despair And Hopelessness’ (Feb. 27, 2025)
• The California Bar Exam Fiasco: Is Provisional Licensure The Appropriate Remedy? (Mar. 1, 2025)
• New York Times: California Bar Exam Fiasco Enrages Test Takers And Clouds Their Futures; Chemerinsky Calls For Provisional Licensure (Mar. 3, 2025)
• 17 Deans Ask California High Court To Provisionally License Grads Who Fail February Exam And Scrap New Kaplan-Written Questions And Revert Back To In-Person Exams In July (Mar. 4, 2025)
• California Supreme Court Reverts Back To In-Person Bar Exam In July (Mar. 5, 2025)
• Law Profs: California State Bar Must Be Held Accountable For Its Catastrophic Failures (Mar. 6, 2025) 
• California Bar Orders Investigation Of February Exam Disaster As Beleaguered Test-Takers Fume (Mar. 9, 2025)
• California Bar Blocked Law School Deans And Professors From Vetting Troubled Bar Exam (Mar. 15, 2025)
• California Bar Recommends Provisional Licensure For All Who Failed And Withdrew From The February Bar Exam (Mar. 16, 2025)
• State Bar Of California Announces Remediation Actions For The February Bar Exam And Plans For The July Bar Exam (Mar. 18, 2025)
• California Bar Still Can't Decide How To Make Applicants Whole After Botched February Bar Exam (Apr. 8, 2025)

Editor's Note:  If you would like to receive a daily email with links to legal education posts on TaxProf Blog, email me here.