California is drawing a statutory line between using artificial intelligence to assist legal work and allowing it to exercise professional judgment.
Senate Bill 574, signed by Gov. Gavin Newsom Sept. 30, regulates generative AI use by attorneys and arbitrators, as fabricated case citations in legal briefs intensify concerns about lawyers’ reliance on automated systems.
The California state statute is the first in the United States aimed specifically at these AI uses in legal work, according to an Oct. 26 analysis by attorneys at Mayer Brown, marking a shift from professional guidance toward binding requirements.
The law addresses two related risks, including unreliable AI output entering legal proceedings and critical decisions being delegated to systems that cannot assume professional responsibility, the analysis said. Many provisions reinforce existing ethical duties, but putting them into statute gives these obligations a more explicit legal foundation.
For lawyers, the rules combine restrictions on delegation with requirements covering confidentiality, verification and disclosure. Attorneys may not delegate the practice of law to generative AI. They must take reasonable steps to verify its output, including every case and statutory citation, and correct erroneous or hallucinated material, according to the analysis.
That requirement targets the recurring problem of AI tools citing plausible legal authorities that do not exist. The filing provisions go further. Documents submitted to court may contain only citations the responsible attorney has personally read and verified, per the analysis. Human accountability extends beyond checking whether an AI-generated answer merely appears credible.
Attorneys must also disclose generative AI use in documents submitted to courts, the analysis said. They cannot enter confidential, personally identifying or other nonpublic information into AI systems unless access is restricted to the attorney and authorized people obligated to protect confidentiality.
For arbitration practices, confidentiality issues may arise from party agreements, institutional rules or procedural orders. Reviewing a tool’s access controls becomes an essential part of protecting case material, according to the analysis.
The new restrictions on arbitrators focus specifically on decision-making. A new section of the California Arbitration Act prohibits delegating any part of that process to generative AI. Arbitrators also cannot rely on AI-generated information outside the record without first disclosing it to the parties and, as far as practical, allowing comment, the analysis said.
As the measure’s author, Sen. Tom Umberg, put it, decisions entrusted to arbitrators should be “made by people, not algorithms,” Law.com reported Oct. 1.
That safeguard addresses procedural fairness. Parties should have an opportunity to challenge information influencing an award. Existing duties of neutrality, competence and diligence remain in place.
California’s action is similar to principles seen in judicial developments outside the U.S. In ARIHQ v. Santé Québec, the Quebec Superior Court annulled an arbitral award over improper delegation of decision-making authority to AI, the analysis said. Together, such rulings and California’s statute suggest that enforceable boundaries are replacing reliance on guidance alone.
Yet the California statute comes as arbitration institutions are experimenting with greater automation. The American Arbitration Association’s International Centre for Dispute Resolution introduced an AI Arbitrator in November 2025, the first service deployed by a major arbitral institution to generate awards rather than merely assisting the tribunal. The service is open to consenting parties in narrowly defined, documents-only construction disputes. It prepares a draft award subject to human review, validation and revision.
Whether these safeguards satisfy rules requiring human decision-making remains unresolved, according to the analysis.
Other uncertainties concern what constitutes delegating legal practice and whether court-focused disclosure and filing provisions extend to arbitral submissions. These questions could shape how firms distinguish permissible assistance from prohibited substitution, the analysis said.
Attorneys and arbitrators should review AI practices before Jan. 1, when the statute takes effect, documenting how services are used and confirming that confidential material stays in restricted systems, according to the analysis. Parties should establish AI protocols at the outset of disputes and discuss expectations with tribunals early.
The stakes extend beyond inaccurate briefs. Awards showing undisclosed AI involvement or improper delegation could face challenges grounded in statute as well as procedural fairness, the analysis said. For legal professionals, retaining responsibility now requires demonstrable controls over how AI participates in the work.
For all PYMNTS AI coverage, subscribe to the daily AI newsletter.
