Updated May 19, 2023: AB 331 has died in committee and will not become law this session.

California continues to take steps to regulate the burgeoning use of artificial intelligence, machine learning, and other data-driven statistical processes in making consequential decisions, including those related to employment. The California Civil Rights Council (CRC)¹ recently issued updated proposed regulations governing automated-decision systems. The agency had issued a working draft in March 2022.  In addition to these regulatory efforts, California lawmakers have introduced two bills designed to further regulate AI in employment.  California’s efforts at oversight now consist of the following:

• The CRC’s Proposed Modifications to Employment Regulations Regarding Automated-Decision Systems;
• Assembly Bill No. 331, to add Chapter 25 (commencing with Section 22756) to Division 8 of the Business and Professions Code, relating to artificial intelligence; and
• Senate Bill No. 721, “California Interagency AI Working Group,” to add and repeal Section 11546.47 of the Government Code, relating to artificial intelligence.²

While these approaches have a common goal of minimizing the potential negative consequences of artificial intelligence when deployed (in relevant part) in the employment and personnel management contexts, they also represent simultaneous efforts from disparate bodies racing in California to be the first to regulate new technology.  Adding to the confusion in an increasingly crowded space is the fact that these entities are proposing their own unique definitions regarding similar terminology (e.g., “adverse impact” by the CRC versus “algorithmic discrimination” by the California Assembly, and “automated-decision system” by the CRC versus “automated decision tool” by the California Assembly), when it remains unanswered who should even be defining these terms as a threshold matter.  Taken as a whole, the CRC, whose mission is to promulgate regulations that implement California’s civil rights laws, appears to be leapfrogging the legislative process with its efforts.

The following summarizes the latest primary updates regarding California’s three-pronged approach.

Civil Rights Council’s Proposed Rules

The latest iteration of the CRC’s Proposed Modifications to Employment Regulations Regarding Automated-Decision Systems was released on February 10, 2023.  Since first publishing draft modifications to its antidiscrimination law in March 2022, the CRC has continued to refine its definitions of key terms without altering the primary substance of the proposed regulations.  The CRC’s most recent proposal includes the following primary updates:

Key Updates to Definitions

• Introduces definition for adverse impact, which includes but is limited to, “the use of a facially neutral practice that negatively limits, screens out, tends to limit or screen out, ranks, or prioritizes applicants or employees on a basis protected by the Act.  ‘Adverse impact’ is synonymous with ‘disparate impact.’”
• Introduces definition for artificial intelligence to mean a “machine-learning system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions.”
• Introduces definition for machine learning to mean the “ability for a computer to use and learn from its own analysis of data or experience and apply this learning automatically in future calculations or tasks.”
• Broadens the definition of agent from a person acting on behalf of an employer to provide services “related to …. the administration of automated-decision systems for an employer’s use in recruitment, hiring, performance, evaluation, or other assessments that could result in the denial of employment or otherwise adversely affect the terms, conditions, benefits, or privileges of employment” to “the administration of automated-decision systems for an employer’s use in making hiring or employment decisions” (emphasis added).
• Introduces definition for proxy to mean a “technically neutral characteristic or category correlated with a basis protected by the Act.”
• Provides a more fulsome list of examples of tasks that constitute automated-decision systems, and clarifies that automated-decision systems exclude word-processing software, spreadsheet software, and map navigation systems.
• Provides an example of the capability of algorithms to “detect patterns in datasets and automate decisions [sic] making based on those patterns and datasets.”
• Renamed and refined the definition of Machine-Learning Data to “Automated-Decision System Data.”

Key Updates to Defense to Unlawful Employment Practice and Recordkeeping Obligations

• Clarifies how an employer or covered entity can defend against a showing that it engaged in an unlawful use of selection criteria that resulted in an adverse impact or disparate treatment on an application, employee or class of applicants or employees on a protected basis, i.e., that the employer or covered entity can show that the selection criteria, as used, is job-related for the position in question and consistent with business necessity and there is no less-discriminatory policy or practice that serves the employer’s goals as effectively as the challenged policy or practice.³ 
• Extends recordkeeping obligations not just to any person who sells or provides an automated-decision system or other selection criteria to an employer or covered entity, but also to any person “who uses an automated-decision system or other selection criteria on behalf of an employer or other covered entity.”
• Clarifies the scope of records to be preserved, rather than simply referring to “records of the assessment criteria used by the automated-decision system.”⁴ 

Assembly Bill No. 331

Introduced by Assembly Member Bauer-Kahan on January 30, 2023, Assembly Bill No. 331 (AB 331) would, similar to NYC Local Law 144, impose obligations on employers to evaluate the impact of an automated decision tool⁵ and to provide notice regarding its use, and provide for formation of a governance program.  SB 331 would also prohibit a deployer⁶ from using an ADT in a way that contributes to algorithmic discrimination.⁷

Impact Assessment

AB 331 would require a deployer and a developer⁸ of an ADT to perform an impact assessment⁹ on/before January 1, 2025, and annually thereafter, for any ADT used that includes:

• a statement of the purpose of the ADT and its intended benefits, uses, and deployment contexts;
• a description of the ADT’s outputs and how the outputs are used to make, or are a controlling factor in making, a consequential decision;
• a summary of the type of data collected from natural persons and processed by the ADT when it is used to make, or is a controlling factor in making, a consequential decision;
• a statement of the extent to which the deployer’s use of the ADT is consistent with or varies with the statement required of the developer;¹⁰
• an analysis of the potential adverse impacts on the basis of sex, race, color, ethnicity, religion, age, national origin, limited English proficiency, disability, veteran status, or genetic information;
• a description of the safeguards that are or will be implemented by the deployer to address any reasonably foreseeable risks of algorithmic discrimination arising from the use of the ADT known to the deployer at the time of the impact assessment;¹¹
• a description of how the ADT will be used by a natural person, or monitored when it is used, to make or be a controlling factor in making, a consequential decision; and
• a description of how the ADT has or will be evaluated for validity or relevance.¹²

Notice Requirements

AB 331 would also require a deployer, at or prior to an ADT being used to make a consequential decision,¹³ notify any natural person who is the subject of the consequential decision that an ADT is being used to make, or is a controlling factor in making, the consequential decision, and to provide that person with:

• a statement of the purpose of the ADT;
• contact information for the developer; and
• plain language description of the ADT that includes a description of any human components and how any automated component is used to inform a consequential decision.

Furthermore, if a consequential decision is made solely based on the output of an ADT, a deployer must accommodate a natural person’s request not to be subject to the ADT and to be subject to an alternative selection process or accommodation, if technically feasible.

Governance Program

AB 331 would also require a deployer or developer to establish and maintain a governance program to map, measure, manage, and govern the reasonably foreseeable risks of algorithmic discrimination associated with the use of the ADT.  In relevant part, the governance program must conduct an annual and comprehensive review of policies, practices, and procedures to ensure compliance with this chapter, and make reasonable adjustments to administrative and technical safeguards in light of material changes in technology, the risks associated with the ADT, the state of technical standards, and changes in business arrangements or operations of the deployer or developer.

Senate Bill No. 721

Senate Bill No. 721 (SB 721), titled “California Interagency AI Working Group,” was introduced on February 16, 2023 and would create a Working Group to deliver a report to the legislature regarding artificial intelligence and be disbanded by January 1, 2030.

The Working Group would consist of 10 members, including two appointees by the governor, two appointees by the president pro tempore of the Senate, two employees by the speaker of the Assembly, two appointees by the attorney general, one appointee by the California Privacy Protection Agency, and one appointee by the Department of Technology.  The Working Group would be chaired by the director of technology, and the members must be Californians with expertise in at least two of the following areas:  computer science, artificial intelligence, the technology industry, workforce development, and data privacy.

The Working Group would be required to accept input from a broad range of stakeholders, including from academia, consumer advocacy groups, and small, medium and large businesses affected by artificial intelligence policies.  The Working Group would be required to:

• Recommend a definition of artificial intelligence as it pertains to its use in technology for use in legislation;
• Study the implications of the usage of artificial intelligence for data collection to inform testing and evaluation, verification and validation of artificial intelligence to ensure that artificial intelligence will perform as intended, and minimize performance problems and unanticipated outcomes;
• Determine proactive steps to prevent artificial intelligence-assisted misinformation campaigns and unnecessary exposure for children to the potentially harmful effects of artificial intelligence;
• Determine the relevant agencies to develop and oversee artificial intelligence policy and implementation of that policy; and
• Determine how the Working Group and the Department of Justice can leverage the substantial and growing expertise of the California Privacy Protection Agency in the long-term development of data privacy policies that affect the privacy, rights, and the use of artificial intelligence online.

SB 721 would also require the Working Group to submit a report to the legislature on or before January 1, 2025 regarding the foregoing, and every two years thereafter.  SB 721, if enacted would remain in effect until January 1, 2030.

With the proliferation of new regulations and laws, it is more important than ever for employers to stay abreast of developments regarding this topic, especially given the potential for a resulting patchwork of obligations for those who choose to incorporate qualifying artificial intelligence into their personnel management processes. Littler will continue to monitor these developments and report on any significant developments.