In July 2023, New York City became the first jurisdiction globally to mandate annual bias audits for Automated Employment Decision Tools (AEDTs) used in hiring and promotion processes. Local Law 144 aims to ensure fairness by requiring independent audits that assess race and gender disparities in AI-driven hiring systems. However, as with any groundbreaking legislation, the implementation has surfaced several challenges that employers must navigate.
What Does Local Law 144 Require?
Local Law 144 prohibits employers from using AEDTs unless they have undergone an independent bias audit within the past year. The audit must evaluate the tool’s impact on race and gender fairness, and the results must be publicly accessible. Additionally, employers are required to post a transparency notice with job listings that disclose the use of such tools and provide information on how to request audit results.
Compliance Challenges: The Reality Check
Despite the clear mandates, compliance has been less than optimal. A study involving 391 employers revealed that only 18 had posted audit reports, and just 13 had transparency notices accompanying job postings. This discrepancy suggests that while employers may be conducting audits, they are not fully adhering to the law’s transparency requirements.
The Role of Independent Auditors
The law specifies that audits must be conducted by independent auditors, but it does not define what constitutes an independent auditor. This ambiguity has led to varied interpretations and practices, complicating the enforcement and standardization of audits. Furthermore, auditors face challenges accessing necessary data and aligning their methodologies with the law’s requirements, leading to inconsistent audit outcomes.
Transparency: A Double-Edged Sword
While the law mandates public disclosure of audit results, the effectiveness of this transparency is questionable. The complexity of audit reports and the technical nature of the findings may render them inaccessible to the average job seeker. This raises concerns about the law’s ability to empower applicants and hold employers accountable effectively.
Practical Steps for Employers
To navigate the complexities of Local Law 144, employers should:
- Engage with reputable, experienced auditors who are familiar with the nuances of the law.
- Ensure comprehensive data collection to facilitate thorough and accurate audits.
- Develop clear and accessible transparency notices that are easy for applicants to understand.
- Regularly review and update hiring practices to align with evolving legal standards and best practices.
Looking Ahead: The Future of AI in Hiring
As other jurisdictions consider similar legislation, the challenges and lessons learned from NYC’s Local Law 144 will serve as a critical reference point. For employers, staying ahead of regulatory trends and committing to ethical AI practices will not only ensure compliance but also foster trust and fairness in hiring processes.
