91ɫ

IPOsgoode

Recent Privacy Regulations Concerning Automated Decision-Making Systems: Implications on AI Commercialization

Posted on 1 June 2022

Luna Li is an IP Innovation Clinic Fellow and a 3L JD Candidate at Osgoode Hall Law School.

Prior to the Covid-19 pandemic, academic discussions indicated that artificial intelligence (AI) would signify with tangible economic benefits and . With remote work becoming the , there is a growing reliance on digital technologies that lack sufficient transparency and . Nowadays, privacy concerns exceed .

With more businesses developing and using AI-based automated decision-making (ADM) systems, algorithmic discrimination (at , , or ) has been in the spotlight (in both the public and governments’ eyes). For instance, ADM systems may lead to bias against marginalized groups based on discriminatory historical data. Algorithms can adversely affect vulnerable populations through , even excluding protected characteristics. Specifically, .

This article summarizes recent global privacy regulations focusing on algorithmic management. AI developers, licensors and licensees may benefit from reviewing these regulations and the business implications following the summary table.

Jurisdictions

Regulations (shortened title)

Keynotes

European Union

(2021)

Prohibitions on:

  • AI systems that are likely to cause “physical or psychological harm” by deploying “subliminal techniques” or by exploiting vulnerable groups due to “age, physical or mental disability”.
  • AI-based social scoring for general purposes by public authorities.

(2021)
  • Enables platform workers to access relevant information about algorithmic decisions.
  • Ensures human monitoring.
  • Gives the right to contest automated decisions.

The U.S.

California: (Warehouse Quota Law, 2022)

Limitations on the use of quotas:

  • cannot prevent compliance with meal/rest periods or occupational health and safety standards.

Canada

Ontario: , (Working for Workers Act, 2022) (1st reading on February 28, 2022)
  • Employers (≥25 employees) to create and publish an electronic monitoring policy.

Federal: (Digital Charter Implementation Act, 2020) (2nd reading on April 19, 2021)

Organizations to provide a(n):

  • general account of ADM applications (including automated decisions, predictions and recommendations) that could significantly impact individuals.
  • explanation about how the personal information used was obtained.

:

Ethical AI Assurance

There are rising requests for AI providers/developers to represent or warrant that the ADM systems were developed ethically. The company using AI systems may need to maintain and adhere to ethical AI policies and data control procedures according to applicable law in the jurisdiction.

Risk Allocation

Algorithmic bias may arise discriminatory historical data and proxies (through machine learning). It is thus crucial to clarify a proper risk allocation to ensure the right party is liable for monitoring and acting on algorithmic issues as they appear. The Canadian legal regime concerning AI currently remains untested. Therefore, parties may clarify expectations on AI ownership and data-use procedures to engage relevant contract law protections.