On Tuesday 25 January, the Internal Market and Civil Liberties Committees held their first joint exchange of views on the proposed legislation on artificial intelligence. Parliamentary work on this central topic will continue this year. In December, on the occasion of its Presidency of the Council of the European Union, Slovenia had presented a compromise text on the draft EU law on AI, including major changes in the areas of social rating, biometric recognition systems and high-risk applications, while also identifying future discussion points. A look back at these changes.

Ahead of the meeting held this Tuesday, co-rapporteur for the Internal Market and Consumer Protection Committee, Brando Benifei (S&D, IT), said:

“Our aim is to protect our citizens and consumers, and at the same time stimulate positive innovation, with a special focus on SMEs and start-ups. A legislative framework ensuring that AI systems entering the EU internal market are safe, human-centric and respect our fundamental rights and freedoms will boost trust among our citizens, which is essential for a successful and inclusive adoption of AI on our continent. This is what we will strive for.”

For his part, Dragoş Tudorache (Renew Europe, RO), co-rapporteur for the Committee on Civil Liberties, Justice and Home Affairs, added:

“The AI legislation is a central piece of the European regulatory environment for the digital future, and it is the first of its kind in the world. We have the chance to lead by example and shape the rules of the digital world according to our values. At the heart of European democracy, the Parliament has a key role to play: we need to find the right balance between strengthening the protection of our fundamental rights and Europe’s competitiveness and innovation capacity.”

A proposal for a regulation on artificial intelligence presented in April 2021

The European Commission is one of the 7 European institutions and the only one that can propose new legislation. It is currently chaired by Ursula von der Leyen, nominated by the 27 member states and elected in 2019 by the European Parliament for 5 years.

To counter the advance of the Americans and the Chinese, while building a digital “responsible” and “trustworthy” to maintain its sovereignty, the European Commission unveiled on February 19, 2020 its digital strategy – a sector defined as a priority among the political guidelines of the new president Ursula von der Leyen.

On April 21, 2021, the commission submitted a proposal for a regulation on Artificial Intelligence with the following specific objectives

• ensure that AI systems placed on the EU market and used are safe and respect the legislation in force on fundamental rights and
  the values of the Union;
• Ensure legal certainty to facilitate investment and innovation in AI;
• strengthen governance and effective enforcement of existing fundamental rights legislation and
  fundamental rights and safety requirements for AI systems
  AI systems;
• Facilitate the development of a single market for legal, safe and trustworthy AI applications and prevent market fragmentation.

On the other hand, Article 5, § 1, of the proposed regulation prohibits four practices:

• AI systems that subliminally influence a person’s behavior in order to cause harm to that person or to a third party;
• AI systems that exploit the vulnerability of a group of people in order to distort the behavior of one of those people and cause harm;
• AI social rating systems marketed, commissioned or used by or on behalf of public authorities;
• Real-time remote biometric identification systems in publicly accessible spaces, with some exceptions.

Desired changes to the AI law

First, the Presidency established a definition of AI systems to better distinguish them from conventional software. According to the compromise, AI systems have the ability to process data or other types of input “to infer the means to achieve a given set of human-defined goals through learning, reasoning or modeling.”

The compromise provides that in the following 2 cases, the regulations would not apply:

• EU countries reject any interference with national security and insist that AI systems developed exclusively for military purposes be removed from the scope of the regulation
• It was decided that AI systems developed solely for the purpose of scientific research and development should also be excluded from the scope.

The provider of an AI system becomes the central actor. It is defined as the natural or legal person, the agency or any other organization that develops an AI system or that owns an AI system already developed in view of placing it on the market or putting it into service, under its own name or its own brand, for a fee or free of charge. As such, it is responsible for ensuring compliance with the requirements of the regulations.

On the other hand, a new category of “general purpose” AI system has been added, which will not fall within the scope of the regulation unless the system is used under a brand name or integrated into another system subject to the regulation.

Social credit prohibited

The Commission’s proposal includes a ban on AI applications that are considered to present unacceptable risks. One of these is social scoring, a practice particularly used in China and considered to promote mass surveillance.

The Presidency now proposes to extend the ban on social scoring from government to private entities, outlawing the exploitation of a “ social or economic situation.” The financial sector could be impacted by these decisions, as loan interest rates are calculated based on the probability of repayment. The use of AI systems to estimate insurance premiums was also included as a high-risk system.

Biometric recognition

In the draft regulations, remote time-lapse biometric identification is not subject to the section nor is biometric identification performed by a private individual on his or her behalf, although both systems remain high-risk. With this compromise, the biometric identification systems covered by the legislation are no longer defined as “remote”, but as any system that leads to the identification of persons “without their consent”.
The ability to use real-time biometric identification systems has been expanded to include actors who are not law enforcement agencies but who work with them to protect critical infrastructure.

Biometric systems can only be used with the approval of the judicial authority. In case of emergency, the initial proposal provided that authorization could also be requested a posteriori. However, this authorization will have to “be requested without undue delay when used.”

High-risk systems

The proposed law includes specific obligations for the use of AI systems that present a high risk in terms of health, safety and fundamental rights. In its proposal, the Commission identified eight high-risk areas, which can be modified but only further defined in the future.

The most important change is the inclusion of digital infrastructure designed to protect the environment, ” AI systems intended to be used to monitor emissions and pollution” . Also of note, in terms of enforcement, the subcategory of crime analysis has been removed.

The compromise text provides for an evaluation of these high-risk systems by the European Commission every 2 years, as well as that of the AI techniques and approaches covered by the regulation.

Translated from Retour sur les importantes modifications au projet de Règlement de l’intelligence artificielle de la Commission Européenne