In a resolution voted today in the European Parliament, MEPs stressed the importance of human control for the supervision of artificial intelligence systems and of using open algorithms. MEPs also took a stand on banning private databases for facial recognition, behavioural policing and citizen rating. Automatic recognition should not be used for border control or in public spaces.

To guarantee the right to privacy, MEPs call for strong safeguards when artificial intelligence tools are used by law enforcement agencies.

In a resolution adopted with 377 votes in favour, 248 against and 62 abstentions, MEPs highlight the risk of algorithmic bias in artificial intelligence (AI) applications and say that strong human and legal controls are needed to prevent discrimination by AI, especially in law enforcement or in a cross-border context. Final decisions must always be taken by a human being and people subject to AI-powered systems must have avenues for redress, say MEPs.

Rapporteur Petar Vitanov (S&D, BG) said:

“We are sending a clear message that fundamental rights are irrevocable. For the first time ever, we call for a moratorium on the deployment of facial recognition systems for law enforcement purposes, as these technologies have proven ineffective and often lead to discriminatory results. We clearly oppose predictive policing based on the use of AI as well as any processing of biometric data leading to mass surveillance. This is a huge victory for all European citizens.”

Fears over discrimination

According to the text, AI-based identification systems already make a disproportionate number of mistakes in identifying ethnic minority groups, LGBTI people, the elderly and women, which is a concern in the context of law enforcement and the justice system.

To ensure that fundamental rights are respected when these technologies are used, algorithms should be transparent, traceable and sufficiently documented, say MEPs. Where possible, public authorities should use open source software to increase transparency.

Controversial technologies

To respect privacy and human dignity, MEPs call for a permanent ban on automated recognition of individuals in public spaces, noting that citizens should only be monitored when suspected of a crime.

Parliament calls for a ban on the use of private facial recognition databases (such as the Clearview AI system, which is already in use) and predictive policing based on behavioural data.

MEPs also want to ban social rating systems that attempt to assess the trustworthiness of citizens based on their behaviour or personality.

Finally, Parliament is concerned about the use of biometric data for remote identification. For example, border control gates based on automatic recognition and the iBorderCtrl project (an “intelligent lie detection system” for travellers entering the EU) should be abandoned, say MEPs, who call on the European Commission to open infringement proceedings against Member States if necessary.

Translated from Les députés européens se prononcent pour une supervision des systèmes d’intelligence artificielle et contre la surveillance de masse