The European Parliament adopted by 377 votes to 248, with 62 abstentions, a resolution on artificial intelligence (AI) in criminal law and its use by the police and judicial authorities in criminal matters.
This resolution addresses the issues raised by the use of AI in criminal law and its use by police and judicial authorities in criminal matters. While recognising the potential opportunities and benefits that AI can bring, it also highlighted the significant risks and consequences that it can bring. It also highlighted the risks it may entail for the protection of people's fundamental rights.
Respect for fundamental rights
Given that the processing of large amounts of data is at the heart of AI, Members believe that the EU legal framework on data protection and privacy must be fully respected and should form a basis for any future regulation of AI for law enforcement and judicial use. The use of AI applications must be prohibited when incompatible with fundamental rights. Moreover, the use of AI applications should be categorised as high-risk in instances where there is the potential to significantly affect the lives of individuals.
Parliament reaffirmed that all AI solutions for law enforcement and the judiciary also need to fully respect the principles of human dignity, non-discrimination, freedom of movement, the presumption of innocence and right of defence, including the right to silence, freedom of expression and information, freedom of assembly and of association, equality before the law, the principle of equality of arms and the right to an effective remedy and a fair trial, in accordance with the Charter and the European Convention on Human Rights.
Risk of discrimination
Many algorithmically driven identification technologies currently in use disproportionately misidentify and misclassify and therefore cause harm to racialised people, individuals belonging to certain ethnic communities, LGBTI people, children and the elderly, as well as women.
Parliament called for algorithmic explainability, transparency, traceability and verification as a necessary part of oversight, in order to ensure that the development, deployment and use of AI systems for the judiciary and law enforcement comply with fundamental rights, and are trusted by citizens, as well as in order to ensure that results generated by AI algorithms can be rendered intelligible to users and to those subject to these systems.
Members considered that strong efforts should be made to avoid automated discrimination and bias and they called for safeguards against the misuse of AI technologies by law enforcement and judicial authorities also need to be regulated uniformly across the Union.
Mandatory impact assessments
Members called for a compulsory fundamental rights impact assessment to be conducted prior to the implementation or deployment of any AI system for law enforcement or the judiciary, in order to assess any potential risk to fundamental rights. These impact assessments should be conducted with the active participation of civil society. They should clearly define the safeguards needed to address the identified risks and be made public, as far as possible, before the deployment of any AI system.
Parliament called for periodic mandatory auditing of all AI systems used by law enforcement and the judiciary where there is the potential to significantly affect the lives of individuals. It also highlighted the need for specialised training regarding the ethical provisions, potential dangers, limitations, and proper use of AI technology, especially for police and judiciary personnel.
Guarantee human intervention
Members called for the precautionary principle to be respected in all law enforcement applications of AI and stressed that in judicial and law enforcement settings, the decision giving legal or similar effect always needs to be taken by a human, who can be held accountable for the decisions made. Those subject to AI-powered systems must have recourse to remedy.
Surveillance and mass profiling
Parliament called for the permanent prohibition of the use of automated analysis and/or recognition in publicly accessible spaces of other human features, such as gait, fingerprints, DNA, voice, and other biometric and behavioural signals. It also called for a ban on the use of private facial recognition databases (such as the Clearview AI system).
Members called for a moratorium on the deployment of law enforcement facial recognition systems for identification purposes, unless they are only used for the purpose of identifying victims of crime, until technical standards can be considered fully respectful of fundamental rights.
Members also supported a ban on mass-scale scoring of individuals using AI.
Lastly, Parliament expressed concern about research projects funded under Horizon 2020 that deploy artificial intelligence at external borders, such as the iBorderCtrl project, a ‘smart lie detection system’ for travellers entering the EU. It called on the Commission to implement, if necessary through infringement procedures, the ban on any processing of biometric data for law enforcement purposes leading to mass surveillance in publicly accessible areas.