After having used artificial intelligence to publish court decisions in open data, the Cour de cassation decided to use it again to detect divergences in case law between different courts and to check whether the justice rendered is the same for all. For this purpose, it called on the scientific experts of the AI Lab, attached to Etalab within the Interministerial Directorate of Digital Affairs (Dinum), in particular the ALMAnaCH project-team from INRIA. They presented their work in the article “Complex Labelling and Similarity Prediction in Legal Texts: Automatic Analysis of France’s Court of Cassation Rulings” published on the HALonline platform.
The Court of Cassation is the highest court in the French judicial system. Its mission is to control the exact application of the law by the courts and the courts of appeal, thus guaranteeing a uniform interpretation of the law.
As a reminder, at the beginning of the year, the Ministry of Justice abandoned the Datajust project, which aimed at creating an official compensation repository, thanks to the analysis of personal injury jurisprudence
Comparing hundreds of thousands of decisions
As the mass of data grows with the number of decisions rendered by the courts, the Court of Cassation must identify, among the decisions rendered by its six chambers, contradictory interpretations of the same legal question or law. This long and tedious task of detection is carried out manually by the Court’s lawyers, who have solid legal analysis skills as well as a perfect command of the law and case law.
They must first detect similar cases, relying on summarized versions of the decisions (summaries and keyword sequences), which are also produced manually and are not available for all decisions. There is also a high degree of variability in the choice of keywords and the level of granularity used.
The Court of Cassation decided to turn to AI to analyze the hundreds of thousands of decisions and detect all discrepancies.
Identifying case law discrepancies with AI
Created in 2019, the Laboratory for Artificial Intelligence (Lab IA) supports administrations in the deployment of their AI projects by cooperating with Inria researchers. Ioana Manolescu, its scientific director, thus connects selected projects with scientific experts.
The members of the ALMAnaCH project-team: Benoît Sagot, Rachel Bawden, specialists in automatic language processing (ALP), and Thibault Charmet, engineer, worked closely with the legal experts and data scientists of the Cour de cassation, for this project.
The identification of similar court decisions can be automated as soon as we know how to automatically measure the similarity between two decisions, so the researchers decided to associate two by two decisions on a database of 80,000 decisions.
For this purpose, they developed a model for predicting titles from the summaries. They assigned a titling to decisions that did not have one, and then provided additional titlings to all decisions, under the assumption that this would facilitate the identification of similar document pairs. To produce these titers automatically, they modeled the prediction of titers from summaries as a machine translation task.
Rachel Bawden, a research fellow at Inria since 2020 in the ALMAnaCH project team, comments:
“validating the adaptation of machine translation technologies to other types of data and to other tasks that meet court needs has been really interesting and opens perspectives for other projects in other domains like finance or biomedicine.”
A relevant approach
The scientists asked the Court’s lawyers to do the same work as their algorithm after defining a hierarchy of similarity levels with them. Experiments conducted not only showed that the automated approach yielded similar results to the experts’ judgments, but also that the additional titrations reinforced those similarities.
The collaboration with the Court of Cassation continues, in order to finalize semi-automatic models to facilitate the writing of titrations by experts. On the side of the Court of Cassation, these final templates will be integrated in the workflows of the Court soon, and the members of the ALMAnaCH project-team are ready to renew the experience with other public structures.
Authors :
Thibault Charmet, Benoît Sagot, Rachel Bawden, ALMAnaCH project team, Inria.
Ines Cherichi, Matthieu Allain, Urszula Czerwinska, Amaury Fouret, Cour de cassation.
Translated from La Cour de Cassation se tourne vers l’IA pour identifier les divergences des jurisprudences
