Machine learning and deep learning are two of the most widely used techniques to train artificial intelligence models and use them in the tasks at hand. However, getting them to forget some of the data they have learned is something much more complex. With this in mind, Facebook has designed a system that allows AI models to forget certain information.
Successfully making an AI model forget data
Every day, our brains are inundated with a constant stream of information, most of which will be forgotten by the evening. The ability to forget details of everyday life is something we take for granted in order to make room for the memories we hold dear or the things we will use later on.
Unlike human memory, artificial neural networks process information indiscriminately: all information is taken into account. On a small scale, this has no disadvantages: the system manages superfluous information and takes the one that interests it. But today, the mechanisms are becoming more and more elaborate, and receive very large amounts of data, which implies increasingly long calculation phases.
To avoid overheating the machines and manage to “forget” certain data to the AI models, Facebook has developed a new deep learning method called Expire-Span.
A deep learning method to forget
The goal ofExpire-Span is to make AI models forget certain information. By forgetting, the AI will be able to retain more interesting information in order to perform better, as it will be able to focus on the most important elements. The method gives neural networks the ability to forget on a large scale.
The tool works by predicting the most relevant information to retain in order to perform a specific task. Then, depending on the context, Expire-Span assigns an expiration date to each piece of information. When this limit is passed, the data is gradually “forgotten”. With more memory available, AI systems can process data on a larger scale.
To give an example, let’s say an AI model has been trained to find a yellow door. The data storage memory beforeExpire-Span is used contains a lot of information, some of which is useful, and some of which is useless. AfterExpire-Span, only the data that is useful for the task is kept.
This method will, according to Facebook, improve the efficiency of the model to perform several actions in language modeling, reinforcement learning, object collision or algorithmic tasks.
The advantage of Expire-Span: gradual forgetting of data
When we want to make an AI forget information, we are very often confronted with a double choice without the possibility of making an intermediate choice: either the information is kept or it is forgotten. One of the objectives ofExpire-Span is to make sure that systems can progressively forget irrelevant information while continuously optimizing these discrete operations. This gradual forgetting is similar to the way the human brain gradually forgets memories that it wants to get rid of.
For example, let’s imagine an artificial neural network containing a chronological series of words, images or videos. For each of these pieces of information, Expire-Span calculates its expiration value, which determines how long the data will be kept in memory. This gradual degradation of some information is essential to keep important information without blurring it.
Although the method is currently in the research phase, Facebook is talking about the possibility thatExpire-Span could be used in real-world applications that require forgetting non-essential information. As part of their next stage of research towards human-like artificial intelligence systems, Mark Zuckerberg’s firm wants to study how to incorporate different types of memories into artificial neural networks.
Translated from Facebook présente une méthode de deep learning pour que les modèles oublient certaines informations
