Facebook AI announces the free access toAugLy, a Python library for artificial intelligence researchers. After the update of Dynaboard with Dynabench to provide an evaluation of natural language processing models, it is the turn of machine learning algorithms to be tested. This library will allow to use data augmentations to evaluate and improve accordingly machine learning models.
AugLy
, an open source Python library
The datasets and models being mined are becoming increasingly multi-modal, and it is useful, according to Facebook, to be able to transform all the data in a project under a unified library and API. With this in mind, they have deployed AugLy, an open source data augmentation library combining multiple media: audio, image, video and text.
The library offers more than a hundred data augmentations focused on actions users can perform around their photos and videos on social networks such as overlaying text, adding emotion or editing screenshots. This set of augmentation is directly informed by the types of data transformations Facebook has seen on its platforms. As a result, the firm says their library could be useful for researchers working on social media-related patterns or data.
A library composed of four sub-libraries
Facebook researchers designed AugLy so that it has four sub-libraries, each corresponding to a distinct modality (audio, image, video and text). Each of the libraries has the same interface: it provides transformations in formats based on functions and classes whose intensity can be known through known parameters. The Python library can generate useful metadata to help researchers understand how their data was transformed.
For example, one of AugLy ‘s proposed enhancements makes it possible to take an image or video and overlay it on a social network interface to give the impression that the file was captured by a user and then re-shared. Facebook says that this augmentation is useful for Facebook since their users share their content in this way and the company wants its systems to be able to identify any content despite the fact that there may be interface elements hindering its viewing.
Facebook wants to advance multimodal machine learning
In its remarks, Mark Zuckerberg’s company clarifies the whole point of data transformations:
“Data augmentations are vital to ensure the robustness of AI models. If we can teach our models to be robust to perturbations in unimportant attributes of the data, the models will learn to focus on the important attributes of the data for a particular use case.”
Facebook cites several applications for its library such as the ability to improve AI models withAugLy ‘s databases to improve detection of illegal or copyright-infringing content such as music or video with the SimSearchNet tool, trained using the data library and designed using a convolutional neural network.
The library supports image augmentations such as cropping, filling, overlaying text to create web selfies, screen capture, or, as mentioned earlier, sharing a photo.
Translated from Facebook AI propose la bibliothèque Python AugLy pour évaluer et améliorer les modèles ML
