Yandex launches the NeurIPS 2021 Shifts Challenge, a competition open to machine learning experts

4 août 2021

As part of the NeurIPS machine learning conference, Russian internet company Yandex announced the launch of the “Shifts Challenge.” The company is launching its three-part competition to solve the problem of shifting distribution in machine learning. Until October 17, machine learning experts can design powerful ML models and hope to win the grand prize of $5,000. </b><h2>A challenge to develop a robust model that detects distribution change </h2>Yandex is inviting researchers and machine learning experts from around the world to participate in the NeurIPS 2021 Shifts Challenge, a competition about the robustness and uncertainty of a real-world distribution shift. The goal of the challenge is to raise awareness of distributional shifts in real-world data. For participants, the goal will be to develop models robust to distributional change and to detect such change via uncertainty measures in their predictions. Participants can take part in the three separate themes of the challenge, for which the Russian company is providing datasets in all three modalities: weather forecasting, machine translation, and vehicle motion forecasting. Participants' solutions will be presented at the Shifts Challenge workshop at NeurIPS 2021, with prizes awarded to the top-ranked participants. Here are more details on the three tracks one can choose from to design their innovative model: <ul> <li><strong>Weather forecasting</strong>: the objective of this track is to train models that predict the temperature at a particular latitude/longitude and time, taking into account all available measurements and climate model predictions. These models must be both robust to changes in weather and climate and capable of detecting them.</li> <li><strong>Machine translation</strong>: here, the models will be trained to translate a sentence from a source language to a target language. These models must be robust to changes such as atypical and unusual language usage, profanity, emojis and incorrect punctuation in translation requests.</li> <li><strong>Vehicle movement prediction</strong>: models will be trained to predict the distribution of possible vehicle positions around the autonomous car at a number of times in the future. The models must be robust to changing location, season, time of day and precipitation.</li> </ul> <h2>A competition open to all machine learning experts and researchers </h2>Mark Gales, who is leading the University of Cambridge's Shifts Challenge collaboration, said around the launch of the project: <blockquote>"As deep learning approaches become more powerful, they are being applied in increasingly interesting and diverse areas. It is increasingly important for these systems to 'know when they don't know', to avoid bad decisions. By participating in the Global Shifts Challenge, researchers have an unprecedented opportunity to evaluate on large-scale real data the ability of their models to measure confidence in their own predictions." </blockquote>The challenge has two development phases: <ul> <li><strong>Phase 1</strong> (between July 20 and October 17): Training and development data is released. Participants build models and submit them to the development rankings. Registration is possible through the <a href="https://research.yandex.com/shifts" target="_blank" rel="noopener">following link</a>.</li> <li><strong>Phase 2</strong> (between October 17 and 31): Holdback evaluation data is released. Participants have two weeks to tune their models and submit them to the evaluation ranking. The best solutions in the evaluation ranking are rewarded.</li> </ul> The organizers will check all contributions throughout November. The results will be announced on November 30 and the top three will be rewarded:$ 1,000 for the third place winner, $3,000 for the second place winner, and the overall winner will receive$ 5,000.

Translated from Yandex lance le NeurIPS 2021 Shifts Challenge, un concours ouvert aux experts du machine learning

Previous article Elon Musk announces the official date of Tesla AI Day, an event dedicated to AI and recruitment

Next article Metron raises €18 million and acquires Dapesco to enhance its offer for a low-carbon industry

Zacharie Tazrout