An AI system capable of solving elementary math problems has been developed by researchers at Open AI (the company behind the GPT-3 language model). This is a step forward since, until now, models have been able to master the manipulation of language to formulate sentences, but not the multi-step reasoning required to solve math problems. This is now done.

Researchers have trained a model to recognize its own mistakes, so it can re-evaluate its answers repeatedly until it discovers a good solution. In tests, the AI system was able to solve almost as many problems as a sample of children aged nine to 12. The children scored 60% on a test from the Open AI database, while the AI system scored 55%.

Although primary school maths is fairly straightforward, for Open AI, the arrival of AI models that can solve basic maths problems is a big leap forward that opens up many opportunities. The researchers explained:

“A major challenge in mathematical reasoning is the high sensitivity to individual errors. Autoregressive models, which generate each solution token by token, have no mechanism to correct their errors. Solutions that go wrong quickly become unrecoverable.”

This problem was circumvented by creating checkers whose role is to evaluate the answers generated by the AI. These checkers were given 100 potential solutions, all generated by the model, and then tasked with determining whether any were correct. The Open AI researchers add:

“Providing correct arguments and recognising incorrect ones are key challenges in the development of more general AI. The problems are conceptually simple, but one subtle mistake is enough to derail an entire solution. Identifying and avoiding such errors is a crucial skill for the development of our models.”

Open AI believes that the verification system that allows its AI systems to solve simple mathematical problems with some accuracy will become increasingly effective as AI is applied to more complex areas.

By combining this research with advances in semiconductors, it will be possible to envision AI models that are much larger than they are today. Thus, the ability to change the way AI approaches a problem could prove transformative.

Translated from L’intelligence artificielle peut désormais résoudre des problèmes mathématiques simples