Large-scale language models (LLMs) have been in the news more and more lately. Last April, Google Research presented PaLM (Pathways Language Model), a 540 billion parameter decoder. Last August, the researchers teamed up with Everyday Robots to develop PaLM-SayCan, a robotic algorithm that combines the understanding of language models with the real-life capabilities of a helper robot. Now they are introducing CaP (Code as Policies), a new AI tool that builds on this work to enable robots to create code from natural language.

PaLM-SayCan aims to leverage knowledge from linguistic models to enable a robot to follow high-level textual instructions to perform physically embedded tasks. Unlike typical robotic demonstrations that perform a single task in a constrained environment, this research has demonstrated that a robot can perform the request in a real-world environment using reasoning. CaP will extend this approach by allowing language models to perform even more complex robotic tasks with the full expression of general purpose Python code.

In order to control robots, in most cases, they are programmed with code to detect objects, sequencing commands to move actuators, and feedback loops to specify how the robot should perform a task. Although these programs are expressive, reprogramming the strategies for each new task can be time consuming and requires domain expertise.

The PaLM model, according to the blog post written by Jacky Liang, research intern, and Andy Zeng, research scientist, robotics at Google, is capable of complex reasoning because it has been trained on millions of lines of code and is demonstrating its effectiveness in producing code that can control the robot’s behavior in addition to regular code.

Code as Policies, a code generation tool

Google researchers present Code as Policies (CaP): a robot-centric formalization of programs generated by language models (LMPs) running on real systems.

CaP uses a language model for writing code that allows it to generate code for new tasks from a few prompts. This approach relies on hierarchical code generation, which encourages language models to recursively define new functions, accumulate their own libraries over time, and self-design a dynamic code base.

Code-writing language models can express a variety of language-based arithmetic operations and feedback loops. Pythonic LMPs can express complex policies using :

• Classical logic structures, e.g., sequences, selection (if/if), and
  loops (for/while) to assemble new behaviors at runtime;
• Third-party libraries to interpolate points (NumPy), analyze and generate
  generate shapes (Shapely) for spatial-geometric reasoning…

CaP also inherits language model features that are not related to code writing, such as support for instructions with languages and emojis other than English.

The experiments allowed Google researchers to demonstrate that code output led to improved generalization and task performance compared to direct learning of robotic tasks and natural language action generation. CaP allows a single system to perform a variety of different robotic tasks without task-specific training but struggles to grasp complex or abstract prompts.

The researchers have published the code needed to reproduce their experiments and an interactive simulated robot demonstration on the project’s Website, which also contains additional real-world demos with videos and generated code.

Article sources:

“Robots that write their own code”

Blog by Jacky Liang, research intern, and Andy Zeng, research scientist, robotics at Google, November 2, 2022

“Code as Policies: Language Model Programs for Embodied Control” arXiv:2209.07753v2 19 Sep 2022

Authors:
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng of Google Robotics

Translated from Google Research présente Code as Policies (CaP), un nouvel outil de génération de code