Robotics company Figure and ChatGPT maker OpenAI are only two weeks into their partnership but the collaboration is already producing some astounding and slightly unsettling results.
Figure this week shared an update on Figure 01’s progress and said it can now have full conversations with people after connecting to a large pre-trained multimodal model. The company said its robot can now describe its visual experience, plan future actions and use common sense reasoning, reflect on its memory and explain its reasoning verbally.
Most Read on IEN:
- CEO Faces Up to 30 Years in Prison for Selling Fake Medical Implants
- Boeing Promises Change
- Podcast: EV Maker's Money Problems; Toyota Gives Away Hydrogen Cars; Bridge Collapse Root Cause
- GM, Major Carmakers Are Telling Insurers How You Drive
In a video accompanying the update, Figure 01 identifies everything in its field of vision, gives someone an apple when they ask for something to eat, and then explains its logic in determining that the apple was the only edible thing on the table. While having a conversation in real-time, Figure 01 also determines how to put away dishes and cleans up trash, all while speaking in a very natural tone of voice. The company confirmed that all behaviors seen in the video are learned and not teleoperated.
OpenAI and Figure said they are relying on the robot’s onboard cameras to feed into a large vision-language model that can break down the entire history of the conversation and determine the best language responses. The model also determines which of the robot’s learned behaviors, which are driven by neural network visuomotor transformer policies, best fulfills given commands.
When Figure and OpenAI announced their agreement two weeks ago, the companies said integrating specialized AI models into Figure 01 should help accelerate the commercial timeline for the robot, which is still in development.
Figure already has a significant client in BMW, which earlier this year announced plans to introduce general purpose robots on its production lines. If Figure 01 continues to progress the way it has, BMW employees will soon have robot coworkers that can not only handle tedious tasks but keep them company as well.
