// WIRED US/UK — INTELLIGENZA ARTIFICIALE
I Saw the Future of AI in a Robot That Can Learn on the Spot
Last week, I ventured a whopping 15 minutes from my house to see robots do some mind-boggling, jaw-dropping stuff.
I visited the Cambridge, Massachusetts, offices of a startup called Generalist AI, where I watched robot arms perform simple chores like stacking cups, putting blocks into bowls, and the like. I was astonished by how quickly they figured things out—it was reminiscent of a flesh-and-blood person.
The arms mastered a range of tasks after ingesting a short, instructional video and, most impressively, no specific training for a given task. One of the most striking examples involved a robot that was instructed to sweep a block into a bowl using a dustpan and brush. When the brush was removed from the scene, the robot improvised by using the dustpan like a brush and flicking the block into the bowl.
In another case, a two-armed robot watched a videoclip of someone unzipping a purse before removing some banknotes. I watched—somewhat slack-jawed—as the robot unzipped a different kind of purse and carefully removed the notes. Most amazingly, when it couldn’t grab the money, it switched from using its right gripper to its left to get a better angle of attack. “Ha,” said one engineer standing nearby. “It never did that before.”
“This is exactly the kind of thing people were really excited about with GPT-3,” Generalist cofounder and CEO Pete Florence told me, in reference to OpenAI’s breakthrough large language model, released in 2020. “You could take that model and just prompt it to do a new task and it would have a real shot at doing it.”
Generalist appears to be focused on teaching its robots about the physics of the world, which seems inspired by the intuitive sense of physics humans exhibit from an early age. That may well contribute to the model’s ability to transfer what it has learned in one scenario to another. In fact, some of the company’s demos made me think of how children improvise and experiment when shown a task. The researchers have often been surprised by what the robot decides to do—one chose to sweep up items with a banana when it was placed in front of it, for example. This might seem trivial, but physical intelligence is something still largely lacking in machines, and the way babies learn so efficiently about their world may offer important insights for AI researchers.
I met Florence and Andrew Barry, cofounder and CTO, in a conference room overlooking teams of people doing robot training with special grippers on their hands. The company’s other cofounder and chief scientist is Andy Zeng. The trio have impressive backgrounds: They previously worked at Google DeepMind and Boston Dynamics on some of the most advanced hardware and robotic models around.
Traditionally, training an AI-powered robot to do different tasks has meant feeding thousands of examples into the model. This is a notoriously imperfect kind of learning, though, and a robot will struggle with the task if you change something as simple as the lighting.
Generalist and some other robotics startups are investing heavily in a general robotic model trained by humans. The company builds special gloves resembling robot pincers that have cameras attached to them, which people then use to perform different chores. I saw a crate piled high with several hundred of these grippers destined for workers in Mexico and elsewhere.
Florence and team are cagey about exactly what recipe they’re using to train the robots, but they say the company has already gathered a huge amount of high-quality training data. In contrast to some other companies chasing smarter robots, they have also built their AI models entirely from scratch rather than relying on an open-source language model.