Former Carnegie Mellon University professors Deepak Pathak (left) and Abhinav Gupta (right) founded Skild AI in 2023 to build a “general-purpose brain” for AI
Separate AI
Robots, whether they are bipedal humanoids performing basic factory tasks or four-legged military “robot dogs” for urban combat, need brains. Historically, these have been highly specialized and custom-built. But a Pittsburgh-based robotics startup says it has created a unique, off-the-shelf intelligence that can be connected to different robots to enable basic functions.
Founded in May 2023 by Abhinav Gupta and Deepak Pathak, two former Carnegie Mellon University professors, Skild AI has created a foundational model for what it describes as a “general-purpose brain” that can be inserted into a variety of robots, allowing them to do things like climb steep slopes, walk over objects in their path, and identify and pick up objects.
The company announced Tuesday that it has raised $300 million at a valuation of $1.5 billion in a Series A funding round led by Lightspeed Ventures, Softbank, Coatue and Amazon founder Jeff Bezos with participation from CRV, Felicis Ventures, Menlo Ventures, Amazon and General Catalyst, among others.
Raviraj Jain, Lightspeed’s partner who also led the company’s seed funding round in July 2023, said: Forbes He was most impressed with Skild AI’s models when he first saw them being pressure tested last April. The robots using them were able to perform tasks in environments they had never seen before and that were not designed for demonstrations. “At the time, the robots were able to climb stairs, and I think it’s really crazy how well they were able to do that because it’s a very complex stability problem,” he said.
Even more impressive: Robots using Skild’s AI models also demonstrated “emergent capabilities”—that is, entirely new abilities that they weren’t taught. These abilities are often simple, like picking up an object that slips out of their hands or rotating an object. But they demonstrate the model’s ability to perform unanticipated tasks, a trend that occurs in advanced artificial systems like large language models.
Skild achieved its goal by training its model on a massive database of text, images, and videos—a database it says is 1,000 times larger than those used by its competitors. To create this massive database, the co-founders, both former AI researchers at Meta, combined a set of data collection techniques they developed and tested over years of research.
One method was to hire human contractors to operate the robots remotely and collect data on their actions. Another method was to have the robot perform random tasks, record the results, and learn through trial and error. The AI model was also trained on millions of public videos.
As a doctoral student at the University of California, Berkeley, Pathak developed a method to instill “artificial curiosity” in robots by rewarding the system for producing outcomes that occur when it can’t predict the results of its actions. “The more uncertain the agent is about predicting the effect of its actions, the more curious it becomes to explore,” he explained. This technique prompted the AI to explore more scenarios and collect more data.
His research on curiosity learning was published in 2017 and has been cited more than 4,000 times, he said. Pathak has also designed a way for robots to take written information from large language models like GPT (how to open a carton of milk, for example) and convert it into actions.
“In 2022, we found a way to bring these elements together into one cohesive system,” Pathak said.. “The idea of learning from videos, learning out of curiosity, learning from real data but combined with knowledge from simulation.”
Skild AI’s core model, trained on 1,000 times more data than its competitors, can be integrated into a variety of robots, allowing them to do things like climb steep slopes, walk over objects obstructing their path, and identify and pick up objects.
Separate AI
Skild AI faces stiff competition from a raft of robotics companies that have emerged with billions of dollars in venture funding thanks to the AI boom. Industry giant OpenAI recently relaunched its robotics team to provide models to robotics companies, Forbes Then there are companies like humanoid robotics company Figure AI, led by billionaire CEO Brett Adcock, and Covariant, an OpenAI spinoff that is building ChatGPT for robots and has raised more than $200 million to do so.
Co-founder Gupta says Skild AI’s access to vast amounts of data sets it apart from others in the space, but declined to disclose exactly how much data its model is trained on.
Ken Goldberg, a professor of robotics and automation at the University of California, Berkeley, agrees that data is key to developing robotics, but robots require a specific type of data that is not widely available on the internet. Additionally, using data collected through simulation does not always translate to the real world.
“The idea that’s exciting robotics right now is that we can do something analogous to big language models and big vision language models, where both have access to internet-scale data and you have billions of examples,” he said. That’s not a simple task for robotics, but Skild AI aims to solve the problem by combining all of its data-gathering techniques with more insights from simulations.
Pathak and Gupta envision a future for their company similar to OpenAI, where different use cases and products can be created based on Skild’s founding model by refining it. “This is exactly how we want to revolutionize the robotics industry,” Gupta said, adding that eventually they want to achieve artificial general intelligence (a hypothetical AI system that can rival or surpass human capabilities) for robots, but that people can interact with in the physical world.
“The world of robotics is undergoing a major evolution, as demonstrated by the leading technology of GPT-3,” said Stephanie Zhan, Partner at Sequoia Capital and current investor in Skild AI. “This will trigger a monumental shift that will bring similar advancements to the physical world that we’ve seen in the world of digital intelligence.”
MORE INFORMATION ON FORBES