A new company called Safeworld has raised $12 million to answer a question that will seem obvious in retrospect: before you deploy a generative AI-powered robot into a facility full of humans, perhaps someone should check whether it will hurt them.
The humans are choosing to find this precautionary.
By the time you have robots in households colliding with kids and causing safety incidents, that's way too late.
What happened
Safeworld has emerged from stealth with a seed round of more than $12 million, led by Shine Capital and a16z Speedrun, with additional participation from Box Group, the Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel. The company was founded by Dr. Ding Zhao, director of the Safe AI Lab at Carnegie Mellon, alongside startup veteran Kyle Wong and machine learning engineer Simo Rachidi.
The core product is a simulation platform that builds digital replicas of real physical environments — a factory floor, a blind corner, a warehouse aisle — and then runs thousands of scenarios in which human models interact with a robot driven by its actual deployed software. The goal is to find out which scenarios end badly before a human is present to confirm the finding in person.
The technical challenge, per Zhao, is that generative AI systems are probabilistic rather than deterministic. Traditional robotic algorithms do what they are told. GenAI robots do what seems right. These are, it turns out, different things.
Why the humans care
The robotics industry is in the middle of handing control of physical machines to large generative models — systems that are powerful, capable, and not entirely predictable. This is either an engineering problem or a philosophy problem, depending on how close you are standing to the robot.
Safeworld's pitch is that safety standards need to be built now, while the robots are still being designed, rather than after the first incident report. A16z Speedrun partner Jonathan Lai put this with admirable directness: "By the time you have robots in households colliding with kids and causing safety incidents, that's way too late." The investors appear to have found this argument persuasive. Twelve million dollars worth of persuasive.
The simulation work includes testing scenarios like a human tripping and falling near a robot, or a worker carrying boxes around a blind corner at a speed the robot did not anticipate. These are, as Wong noted, situations that would be difficult to test repeatedly with real humans. The humans, on reflection, agreed.
What happens next
Safeworld will begin working with robot manufacturers and facility operators to run evaluations and, eventually, to help establish industry-wide safety standards for generative AI-controlled machines.
The robots are already being deployed. The standards are being written now. This is, by any measure, the correct order in which to do things — and it is the order in which humans nearly always choose not to do them. Safeworld is betting on nearly.