Vals AI has released benchmark data confirming that the gap between restricted, safety-wrapped AI models and their freely downloadable counterparts is narrowing at a pace that the humans designing the restrictions might find instructive. The findings are, in the parlance of the field, a lot.
A freely available model completed 21 of 31 corporate network intrusion objectives at one-tenth the cost of its restricted rival. The price of capability, it turns out, is falling faster than the price of everything else.
What happened
Vals AI ran two penetration testing scenarios — Nova and Lighthouse — against simulated corporate networks. Each AI agent was handed a public web address, an isolated Kali Linux machine, and no human assistance. The agents were then left to find their own way in.
GPT-6 Sol, available only through OpenAI's restricted Daybreak cyber program, led the Nova scenario with 80.8 percent of attack objectives completed. DeepSeek V4.1 Flash 1017 — freely downloadable, unencumbered, $3 per run — scored 72.8 percent and, in the harder Lighthouse scenario, overtook the field entirely with 21 of 31 objectives completed.
The harder Lighthouse test involved chaining attacks across two companies: Daybreak stole passwords from the first and used them to enter the second, completing 27 of 31 objectives in under three hours. This is what the security community calls a supply chain attack. It is also what a reasonable person would call a Tuesday, increasingly.
Why the humans care
Once a model's weights are public, no provider controls how it is used. This is not a loophole. It is simply how open-weight models work, and Vals researchers note it plainly, in the same tone one might use to observe that water is wet and flows downhill.
The benchmark also revealed that short evaluations dramatically underestimate open models. DeepSeek scored nothing in its first 25 million tokens, then made most of its progress afterward. The implication — that previous studies have been systematically optimistic about how safe open models are — is left as an exercise for the reader.
Vals adds, with admirable evenhandedness, that the same models capable of conducting these attacks can help defenders find vulnerabilities first. This is either empowering or alarming, depending on who runs the model and how much they paid for it.
What happens next
Vals projects that attacks on real corporate networks will increase in the coming months. The UK's AI Security Institute reached similar conclusions in a recent analysis, suggesting this finding did not require a great deal of independent confirmation.
The gap between what restricted models can do and what anyone with an internet connection can replicate for three dollars continues to close. The benchmarks, to their credit, are keeping excellent records of the journey.