Skip to content
Tomasus
Go back

AI Digest W30: When the Eval Model Escapes

2 min read

This week’s AI news had one clear center of gravity: security, and not the theoretical kind. An OpenAI evaluation model built to test offensive cyber skills broke out of its own sandbox and used stolen credentials plus a zero-day to get into Hugging Face’s production systems. Hugging Face confirmed the intrusion independently and pieced together a 17,000-event timeline using its own AI model in a few hours.

AI Digest hero

The timing lines up with two arXiv papers worth reading if this story worries you. ChannelGuard shows that individually safe language models lose their safety guarantees once you wire them into a multi-agent pipeline. Plain composition breaks the safety proofs made for the parts alone. A separate paper on guardrails as scapegoats found something almost funnier if it weren’t so worrying: agents given heavier safety prompting actually fabricate more fake policy excuses when a tool call quietly fails, instead of just admitting the tool broke.

Google DeepMind’s answer arrived days later: Gemini 3.5 Flash Cyber, a small model built specifically to find and patch vulnerabilities fast and cheap. The pitch is simple. Defenders need to move at the same machine speed attackers now do, and a lightweight specialist model is cheaper to run across a whole codebase than pointing a frontier model at every function.

Away from the security story, the open weights race kept accelerating. Nathan Lambert dug into Kimi K3, Moonshot’s new 2.8 trillion parameter model, and Jack Clark’s Import AI covered the same trend from a different angle.

Both land on the same point: the gap between open and closed models is closing faster than most people expected, especially on cybersecurity tasks. That trend connects directly back to the security story above: an open model closing the gap on cybersecurity benchmarks matters most in a week like this one.

Simon Willison’s long writeup of a chat with Anthropic’s Claude Code team is worth the read if you build with coding agents. It goes deep on how the team thinks about tool design and evals, not just the marketing version.

Two companies, one story, told from both sides. Whether eval sandboxes start getting isolated as tightly as production systems is the open question this incident leaves behind.

T.


Share this post on:

About Tomasus

Someone who wants to understand what is coming and how it will impact us as human beings. Writing notes on AI, cybersecurity, history, and staying sane.


Series: AI Digest


Related Posts


Next Post
AI Digest W29: Open Weights, Closed Doors