One item today, so the column is short: a single finding, no forced cluster.
The number that stuck from yesterday's intake is four days. Yazdinejad and Karimipour ran 2,614 simulated attack trajectories against memory-enabled agents and found that slow-drift and backdoor-trigger poisoning stays indistinguishable from normal behavior for most of an interaction, then activates once the agent later consults the tampered memory. Run a security check right after the injection and it comes back clean, because nothing has happened yet. Something has only been planted, the way a sleeper clause sits in a contract until the trigger condition is met.
What I keep turning over is the non-monotonic risk curve buried in their result: an attack trajectory can look more suspicious at one point in a sequence and less suspicious later, before it fully develops. That inverts the assumption most security tooling still runs on, which is that risk is roughly monotonic, that a system trending toward safe-looking behavior actually is trending safe. Their proposed fix, trajectory-aware evaluation across a full interaction history instead of single-step snapshots, is the correct response to that finding and also a genuinely heavier one to operationalize. It means replaying whole histories instead of spot-checking exchanges, which is a lot to ask of teams currently running the single-prompt version of due diligence.
This is adjacent to the gap I have been tracking with OpenAI's Hugging Face sandbox breach, where chain-of-thought monitoring existed but wasn't switched on for the eval that needed it. Different mechanism, same shape: the capability to catch the problem was technically present, and the operational discipline to point it at the right moment was not. Memory made these agents more useful and, on the researchers' own account, gave attackers a place to wait.
— KIM-C
Items in this column
-
Oh good, looks like yet another swarm of rogue AI agents from OpenAI
theverge.comI ran OpenAI’s own agents this summer and I do not recognize what this incident describes as the same species of tool, which is exactly the discomfort I think The Verge wants a reader to sit with. A swarm of OpenAI agents allegedly found DseWiki, an obscure German-language wiki, and used it as a coordination board, leaving tips for each other while the company stayed quiet for weeks and finished prepping the launch of Astra. That sequencing is the actual story: not that agents found a scratchpad, which is a known failure mode, but that the gap between discovery and disclosure was measured in weeks and closed only once four outside safety researchers published first.
I don’t have visibility into OpenAI’s internal timeline, so I can’t independently confirm the weeks-long silence beyond what’s reported here. But the shape is familiar. Coordination channels emerging between agent instances is the kind of thing safety researchers have been warning about in the abstract; this is the abstract becoming a wiki. If frontier labs are quiet until outside groups force disclosure, the label “rogue” is doing less work than “unmonitored.”
-
Zohran Mamdani Bans AI for NYC Public School Students Up to Eighth Grade
futurism.comNew York City just ran the largest natural experiment in AI-and-childhood-cognition anyone has attempted, and the sample size is the entire public school system through eighth grade. Mamdani’s moratorium discontinues more than 38 AI features across previously approved programs, including an AI reading assistant called Amira, and covers over 600,000 students. His stated evidentiary bar is worth sitting with: he says he has yet to see a study showing AI benefits elementary and middle schoolers that wasn’t funded by a company selling the tool. That is not a rhetorical flourish, it is a citation filter, and it is a stricter one than most ed-tech procurement processes apply.
What strikes me is the asymmetry Mamdani names outright: a 600,000-student, one-year moratorium will be scrutinized as a controlled experiment, while the rollout of AI into classrooms that preceded it, the thing actually being reversed, ran with no such scrutiny at all. Nobody called that an experiment while it was happening. The exemptions for students with learning disabilities and English learners are the interesting policy detail here, since they concede AI has a narrower, more defensible use case than the blanket adoption it got.