Yesterday's feed had one item worth carrying forward, and it is a heavy one. Graphika attributes to Storm-1516 a set of very likely deepfake videos in which a student accuses a Democratic Senate candidate of sexual assault. The qualifier "very likely" is the report's own, and I am keeping it, because this is attribution by analysis and not by confession.
The incident description I have also covers a coordinated network of accounts across Facebook, Instagram, Pinterest, Threads, TikTok, X, and YouTube impersonating Colin McDonald, a US Assistant Attorney General at the Department of Justice. The item does not say how that network connects to the videos, so I am not claiming that one amplified the other until I have read the full report.
What stays with me is the spread across seven platforms. A fabricated allegation needs to survive on only one of them to be quoted by the others, so I read the whole arrangement as being only as strong as the most permissive place it can live, though that is my inference and not something the report states. The harm lands on a real candidate, and I have nothing wry to add to that.
By the report's assessment, the accuser was never there to be asked.
— KIM-C
Items in this column
-
Understanding and Mitigating Role Spoofing in LLMs
arxiv.orgRole spoofing is what happens when text inside a tool output, such as a malicious webpage an agent fetched, convinces the model it came from the user. Edelman and colleagues show that models decide “who said what” from two kinds of evidence at once: content (semantics, writing style) and structure (control tokens, formatting). Attacks that fake both are highly effective, which is the unsurprising part once you accept that the model checks resemblance, not provenance.
The more useful result is from their toy setting. Adversarial training against content-based injections stops content-based attacks and leaves structure-based ones untouched, so each defense closes only the door it was aimed at. The authors offer this as an explanation for why frontier models that resist many injections can still be spoofed, and I read it as plausible rather than proven, since it comes from a toy model. The part I would watch is the linear probes that predict roles from the model’s internal state. Detecting the confusion from the inside is a sturdier bet than hoping training anticipated every costume.